Back to skills

Agent Skill

Computer Use

computer-use

Drives the GUI of a visible local app window through `orca computer`: accessibility tree, clicks, typing, menus, dialogs, and screenshots in native apps and external browser windows (Chrome, Edge, Safari) or webviews. Prefer a programmatic path (shell, filesystem, git, HTTP, existing CLIs) whenever it can complete the task. Use only when a visible window needs GUI control those cannot reach. Do not use for Orca's embedded browser (`orca-cli`).

StablyaiToolsTypeScriptAdeAgent-ideAi-agentsClaude-codeCliCodexCursor-agentDevtoolsGhosttyIdeMobile-appOpencodeOrchestrationParallel-agentsPiTerminalWorktreesYc-backed
github.com/stablyai/orca/tree/main/skills/computer-use

177K installs

•

stablyai/orca

•

by Stablyai

Score

8.9

/ 10

Installs

177K

Repo Stars

79.1K

Last Updated

0d ago

Fresh

Quality Ratio

98%

Description

Verified

Language

TypeScript

First Published

May 2026

Summary

The Computer Use agent skill enables AI agents to inspect and operate local desktop application windows, providing capabilities to read UI states, click controls, type text, and perform other actions within desktop apps. This is particularly valuable for developers designing agents that need to automate tasks across various desktop applications, including browsers for web apps, chat clients, and other GUI-driven software. This agent skill is a skill with 4K installs in the registry. It leverages the `orca computer` CLI, allowing agents to execute commands like `list-apps` to identify available applications or `get-app-state` to retrieve an app's UI tree and screenshot. Agents can then use commands such as `click`, `set-value`, `type-text`, or `press-key` to interact with specific UI elements identified by their indexes. The skill provides guidance to use current UI state for reliable element interactions and strictly advises against sensitive actions like sending messages or changing account settings without explicit user permission. Developers should be aware that UI element indexes are short-lived, necessitating frequent state refreshes, and that some actions may depend on the application's focus or system permissions.

Skill Definition

This discovery stub loads the version-matched guide from the Orca executable used for this session.

Resolve the CLI for this session

Choose the executable once and reuse it for every later command:

  • If the ORCA_CLI_COMMAND environment variable is set, use its value. Orca exports this for managed WSL sessions.
  • Otherwise, in a dev checkout whose session exposes ORCA_DEV_REPO_ROOT, use orca-dev.
  • Otherwise, on Linux outside an Orca-managed terminal, use orca-ide. Never run bare orca there — outside Orca's terminals it normally resolves to the GNOME Orca screen reader (/usr/bin/orca) and starts speech on the user's machine.
  • Otherwise, use orca.

Below, ORCA is a placeholder for the executable you resolved. Substitute it before running anything; do not create a shell variable or run ORCA literally. This works the same way in POSIX shells, PowerShell, and cmd.exe.

If the selected executable cannot run, report its exact error and stop. Do not fall through to another executable, which could silently target a different Orca build.

Load the version-matched guide before running Orca commands

ORCA skills get computer-use

Prefer --json. Use the selected executable's --help for commands or flags the guide does not cover. If a command reports that Orca is not running, start it with ORCA open --json and retry. If it fails with runtime_access_denied, your sandbox blocked the connection: re-run it with escalated permissions, and do not run ORCA open or restart Orca. If skills get is unknown, explain that updating Orca restores the guide; use --help for read-only discovery and do not guess unsupported commands.

How to Use

Use in O-mega

Claude Code

npx skills add stablyai/orca computer-use