Adds a deployment-wide setting to select the computer-use provider
(Anthropic or OpenAI) for AI agents, plus the OpenAI computer-use runner
needed to honor that selection.
The setting is stored in `site_configs` under
`agents_computer_use_provider`, defaults to Anthropic when unset, and is
exposed via experimental GET/PUT endpoints under
`/api/experimental/chats/config/computer-use-provider`. The chatd
computer-use tool now dispatches to either `runAnthropicComputerUse` or
`runOpenAIComputerUse` based on the resolved provider, with
provider-specific result metadata for OpenAI screenshots.
Frontend adds a provider dropdown to the Agents Experiments settings
page nested under the virtual desktop toggle, with disabled state
handling while virtual desktop is off and skeleton loaders while config
queries are in flight.
Hugo and Codex review follow-up:
- Uses shared provider validation and clearer computer-use constant
names.
- Removes stale OpenAI pending-safety-checks commentary.
- Documents why provider result metadata is needed for OpenAI
screenshots.
- Keeps the computer-use subagent visible when provider credentials are
missing, then returns a clear spawn-time configuration error.
- Uses OpenAI's recommended 1600x900 screenshot geometry to preserve the
native 16:9 aspect ratio.
- Moves OpenAI-specific computer-use helpers into
`coderd/x/chatd/chatopenai/computeruse` after rebasing onto the provider
package refactor in `main`.
- Converts OpenAI pixel scroll deltas to Coder desktop wheel-click
amounts.
- Preserves OpenAI pointer modifiers with key down/up desktop actions
and rejects unsupported non-left double-click buttons explicitly.
- Maps OpenAI back/forward side-button clicks to browser navigation key
actions.
- Defaults omitted OpenAI click buttons to left-click.
- Retries mouse release cleanup if the final OpenAI drag release fails.
- Keeps computer-use subagent availability messages stable when provider
config cannot be loaded, while logging the backend error.
- Releases remaining OpenAI modifier keys if a synthetic key-up cleanup
action fails.
- Updates Storybook interaction stories so provider snapshots show the
selected final provider.
> Mux updated this PR description on behalf of Mike.
Extracts OpenAI-specific logic from `coderd/x/chatd` into
`coderd/x/chatd/chatopenai` so the main chat path no longer references
`fantasyopenai` directly for chain mode info, response IDs, web search
tooling, or option mapping.
Structural refactor. The only deliberate behavioral narrowing is
consolidating Responses store checks and related keyed option or
metadata access on `opts[fantasyopenai.Name]`. That is documented by
`TestIsResponsesStoreEnabledIgnoresMalformedNonOpenAIKey` and is
unreachable in production where Responses options always live under
`fantasyopenai.Name`.
Summary:
- Moves OpenAI Responses chain mode info, response ID helpers, web
search tool construction, and provider option conversion into
`chatopenai`.
- Keeps Anthropic, Google, OpenRouter, and Vercel provider branches as
thin, existing code paths.
- `chatopenai` only imports `chatprompt` from chatd subpackages. It does
not import `chatd`, `chatloop`, `chatprovider`, or `chaterror`.
- Follow-up review fixes align helper names, keyed provider option
access, map cloning behavior, and PR documentation with the extracted
package boundary.
- Final sweep trims unused chain-mode state, removes a duplicate
store-check test case, drops an unused provider-tool parameter, and
shares the chat-message test helper through `chattest`.
> Mux is updating this PR on Mike's behalf.
Michael Suchacz