Models and providers

Deskfish talks to the model through a thin adapter, and there is one adapter per kind of API. Choose with deskfish.provider.

Anthropic#

deskfish.provider = anthropic, deskfish.model = claude-opus-5 (the default), deskfish.baseUrl empty. Set the key with Deskfish: Set LLM API Key.

This adapter uses Claude's native computer-use tool, the one the models were trained on, so clicks are the most accurate of all the options. It also uses adaptive thinking, and it caches the system prompt so repeated steps cost less. If Claude refuses a task, the refusal and its explanation appear in the chat and the task ends.

Any Claude model with computer use works; <your-computer-use-model> is the strongest, and claude-sonnet-5 is a good cheaper choice for simple tasks. If you use a gateway in front of the Anthropic API, put its URL in deskfish.baseUrl.

OpenAI-compatible endpoints#

deskfish.provider = openai-compatible, deskfish.baseUrl set to the endpoint, and a model name that supports vision and tool calling. This one adapter covers a lot of ground:

Models here are shown a generic computer tool whose actions mirror the vocabulary Claude uses, plus zoom, ask_user and read_docs, so instructions and habits transfer. Accuracy depends entirely on the model. Large hosted models do well; small local ones can find the right button but misclick often, and the zoom tool exists partly for them.

When the endpoint is on localhost no key is required and the key chip says Local, no key.

A note on OpenRouter#

OpenRouter is the easiest way to try many models with one key, and it works with this adapter (https://openrouter.ai/api/v1, a model name such as anthropic/claude-opus-5, google/gemini-2.5-pro or moonshotai/kimi-k3). It is also the way to use models from companies you would rather not hold an account with: your key and your payment stay with OpenRouter, and OpenRouter lets you choose which hosts may serve a model. Three things to know before choosing it over the direct path:

• Clicks are less precise. Through OpenRouter, Claude is driven with generic tool calls and screenshots, not its native computer-use tool. It works, but the direct Anthropic path is the one that hits small targets reliably; the zoom tool helps.
• Caching works for Anthropic and Gemini models. Deskfish marks the conversation with cache breakpoints when the endpoint is OpenRouter, so the prefix is cached the same way as on the direct path; the status row shows the cached share. Other models cache or not on their own (OpenAI and xAI do it automatically). OpenRouter's fee still applies.
• Only vision-and-tools models can drive a desktop. Check both boxes on OpenRouter's model page before picking one.

Anthropic direct is the recommended path; OpenRouter is for every other model. It has not yet been tested as thoroughly as the direct path. For other gateways that pass Anthropic's cache markers through, such as a LiteLLM proxy, set deskfish.promptCaching to on; for a strict endpoint that rejects unknown fields, off.

The demo model#

deskfish.provider = mock. A scripted model that needs no key and no internet: it opens the browser, goes to the site named in your task (or example.com), knocks on the glass once so you can see the hand-over, looks around with a zoom and a scroll, and declares itself done. Use it to check that the tank, the Desktop tab and the chat all work before adding a real model.

Keys#

The API key is stored in your operating system's keychain through VS Code's secret storage, never in a settings file, and it is sent only to the endpoint you configured. Enter it with Deskfish: Set LLM API Key or the Change button next to API key; leave the box empty to clear it. Keys are kept one per provider (Anthropic, and one per endpoint host such as openrouter.ai or api.x.ai), so switching providers does not lose the other key; the model picker asks for a key only when the chosen provider has none.

What a task costs#

Each step sends the model the task, the conversation so far, the newest screenshot (a JPEG of about 1280 × 800 pixels) and the results of its last actions. The status row shows the running token count, input and output, for the whole conversation. Three things keep the bill in check:

• older images are pruned in batches, keeping the most recent three after a prune;
• an optional step limit, deskfish.maxSteps, with no limit by default;
• on Anthropic, prompt caching of the whole conversation: each step re-sends everything so far, but everything before the newest message is read from the cache at a tenth of the price;
• on Anthropic, server-side compaction: when a conversation grows very long the API summarizes its older part, so a task never dies at the context window.

The counter in the status row shows the total input, the output, the share that came from the cache, and, for Claude models, a cost estimate at list prices. A long task can process a million tokens or more in total; with caching, most of that is the cheap kind. Local models cost nothing but time.

Through OpenRouter the status row shows the exact charge instead of an estimate: OpenRouter reports the cost of every request, and Deskfish adds them up. deskfish.maxCostUsd works there too. Other OpenAI-compatible endpoints report tokens only, so no figure is shown.

Changing models#

Click Change next to Model in the sidebar, or run Deskfish: Choose Model…: pick the provider, then the model, and enter the key if that provider has none yet. The settings behind it can also be edited by hand. The next task uses the new model and starts a fresh conversation, since another model cannot continue this one's memory. Changing the model while a task is running does not affect the running task.

Running several providers behind one URL#

The docker/ folder in the source tree contains a compose file that starts the tank next to a LiteLLM proxy, which exposes xAI, Anthropic, OpenAI and a local Ollama under one OpenAI-compatible URL with model names of your choosing. It is optional, and aimed at people who like to switch models by name. See Advanced setups.