Advanced setups

None of this is needed to use Deskfish. It is here for people who like terminals, run things on servers, or want to try the agent without VS Code.

Peeking at the screen#

The tank serves its current screen as a plain image. Open http://localhost:9990/screenshot.png in any browser, or:

curl -s http://127.0.0.1:9990/screenshot.png -o screen.png

The tank from a terminal#

scripts/desktop.sh in the source tree does what the power button does, with plain podman run or docker run:

scripts/desktop.sh up            # build the image if missing, then start
scripts/desktop.sh up --build    # rebuild the image first
scripts/desktop.sh down          # stop and remove the container (the home volume is kept)
scripts/desktop.sh status
scripts/desktop.sh logs
scripts/desktop.sh shell         # a shell inside the tank, as the bot user
scripts/desktop.sh screenshot

It reads these environment variables:

The container itself understands SCREEN, DAEMON_TOKEN, VNC_PASSWORD, DAEMON_BIND, DAEMON_PORT and DISPLAY_NUM.

A gateway on another machine#

Deskfish is one program, the gateway, and it does not have to run on your laptop. Put it on a machine that stays awake — an old laptop in a cupboard, a mini PC, a rented server — and her schedules run whether or not you are at your desk, and an interrupted task is picked up when the machine comes back.

What follows is the whole recipe for a fresh Ubuntu 24.04 box with systemd. Other Linux distributions differ only in the package names.

1. Podman and Node#

sudo apt update
sudo apt install -y podman passt

passt is what lets rootless Podman give the tank a network of its own; without it Deskfish still works but shares the machine's network namespace.

Ubuntu 24.04 ships Node.js 18, which is too old — Deskfish is built for Node 20 or newer. Take Node 22 from NodeSource:

curl -fsSL https://deb.nodesource.com/setup_22.x | sudo -E bash -
sudo apt install -y nodejs

2. Deskfish#

sudo npm i -g https://github.com/0x11c11e/deskfish/releases/latest/download/deskfish.tgz
deskfish --help

That link always serves the newest build. Run the gateway once by hand so it creates her data folder and her token, then stop it with Ctrl+C:

deskfish serve

deskfish status (in another shell, while it runs) prints the web address and the path of the token file. Keep that token: it is what every window will sign in with.

3. Keep it running#

A user service, so it starts at boot and comes back after a crash. Write ~/.config/systemd/user/deskfish.service:

[Unit]
Description=Deskfish gateway
After=network-online.target

[Service]
ExecStart=/usr/local/bin/deskfish serve
Restart=on-failure
RestartSec=5

[Install]
WantedBy=default.target

Then:

systemctl --user enable --now deskfish
loginctl enable-linger $USER

enable-linger is the important one: without it the service stops when you log out of the SSH session. Her own log is ~/.local/share/deskfish/logs/gateway.log; if the service will not start at all, journalctl --user -u deskfish says why.

4. Reach it from your laptop#

With an SSH tunnel. Nothing is opened to the network; SSH carries everything:

ssh -L 9980:127.0.0.1:9980 my-box

If your laptop also runs Deskfish, port 9980 is already taken there, so use another local port:

ssh -L 9981:127.0.0.1:9980 my-box

While the tunnel is up:

• In a browser, open http://127.0.0.1:9981/?token=<the token> once. The page keeps the token, so after that http://127.0.0.1:9981/ is enough.
• In VS Code, set deskfish.gateway.placement to remote and deskfish.gateway.url to http://127.0.0.1:9981, reload the window, and paste the token when it asks (or run Deskfish: Set Gateway Token).
• In the app: not yet. The app always runs or finds a gateway on the machine it is on; a field for another machine is still to come. Use the page or VS Code for this.

With Tailscale. A private network between your own machines, with no port open to the internet and no tunnel to remember. (Tailscale is made by Tailscale Inc., Toronto, Canada; the gateway token travels through it, which is why the origin is worth naming.) Install it on both machines, then on the box:

deskfish serve --host 100.x.y.z --allow-remote

with its Tailscale address, and use http://100.x.y.z:9980 as the address in the browser, in deskfish.gateway.url, or on your phone. In the systemd unit, put the same flags on ExecStart.

What lives on that machine#

Everything of hers except the screenshots:

• the model API key (secrets.json, readable only by that user) and the gateway token;
• her facts, the page about who she is and its history, her journal, her playbooks, the charter;
• every chat transcript (text only), config.json and state.json;
• the tank's volume, with the browser profile and whatever websites it is logged into.

Screenshots are never written to disk. They go to the model and to the windows watching, and then they are gone.

On a machine you rent, the disk belongs to someone else. Two things follow. Choose encryption at rest, or put her data folder on an encrypted volume. And keep the authoritative copy of her memory at home: export it from the server's page and import it on your own machine now and then, and the other way around after a stretch of work on the server. It is a manual step today — see Memory.

Headless runs without VS Code#

The deskfish command is the whole program without a window. Install it as above, and:

deskfish serve                 # the gateway itself; Ctrl+C stops it
deskfish status                # what is running, on what model, and where the token file is
deskfish run "open wikipedia.org and tell me when Debian 1.1 was released"
deskfish stop                  # stop the gateway (the tank keeps running)
deskfish mcp                   # an MCP server on stdio (below)

deskfish run prints her replies and her actions as they happen. Ctrl+C detaches and leaves the task running — deskfish status shows it, and any window that connects later picks it up mid-task with the whole chat.

All of them take --data-dir DIR and --port N when the gateway is not the default one; serve also takes --host and --allow-remote. DESKFISH_HOME moves her data folder, which is ~/.local/share/deskfish on Linux, ~/Library/Application Support/deskfish on macOS and %APPDATA%\deskfish on Windows. Only one gateway may run per data folder; a second serve on the same folder exits rather than compete for her files.

Settings are hers, in config.json in that folder, and the way to change them without a window is the web page the gateway serves (deskfish status prints the address), or editing that file before a start.

Deskfish as an MCP server#

A coding agent — Claude Code, Codex, anything that speaks MCP — can talk to Deskfish the way you do: give her a task, watch it happen, look at her screen, read what she has written, and tell her what to do differently. Two uses, one door:

• Errands. An agent working in your repository has no browser and no hands. Hers are next door: "log in to the dashboard and download last month's invoice" goes to her tank, where the browser, the logins and the Downloads folder are.
• Teaching. Give her a task from a session that can judge the result, then tell her in the same chat what was good and what to do differently. She keeps what she keeps: her playbooks and her reflection are hers, and nothing here writes a file of hers.

deskfish mcp is that door. It speaks MCP over stdin and stdout and is a client of the gateway — the same port and the same token as a window. It never starts a gateway; if none answers it says so and exits, so start Deskfish first (VS Code, the app, or deskfish serve).

Registering it#

Once, outside any project:

# installed from the tarball (npm i -g … deskfish.tgz)
claude mcp add --scope user deskfish -- deskfish mcp

# from a source checkout
claude mcp add --scope user deskfish -- node /path/to/deskfish/dist/cli.js mcp

# the app, which brings its own Node inside Electron: the .deb installs under /opt/Deskfish
claude mcp add --scope user deskfish -- env ELECTRON_RUN_AS_NODE=1 /opt/Deskfish/deskfish /opt/Deskfish/resources/deskfish/dist/cli.js mcp

# the AppImage mounts itself somewhere new on every start, so unpack it once and use the copy
./Deskfish-linux-x86_64.AppImage --appimage-extract
claude mcp add --scope user deskfish -- env ELECTRON_RUN_AS_NODE=1 $PWD/squashfs-root/deskfish $PWD/squashfs-root/resources/deskfish/dist/cli.js mcp

claude mcp list should then say Connected, and the tools appear as mcp__deskfish__run, mcp__deskfish__wait and so on. Codex takes the same command:

codex mcp add deskfish -- deskfish mcp

or, by hand in ~/.codex/config.toml:

[mcp_servers.deskfish]
command = "deskfish"
args = ["mcp"]

Add --port N and --data-dir DIR when the gateway is not the default one. For a Deskfish on another machine, use --url http://host:port and put its token in DESKFISH_GATEWAY_TOKEN — over a tunnel or Tailscale, as in A gateway on another machine above. The token is the same one the web page asks for.

The tools#

Nothing in that list writes a file of hers, sets a key or changes a setting; those stay with the windows. An agent using it is a person in her chat as far as she is concerned: what it says is in the transcript, the run is journaled with what put it there, and Stop in any window still stops her.

One URL for many models#

docker/docker-compose.yml starts the tank together with a LiteLLM proxy on 127.0.0.1:4000, configured by docker/litellm.config.yaml. Each entry in that file is a model name the extension can use, backed by xAI, Anthropic, OpenAI, or an Ollama running on the host. Keys go in docker/.env. Then set the provider to openai-compatible, the base URL to http://localhost:4000/v1, and the model to one of the configured names.

The compose file is a convenience for this setup only. The power button in the sidebar does not use compose.

Other desktop images#

The tank's control API speaks the same protocol as Bytebot's desktop daemon, so a container built from that project's image can stand in for Deskfish's own, within limits: only the basic mouse, keyboard and screenshot actions work there. There is no page bridge, so find, read_page and click_element fail; no run_command; no release of stuck keys; and none of the fixes that let the agent type line breaks and non-ASCII text. Deskfish's own image is the default, the smallest, and the only one Deskfish vets. Before pointing Deskfish at any other image, remember that it will hold the agent's browser logins: check who builds it and where it comes from as carefully as you would for a password manager.