A lot of the software businesses depend on has no API and never will: admin panels, old portals, internal tools. Agents that can use a computer exist, but they work one step and one inference at a time, which is slow and expensive for a task that needs doing five hundred times.
Crewmate splits the learning from the doing. A small Electron window records your screen while you do the task once. The recording is cut into about 90 stills and read by a vision model, which returns a plan: ordered steps, the variables that change from row to row, any conditions, and a list of the dead ends it left out and why.
Steps never store coordinates. Each one names the control it acts on, and at run time every sandbox finds that control in the live accessibility tree through Daytona's Computer Use API. The model runs once, when it reads the recording, rather than at every click, so a ten step task finishes in about 13 seconds and does the same thing every time.
Each sandbox boots from a snapshot that was logged in by hand, so Crewmate never handles a password. A row whose condition does not apply finishes as skipped rather than failed, and sandboxes are thrown away when a run ends, including when it crashes.
It was built by three coding agents working in parallel against a frozen schema, each owning its own folders. The full story of the day, and why it did not win, is in My first hackathon.