Turns a written task into clicks and keystrokes on your desktop

You type an instruction like "export this spreadsheet as HTML and open it in Chrome". The model looks at a live screenshot of your desktop, reasons about what has to happen next, and answers with a concrete mouse or keyboard action. A small runner executes that action, takes a fresh screenshot, and the loop repeats until the task is complete. Because it reads the actual pixels on screen, it is not tied to one fixed layout: when a dialog pops up or a menu moves, it re-plans from what it sees.

It handles long jobs that span several applications, works on both Ubuntu and Windows, and posts strong scores on the standard obstacle courses for desktop agents. Tencent ships it as a 27B model with weights free for commercial use, alongside a lighter 9B version and a "democua" variant that adapts a workflow from a single recorded demonstration you show it.

There is no hosted demo, so this one is for readers comfortable running models. Download the weights from the Hugging Face page, serve them with vLLM (a standard tool for hosting models; Tencent recommends two GPUs at this size), and run the example agent from the UI-Mate GitHub repository. For a first test, hand it a screenshot of a real spreadsheet with the instruction "export this as HTML and open it in Chrome", then read the reasoning it prints before each click: you can see exactly why it chose every step.

This is an agent checkpoint, not a chat model: it needs Tencent's harness to parse and execute its actions, plus serious GPU hardware to serve. And like every computer-use agent it can misclick or be misled by what appears on screen, so run it in a disposable environment, not on the desktop where your real work lives.

• Hand off a repetitive back-office flow, like copying records from a spreadsheet into another program, and watch it click through.
• Let it drive software you rarely use: describe the outcome and let it find the menus.
• Automate multi-step exports and file chores that scripts cannot reach because they live behind GUI dialogs.
• Build a screen-driven assistant into your own product; the license allows commercial use.
• Read its step-by-step reasoning to learn how a computer-use agent decides where to click.

Try the demo

View model page

Read this issue on neuralexpedition.com →