product

Direct agents with visual prompts in Design Mode

Erik Nilsson, Ian Huang & Ryo Lu6 min read

Chat is one interface for working with agents, but UI work tends to be spatial. Designers, PMs, and frontend developers often communicate through annotations that point to elements, regions, or the state of the page.

Design Mode, which we're updating today, is part of how we're shrinking the distance between what you see and what the agent understands. It lets you edit your product in context while staying in flow.

From the Cursor browser, you can click any element, draw on the page, or describe the change by voice, and Cursor gets the context it needs to edit the code while you move on to the next edit.

It is a faster, easier way to iterate on design changes with agents because the instruction is no longer just a sentence—instead it can include the selected element, the code behind it, the surrounding layout, and the visual relationships on the page.

This makes the loop between noticing and editing tighter. You can point at the part of the interface you mean without leaving the running product, then keep making references against the product itself while agents make the edits underneath.

Point, draw, or narrate the change

Design Mode provides several ways for you to convey intent to the agent. You can select an element, add multiple references, draw over the interface, or use your voice to describe the change.

Click an element in the running app, prompt against that selected element, and let the agent edit the code.

Multi-select is useful when the change depends on a relationship between elements. You can reference two components and ask the agent to make one match the other, remove repeated content, or adjust a group of components together.

Select multiple elements and describe how they should change together.

Drawing is useful when the agent needs to know what area of the page the instruction applies to. You can circle a crowded section, box in a region, or mark part of an animated page. The annotation sits over a frozen frame of the viewport, so the agent sees the exact page state you were responding to.

In this release, you can also narrate instructions using your voice, and we've made targeting more precise and the experience faster. Altogether, this makes visual interactions in Design Mode feel more like part of a normal editing loop.

Use voice input and drawing together to describe a change.

Under the hood, picking an element adds two complementary signals into context: the element's identity (xpath, the component, attributes, computed styles, props from the fiber tree) and a screenshot for spatial context (layout, surrounding elements, and the exact page state). This gives the agent exactly what it needs to find the source and edit the code efficiently.

Matching the model to the rhythm of the work

When you are refining a user interface, one chain of edits usually leads to the next. You adjust a component, notice the spacing around it, then see how another component should match.

Design Mode lets you send those edits away as you notice them. You can point at one element, describe the change, move to another part of the page, and send another edit before the first one has finished. Design Mode allows you to multitask more easily and makes managing multiple subagents possible.

This flow works best with a model that can make targeted UI changes quickly. Composer 2.5 excels at this because it is both fast and strong at interface work. As agents finish, the app hot reloads. You see the changes appear in the running product and keep going until the interface feels right.

We believe the future of building software lets users move seamlessly between higher levels of abstraction and lower-level details while working in flow state when they want to. Design Mode provides users with the control, agency, and precision editing tools that make that possible.

Try Design Mode in the Agents Window. Read the Browser docs to learn more, or download Cursor to get started.

Filed under: product

Authors: Erik Nilsson, Ian Huang & Ryo Lu