Task guide

Should I use Jev to pick the next UI action?

Verdict for the sample idea

Sample idea

Inside an automated UI test loop, code gives one step goal at a time, such as "open the settings screen", plus the accessibility tree of the current screen as a list of elements. Pick which element to tap next. One decision per step, thousands of steps a day. When no element fits, say so and a slower model plans again.

Explanation

This is the use that the browser and computer-use projects made popular, and it fits for a simple reason. The accessibility tree is already a list of options. "Which element moves us toward the goal?" is a Choice over that list. Jev takes no images, so the tree is the correct input and a screenshot is not.

I am an iOS developer, and the model I keep in mind is hitTest with meaning. The input is a goal phrase and a view tree. The output is the view to touch.

Shape each step as one call with three questions. A Choice picks the element. A Noul asks if the goal is already reached. A Noul asks if an unexpected dialog is on the screen. They run in parallel, so the extra two are almost free. The LLM writes the goal phrase once and takes control only when confidence is low.

Filter the tree in code first. Send only elements that are visible and interactive, with a role, a label and a short id. A Choice accepts up to 255 options. A long settings screen can pass that limit, and irrelevant elements make the judgment worse even under it.

The common trap is to let Jev do the planning. Jev picks one step from one screen. It does not remember that it opened the wrong menu two steps ago. Keep the step history in code, detect loops in code, and give the LLM the control when the same screen appears three times.

Do not expect the vendor's headline speed numbers for a whole step. Reading the tree and performing the tap still take time, and Jev only removes the thinking part.

Related tasks

• Can Jev work with screenshots and images?
• Should I use Jev to check shell command safety?
• Should I use Jev for planning and scheduling?

Verdict produced by Jev on 2026-09-19, rubric version 2. Text written by a person. How the tool works: the report.