Should I use Jev for summaries and writing?
Verdict for the sample idea
Run it on
a reasoning LLM
Every case needs generated text, planning or multi-step reasoning.
Sample idea
Summarise each customer call transcript into five bullet points and write a follow-up email to the customer in a friendly tone.
Explanation
Jev does not generate text. It is not bad at it. The model has no mechanism for it. It returns choices, levels and probabilities. A summary, an email, a code comment, a translation or a product description is generated text, and you need an LLM. No question design changes that.
If you evaluate Jev because your LLM bill is high, the useful question is different. Which share of your LLM calls are decisions that only look like writing? Many pipelines ask an LLM "should we follow up on this call?" and parse a yes from the prose. That part can move to Jev. Your saving is that share of the spend, and not more.
Jev is useful on both sides of the writer.
Before the LLM, it decides if the work is needed at all. Does this transcript contain a customer commitment? Is the call longer than small talk? Which of four email templates fits? Calls that need no follow-up never reach the LLM, and the rest reach it with a label that makes the prompt shorter.
After the LLM, it checks the result. Does the email promise something that the transcript does not contain? Is the tone friendly? Does it mention a competitor? These are Noul questions with the transcript and the draft as state. They are cheap enough to run on every output, and an LLM judge usually is not.
The common trap is to approximate writing with choices. You can build a "summary" from a Choice over 40 canned sentences. It works in a demo. Then a customer says something that is not in your list. If the output must reflect content that you cannot list in advance, it is generation.
Jev is the referee. It does not play.
Related tasks
Verdict produced by Jev on 2026-09-19, rubric version 2. Text written by a person. How the tool works: the report.