Task guide

Should I use Jev for RAG relevance filtering?

Verdict for the sample idea

Sample idea

Our search returns 50 passages for each user question. Before we send them to the LLM, decide for each passage if it helps to answer the question, and keep only the useful ones. About 5,000 questions a day.

Explanation

This is a filter between retrieval and generation. For each passage, the question is a yes or no with a probability: does this passage help to answer the user's question? That is one judgment per item, on text, at high volume. 5,000 questions with 50 passages each is 250,000 judgments a day. At Jev's price that is small money. With an LLM it is a real bill and a real delay.

Shape it as one Noul per passage. The state holds the user question and one passage. Do not put all 50 passages in one state and ask "which are relevant". TypeSafe lists large irrelevant state as a weakness. One passage per question keeps each judgment clean, and the calls run in parallel.

Define "relevant" in the criteria. A passage that mentions the same product is not enough. A passage that contains the fact needed for the answer is. Write the false case too: same topic, no answer.

Then let code do the rest. Sort by probability, keep the top few, and drop everything under a threshold that you fitted on labelled pairs.

The common trap is to expect Jev to replace retrieval. It reads what you give it. It does not search. Use Jev as the second stage after a cheap first stage, and measure answer quality with the filter on and off.

A second trap is to filter so hard that the LLM receives nothing. Keep a minimum number of passages, even when all probabilities are low, and log those cases. They show you where retrieval fails.

Related tasks

• Should I use Jev for claim and citation verification?
• Should I use Jev to pick the next UI action?
• Should I use Jev for summaries and writing?

Verdict produced by Jev on 2026-09-19, rubric version 2. Text written by a person. How the tool works: the report.