Benchmark · Parieto Research
A Benchmark for Repository Intelligence
Across 848 repository-search cases, Parieto outperformed Claude Code, Codex, and Mini-SWE-Agent on the retrieval measures that most directly shape an AI product: reaching the right file, returning answer-bearing lines, balancing signal and coverage, and keeping the context budget compact.
The practical result is simple. The model starts closer to the answer, has more room to reason, and spends fewer tokens getting there.
What better retrieval changes
The right file, more often
A relevant file appears in the returned regions in more than 95% of evaluated cases, so the answer model begins in the right part of the repository.
Better context for the answer model
Leading F1 combines useful precision with stronger line coverage, giving the model more of the code it needs without burying it in unrelated context.
Far fewer tokens per answer
A compact evidence set leaves more context for reasoning and lowers the cost of repeated searches across large codebases.
The comparison
Parieto leads the supplied comparison on HitFile, precision, LineRecall, F1, and tokens to answer.
How we built the benchmark
Each measure tests a different part of the path from a repository question to answer-ready code.
The evaluation covers 848 repository-search cases. For each case, a system retrieves regions from a codebase and those regions are checked against the files and lines needed to answer the question.
Together, the measures capture whether search reaches the right file, how much useful code it returns, how much answer-bearing code it covers, how the result is ordered, and how much context the answer model must read.
Retrieval quality, made visible
Parieto reaches answer-bearing code more consistently while keeping the resulting context compact.
HitFile
Share of cases that reach a gold fileFigure 01 · Reaching the right file gives the answer model a dependable starting point.
F1
Combined precision and LineRecallCompared with approximately 200k to 630k tokens across the external systems in the supplied evaluation.
Figure 02 · Stronger retrieval gives the model more answer-ready context with far fewer tokens.
What each metric changes in the product
The benchmark becomes useful when each number maps to a better answer experience.
HitFile
How often search reaches a file that contains the answer.
Find the right place first.Precision
How much returned code directly contributes to the answer.
Give the model less noise.LineRecall
How much answer-bearing code reaches the model.
Return enough to finish the task.nDCG@500
How relevant code is ordered within the result.
Put useful evidence near the top.F1
How well retrieval balances signal quality and coverage.
Improve the working context.Tokens to Answer
How much context is spent before answering begins.
Leave more room to reason.Published August 27, 2026 by Parieto Research.
Bring us one repository question ↗