Why We Open-Sourced Our Recall Harness

Last week we published nfrecall under Apache 2.0. It is the harness we use to score every index we run: it samples a query log, computes exact nearest neighbours by brute force, and reports recall@k with confidence intervals against whatever engine you point it at.

It is also the tool behind every recall number on this website. Publishing it means anyone can check us. That was the point.

What it does

Give it a collection, a query sample and a value of k. It computes the true top-k the slow way, runs the same queries against your index, and reports the overlap, the per-query loss, and how large a sample you need before the figure means anything. That last part is the bit most homegrown scripts get wrong.

It works against Nodeform, pgvector, and two of the open-source engines we are most often compared to. We did not make it easier to score ourselves.

Why now

Three reasons, honestly.

1. It is not a moat. Measuring recall is arithmetic. The hard part is holding an index to the number afterwards, week after week, and that is the product.
2. A claim you cannot check is marketing. We publish a 99 percent recall@10 target on the Scale tier. Shipping the measuring instrument with the claim is the only version of that sentence worth writing.
3. Hiring. Engineers who care about recall methodology are exactly the people we want to talk to, and a repository says more than a careers page.

What we are not publishing

The shard planner, the graph implementation, the rebuild machinery. Everything that is actually a moat. This is not a strategy shift; it is nfrecall-shaped. If we open-source something else later it will be for nfrecall-shaped reasons.