# Spend reranking time where it improves the first useful result

Measure the quality gained by each retrieval stage against its latency and operating cost. More candidates and more models do not automatically produce better search.

By Cobnex editorial. Published 2026-09-10. Updated 2026-09-11.

## Establish a simple measured baseline

Start with the supported lexical or existing retrieval configuration and a judged query set. Record first-page usefulness, candidate coverage, latency and errors.

A known-item query may already work well with exact matching, while a conceptual question may benefit from another retrieval stage. Report those categories separately so expensive processing is justified by the queries it helps.

Do not use raw engine score increases as evidence of improvement. Compare ranked documents against the same task judgments.

## Vary candidate depth deliberately

A reranker can only reorder the candidates it receives. Too few candidates may exclude the useful document, while too many add work with little visible benefit.

In an illustrative experiment, candidate depths of 20, 50 and 100 produce different recall and latency. Choose from measured per-query outcomes, especially the difficult cases, rather than assuming the largest set is best.

Keep timeouts and fallback behaviour in the evaluation. If the expensive stage fails, the baseline results should remain understandable and permission-correct under the chosen design.

## Include indexing and maintenance costs

Vector representations, additional fields and multiple indexes add ingestion and update work. Content changes may require recomputation, while stale representations can reduce usefulness even when query latency remains low.

Budget for human judgments and regression review. A sophisticated ranking pipeline without maintained evaluation data can become harder to tune than a simpler system with clear evidence.

Cache only where the query, access context and freshness requirements permit it. A shared cache hit is not a saving if it exposes another user's results or preserves obsolete procedures.

## Choose a workload-specific operating point

Measure cost per completed search task where practical, alongside request cost. An inexpensive query that forces several reformulations may not be the cheaper user journey.

Keep critical-query regressions visible. A small average gain does not justify making an essential recovery procedure difficult to find.

Document which stages are enabled, their candidate limits and their fallback. That gives the next team a reasoned configuration to maintain instead of a chain of expensive defaults nobody can explain.

## Sources

- [Microsoft: relevance and ranking overview](https://learn.microsoft.com/en-us/azure/search/search-relevance-overview)
- [Elastic: ranking evaluation](https://www.elastic.co/docs/reference/elasticsearch/rest-apis/search-rank-eval)