# A search team needs someone who can judge the answer

Engineers can tune retrieval, but domain owners must explain usefulness. Keep query intent, content ownership and configuration changes connected after launch.

By Cobnex editorial. Published 2026-09-10. Updated 2026-09-11.

## Assign ownership beyond the engine

Name the technical owner for indexing and query configuration, and the domain reviewers who can judge whether results answer the task. Identify content owners for missing or obsolete knowledge.

A search complaint may require a parser fix, a permission correction or a new procedure. Routing every complaint to ranking engineers wastes time and can lead to inappropriate boosts.

Keep a simple intake format with query, intent, expected content and access context. Sanitise examples before adding them to the shared evaluation library.

## Preserve the experiment record

Store corpus references, query fixtures, judgments, configuration versions and per-query results. The next team should be able to reproduce the accepted baseline and compare a proposed change.

Record important disagreements in relevance judgments. They often reveal an ambiguous user task or unclear document scope that no algorithm can resolve alone.

Keep critical operational queries visible in the review schedule. Popularity should not be the only reason a query remains in the benchmark.

## Rehearse diagnosis and rollback

Give the incoming team a synthetic missing-result case and ask them to distinguish access, ingestion and ranking causes. They should inspect candidacy before changing weights.

Document how to revert a query configuration or route to a fallback index, including any freshness limitations. Keep permissions enforced on every route.

Provide support with the information needed to identify the index and configuration active at the time of a complaint, without exposing private query contents broadly.

## Refresh the judgments as the organisation changes

New products, terminology and procedures change what people search for. Review emerging queries and retire obsolete judgments through a recorded process.

When a document is replaced, update its relationships and expected-result fixtures rather than leaving the old title artificially promoted. Search maintenance includes content lifecycle work.

The handover is complete when another team can explain why a result is useful, reproduce its ranking and identify who fixes the problem if that answer becomes wrong. An engine dashboard alone cannot provide those three things.

## Sources

- [Elastic: ranking evaluation](https://www.elastic.co/docs/reference/elasticsearch/rest-apis/search-rank-eval)