Build the lexical baseline before adding embeddings

Put identifiers in fields that preserve their meaning

Take a representative document and separate its useful fields. A product identifier, title, version, document type and body text do not all need the same search treatment. An identifier may need exact matching, while the body benefits from normal text analysis.

Text analysis is the process that turns words into searchable tokens. Lowercasing may be harmless for one field, while removing punctuation can merge two identifiers that should remain distinct. Test the actual identifiers used by the business before accepting the default analyser for every field.

Retain the original display value as well as any normalised value used for matching. The application should not have to reconstruct the official part number from tokens when it presents a result or a citation.

Establish a small, repeatable baseline

Collect questions from the intended workflow, with sensitive details removed where necessary. For each question, record the source that should be found and why. Include a few requests that the collection cannot answer. Keep the collection small enough for a domain specialist to review.

Run keyword search first and inspect the misses. Some are ingestion problems, such as a missing manual. Others are field or vocabulary problems. Correct those before adding a second retrieval method. An embedding cannot recover text that was never indexed, and it may conceal a poor identifier field by returning a plausible alternative.

Save the baseline result identifiers, not only screenshots. They let the team compare the same questions after a schema change without depending on memory.

Add a separately observable vector branch

Generate embeddings for the indexed passages and record the model and configuration used. Query embeddings must be compatible with the stored vectors. Do not mix representations from different models just because they happen to have the same number of dimensions.

Run the vector query as a separate branch with the same organisation, permission and document-version restrictions as keyword search. Capture its candidate identifiers independently. During development, make it possible to see whether a useful result came from keyword matching, vector matching or both.

The application-level sequence can be expressed without committing to a particular search SDK.

scope = resolveCurrentAccess(session)
keywords = keywordCandidates(question, scope)
vectors = vectorCandidates(question, embeddingVersion, scope)
candidates = mergeBySourceIdentity(keywords, vectors)
selected = rankAndLimit(candidates, question, contextBudget)

This is a conceptual sequence. Real implementations also need deadlines, cancellation, input limits and handling for a failed branch.

Compare the addition before making it the default

Evaluate the combined path against the keyword baseline. Count queries that improve and queries that get worse. Read the latter closely. A small overall gain can hide a serious regression for exact product identifiers.

Decide how the application behaves if vector generation or vector retrieval is unavailable. A keyword-only fallback may be reasonable, but it should be an explicit, observable mode. Do not label a partial result as a fully evaluated hybrid answer if that distinction matters to the user.

Once the combined path earns its place, add reranking only if the remaining errors are ordering errors. Keep the baseline test and the separate branch diagnostics. They remain useful when a later model update changes which documents are retrieved.