Hybrid knowledge search
Change the embedding model without losing identifier search
An embedding change is an index migration. Keep query and document representations compatible, and preserve the exact-match path while comparing the new retrieval behaviour.
In this article
Treat the representation as a versioned dependency
An embedding model turns text into a numeric representation used by vector search. Stored document vectors and newly generated query vectors need to belong to a compatible representation. Matching vector dimensions alone does not establish compatibility.
Record the model, relevant configuration and passage-generation version used for each index build. If the new model also requires a different passage strategy, make that an explicit part of the migration. Otherwise, a changed result may be attributed to the model when the underlying text boundaries changed too.
Keep the keyword fields intact unless there is a separate reason to change them. They provide a useful comparison path and protect exact identifiers from becoming an accidental casualty of the vector migration.
Build the new representation beside the old one
Create a new vector field or index according to the search system's supported migration process. Populate it without directing ordinary queries to incomplete data. Track which source versions have been embedded and which failed.
A successful job count does not prove the whole collection was processed. Reconcile source identifiers and versions, and inspect failed or skipped records. Long documents, unusual encodings and recently changed files can be overrepresented in the gaps.
Define how ongoing document updates reach both serving versions during the comparison period. If the old index stops receiving changes, it becomes a progressively less reliable fallback. If both versions ingest updates, monitor the extra cost and the possibility of one path falling behind.
Compare behaviour by query type
Run the maintained query collection against both versions with compatible query embeddings. Compare candidate identifiers, final selected passages and answer support. Do not compare raw similarity scores as though they were on a shared scale.
Pay particular attention to exact codes, short abbreviations and unsupported questions. A new model may improve broad semantic matches while making these groups less predictable. Keep a separate view of changes that affect the wrong product or document version.
Use a limited rollout only after the offline comparison is understood. Record which requests use which representation so a reported problem can be reproduced. Avoid a rollout in which a request can unpredictably combine a new query vector with an old document field.
Keep the rollback pair together
The rollback unit includes the query embedding configuration, the target vector field or index and the fusion settings that were evaluated with it. Returning only one of these to an earlier value can leave a mismatched system.
Before cutover, demonstrate that the old path still answers the fixture using current source permissions. Record the last source checkpoint it contains. If it cannot safely serve current data, use a deliberately restricted fallback rather than describing it as a ready rollback.
After the observation period, remove unused vectors and migration credentials through a planned cleanup. Keep the evaluation record and configuration identifiers. They explain why the new model was adopted and give the next migration a baseline that is more useful than a vague statement that search improved.
Primary sources
Microsoft Learn: rebuilding search indexesMicrosoft Learn: hybrid ranking behaviourReferences checked 11 September 2026.