Investigate a payload found in unexpected diagnostics
Establish which data was copied, who could access it and which path created it. Contain further logging while preserving the evidence needed to assess the incident.
Read articleCobnex blogs / Australia
Cobnex blog
AI implementation, software architecture, cloud operations and Australian technology policy.
150 articles in AI engineering
Page 5 of 9
Stop an over-budget run with a recoverable status
Prevent new work, account for operations already accepted and preserve the useful result. Cancelling the conversation alone may leave costs and effects running in the background.
Read articleRoute an unresolved answer to someone who can help
A handoff should carry the unanswered question and evidence already checked. Make the next owner clear without promising work the application has not actually submitted.
Read articleReindex protected documents without opening a gap
A search migration must preserve access rules as carefully as content. Use a parallel index and an explicit cutover check so incomplete permissions never become public results.
Read articleChange the embedding model without losing identifier search
An embedding change is an index migration. Keep query and document representations compatible, and preserve the exact-match path while comparing the new retrieval behaviour.
Read articleRechunk a corpus with stable source references
Changing passage boundaries should not make citations point to unrelated text. Separate source identity from the temporary shape of an index build.
Read articleKeep older answers inspectable after a source revision
Updating the knowledge base should not silently change what an earlier answer claims to have used. Plan how source versions and saved answers move together.
Read articleRun old and new indexes with a clear activation point
A replacement index needs a source checkpoint, a catch-up plan and compatible answer caches. Switching a query alias is only one step.
Read articleRetest boundaries whenever a new tool is added
A model that previously could only summarise may become much more consequential when it can send, update or retrieve additional data. Treat that capability change as a new security review.
Read articleIntroduce a write tool behind a bounded cohort
Moving from suggestions to mutations changes the product's responsibility. Release one explicit operation with observable effects and a way to stop it.
Read articleAdd approval without leaving an older bypass open
A new review screen is ineffective if another endpoint can still execute the same mutation directly. Migrate the authority rule across every execution path.
Read articleUpgrade the agent while old workflows are paused
A deployment can change tool schemas and state meaning while runs are waiting for approval. Plan compatibility for those saved records before switching new traffic.
Read articleVersion the evaluation set alongside the release
Changing questions, sources or labels can change the score without changing the application. Keep dataset revisions explicit and preserve comparisons across the transition.
Read articleChoose a release cohort that includes difficult work
A small rollout is useful only if it exercises the behaviours you need to observe. Select the cohort by tasks and risk, not just by a random fraction of traffic.
Read articleIntroduce automatic acceptance one document class at a time
Begin with a class whose fields and validation rules are understood. Expand using reviewed evidence rather than assuming every supplier format behaves the same way.
Read articleMove inference without overlooking the embedding store
Changing the answer model's endpoint leaves ingestion, retrieval and diagnostics untouched unless the migration explicitly includes them. Review the entire dependency set.
Read articleRoll out cheaper routing against the same acceptance bar
A lower-cost model route should satisfy the task's existing quality and authority requirements. Evaluate routing errors and fallback work as part of the candidate.
Read articleAdjust abstention thresholds with observed task outcomes
Changing a threshold alters which questions receive answers. Review newly accepted and newly withheld cases before applying the change broadly.
Read article