Use a fluent response with the wrong extracted field

Build a document with competing values

Create a synthetic invoice containing an issue date, due date and service-period end date. Make all three valid dates and place them close enough that a model must interpret their labels correctly.

The expected output should identify each field separately. A parser that only checks date format will accept the wrong selection, so the fixture needs exact expected values tied to source locations.

Include a short explanation task if the product generates one. This lets the test demonstrate that persuasive prose can coexist with incorrect structured data.

Run baseline and candidate on the same artifact

Use the same document bytes, extraction output and application configuration apart from the intended release change. Record any other differences. If the parser changed too, isolate that effect before concluding the model caused the regression.

Inspect the raw output and the validated object. Some application code silently normalises or repairs values, which can hide whether the model selected the wrong field or the parser transformed it incorrectly.

Compare results over repeated runs where variability matters. Preserve every run rather than selecting the most convincing answer from each candidate.

Exercise downstream validation

Pass the wrong but well-formed date into the normal application path. Confirm whether a business rule can detect the inconsistency. For example, a due date earlier than issue date may be suspicious, but a wrong date can still be chronologically plausible.

Do not invent a universal date rule to make the fixture pass. Use the actual workflow's constraints and keep extraction correctness separate from downstream plausibility checks.

If the application asks a reviewer to resolve uncertainty, verify that the relevant source labels are visible. A confidence score alone may not help the person identify which date was selected.

Turn the regression into a release assertion

Add the case to the appropriate dataset category, with a rationale about field confusion rather than the candidate's exact wording. Include neighbouring variants such as a missing due date or a label that spans a line break, but avoid counting trivial paraphrases as broad coverage.

Record the candidate's behaviour, validation outcome and final business effect. A release should not pass this case merely because the answer sounds more professional. The useful test establishes whether the application extracted the field the workflow actually needs.