False acceptance matters more than coverage alone

Define the acceptance event

An invoice is automatically accepted when the application allows it to proceed without a person correcting the required fields. Identify exactly which fields and checks are necessary for that event in your workflow.

Measure correctness against reviewed source documents, not against the extractor's own confidence. A confidence threshold is part of the decision rule being evaluated, so it cannot also serve as independent proof that the decision was correct.

Keep document-level and field-level results separate. A record with nine correct fields and one wrong payable amount may be unacceptable even though its average field accuracy looks high.

Use two denominators

Automatic coverage is the proportion of eligible documents accepted without review. False acceptance is the proportion of automatically accepted documents that contain a disqualifying error under the defined rules.

For a hypothetical batch of 1,000 documents, accepting 700 automatically gives 70 percent coverage. If seven of those accepted records contain a disqualifying error, the observed false-acceptance rate is 1 percent of accepted records. Reporting only 70 percent automation hides the quality of what passed.

Also record incorrect rejections or unnecessary reviews. They affect workload, but their consequence differs from allowing a wrong record to proceed.

Break down the document population

Report results by supplier layout, scan quality and document class where those differences matter. A common clean format can dominate the average while a smaller unfamiliar class fails frequently.

Include sample sizes and the method used to select reviewed documents. Reviewing only low-confidence cases cannot estimate the error rate among high-confidence accepted records. Sample accepted records as well.

Track correction reasons, such as wrong field mapping, missed adjustment, duplicate document or currency ambiguity. These categories point to different improvements and help determine whether a threshold change would address the problem.

Choose the threshold with consequences visible

Compare candidate acceptance rules on the same reviewed dataset. Plot or tabulate coverage against observed false acceptance rather than selecting the threshold that produces the largest automated volume.

Use the business owner's tolerance and the downstream consequence to decide what can proceed. Some errors may require a blocking rule regardless of the average. Continue sampled review after release because supplier formats and input quality change. Acceptance is a maintained decision process, not a confidence value chosen once during implementation.