A team tests their invoice extraction pipeline on twenty documents.95 Percent field accuracy. They ship it.

Six weeks later, it's dropping line items from every vendor it wasn't tested on. No crash. No error.

Just missing data, piling up until someone in accounts payable notices a four-month reconciliation gap.

This isn't a one-off. Manual data entry runs at roughly a 1% error rate under good conditions and 3 to 4% once fatigue and time pressure kick in, based on benchmarks compiled across the data-entry industry.

Teams building AI extraction to fix that error rate keep hitting a different problem: the model was never what broke.

This issue breaks down why AI document processing pipelines pass every demo and still fail in production, with one real example of a fix that treats accuracy as a measured number instead of a gut feeling.