A validation layer for AI invoice extraction, in Python
What to run after the model reads an invoice, in what order, and what each check caught in my test of 30 invoices.
Checking that net plus VAT equals the gross total is the first rule most invoice pipelines add, and in my test it caught 3 of the 63 wrong invoices. Two rules on the VAT IDs caught 31 of the 32 wrong ones from the three best models. So the order of your checks matters more than the model you picked.
I'm Denys, a Forward Deployed Engineer. This article was written with AI assistance and checked by me. The numbers come from a test I ran on 22 October 2026: 30 fictional Czech, German and Portuguese invoices, four models, each invoice read clean and as a simulated scan, 240 calls. An invoice counted as right only when all nine fields matched.
What does the validation layer look like?
The model returns nine fields: invoice number, issue date, due date, supplier VAT ID, buyer VAT ID, currency, net, VAT and gross. It also lists the fields it is unsure about. The layer takes that result and returns a list of reasons to send the invoice to a person. Empty list means it goes to the ledger.
from decimal import Decimal
from stdnum.cz import dic
from stdnum.de import vat as de_vat
from stdnum.pt import nif
VAT_CHECKS = {"CZ": dic.is_valid, "DE": de_vat.is_valid, "PT": nif.is_valid}
def vat_ok(country, number):
check = VAT_CHECKS.get(country)
return bool(check and check(number))
def review(inv, own_vat_id):
reasons = []
if not vat_ok(inv["country"], inv["supplier_vat"]):
reasons.append("supplier VAT ID fails check digit")
if not vat_ok(inv["country"], inv["buyer_vat"]):
reasons.append("buyer VAT ID fails check digit")
if inv["buyer_vat"].replace(" ", "").upper() != own_vat_id:
reasons.append("buyer VAT ID is not ours")
if abs(inv["net"] + inv["vat"] - inv["gross"]) > Decimal("0.01"):
reasons.append("net + VAT does not equal gross")
for field in inv["unsure_fields"]:
reasons.append(f"model unsure: {field}")
return reasons
Status: it uses the python-stdnum library, whose modules expose is_valid() and validate(). I checked that the imports work and ran review on a dummy dict. I did not run it on the test invoices, so treat it as a sketch. The 0.01 tolerance and the dict shape are my choices.
Why do the VAT ID rules come first?
A VAT ID ends in a check digit computed from the other digits. It is modulo 11 for the Czech company number behind a DIČ and for the Portuguese NIF, and ISO 7064 for the German USt-IdNr. A misread digit almost always breaks it. The second rule needs no library. On an incoming invoice the buyer is you, so any other buyer VAT ID is a misread or an invoice that was never meant for you.
Claude Haiku 5.5, Claude Sonnet 5.5 and Gemini 3.8 Flash made 32 wrong extractions between them. Only two had a wrong amount. The rest misread a digit in a VAT ID, or in an invoice number such as FV-2026-0738 read as FV-2026-0758, because on a grey, blurred scan a 3 looks like a 5. An arithmetic check has nothing to say about that. These two rules caught 31 of the 32.
If you want the whole test, the setup and the scripts, they are in my write-up on invoice data extraction with AI. It has the table for all four models, the costs and an archive with every result.
What is the totals check still good for?
It stays in the list, second in line. The one invoice the VAT rules missed was a Gemini scan with a wrong gross total, and only the totals check caught it. A model also writes totals that agree with the rest of what it read. GPT-5.6 Luna got amounts wrong on 7 invoices, and on 5 of them net, VAT and gross still added up. A Haiku scan had net and VAT slightly off while the sum still matched gross.
Is the model's own doubt a control?
I asked each model to list the fields it was unsure about. Alone, that flag caught nothing. Across the four models the number of wrong invoices caught only by the flag was 0. It is still worth reading, with a cost you should know about:
| Model | Wrong scans of 30 | Correct invoices it still flagged |
|---|---|---|
| Claude Haiku 5.5 | 16 | 13 of 44 |
| Claude Sonnet 5.5 | 8 | 19 of 52 |
| Gemini 3.8 Flash | 8 | 0 of 52, it never lists unsure fields |
All the Claude flags were on scans, and neither flagged a clean invoice. So with these two models the queue held 30% and 37% of the correct invoices next to the wrong ones. In accounts payable I accept that, because a clerk's glance costs less than a wrong payment. Gemini matched Sonnet at 22 of 30 scans right. For Gemini the rules are the whole control: they caught 7 of its 8 wrong invoices and the totals check the eighth.
What does this layer not catch?
A check digit cannot catch every misread, because some wrong numbers are valid too. In my run no wrong VAT ID passed both rules, but that is 30 invoices. An invoice number has no check digit at all, so a misread there is only caught when the same invoice also has a bad VAT ID. I did not split that out. A duplicate check against your ledger might help, and I have not tested it.
The invoices were fictional, one layout per country, and the scans were simulated with blur, rotation and compression. Each country had one buyer VAT ID repeated on 10 invoices, so one misread string can break up to 10 at once. On an earlier set of 30 the scan scores for Haiku, Sonnet and Gemini were 22, 22 and 17, and on this set 14, 22 and 22. So measure the layer on your own invoices, with the worst scans you receive.
One more thing, before the model runs at all. In Portugal certified invoice software has to put a QR code on the invoice with both NIFs, the date, the number, the ATCUD and the base and VAT per rate. Read it first, and send only the rest to a model.
Questions about any of this, or a check you would add? Ask in the comments.
Comments
Join the discussionThe review sketch currently applies inv['country'] to both supplier and buyer VAT checks. I would give those identifiers their own country information, since an invoice can have a supplier and buyer in different countries. Otherwise a correctly extracted buyer identifier can be rejected by the supplier country's validator.
A useful next fixture would pair a domestic invoice with a cross-border one and preserve the original identifier text beside its normalized form. That would separate extraction mistakes from validation-routing mistakes before measuring how many of the 32 failed extractions the layer actually catches.