Design notes

How the Invoice Agent works

A process that takes a vendor invoice PDF and produces a reasoned, auditable accounts-payable decision, with everything in between visible.

Try it

Design principles

Rules decide, AI reads and explains

GPT-5.1 extracts fields, maps invoice lines to PO lines and writes the explanation. Whether money moves is decided by a fixed, ordered policy in code, so every decision is reproducible and auditable, and a cleverly worded invoice can't talk its way to approval.

Two independent readers

Azure Document Intelligence (a purpose-built invoice model) and GPT-5.1 read every document. Key fields are cross-checked; agreement raises confidence, disagreement lowers it and is surfaced to a human.

State, not just one document

Approved invoices post to an AP ledger. The next invoice is checked against what has already been billed on the PO and what has been received, which is how split invoices, over-billing and duplicates are caught.

Never trust the document

Bank details are compared with the vendor master, not taken from the invoice. Text addressed to 'automated systems' is ignored and flagged. Vendor emails are never drafted for fraud or duplicate holds: those are verified through contacts already on file.

Fail safe, not fail silent

Every AI call has a timeout, retries and a fallback (embedded PDF text if OCR is down, Document Intelligence fields if the LLM is down, a template if the explainer is down). A broken step routes the invoice to a human instead of guessing.

Humans own the exceptions

Anything that isn't clean goes to a named owner (buyer, AP, vendor management, tax, finance manager) with a note saying exactly what to verify. Their decision is written to the audit trail and the ledger follows it.

Decision policy

Evaluated top-down; the first rule that matches decides. Every failed or noted check is still listed as a reason. Risk rules sit above everything else, so a duplicate is held even if it also has other problems.

#	When	Decision	Routed to
1	Not a PDF / not an invoice (quote, statement)	Rejected	AP Team
2	Password-protected file	Info requested	Vendor (via AP)
3	Unreadable, too long, or a step failed	Needs review	AP Team
4	Credit note	Needs review	AP Team
5	Duplicate: same file, same or reformatted number, same amount + date	On hold	AP Team
6	Bank details differ from vendor master	On hold	Vendor Management
7	Instructions aimed at automated systems	On hold	AP Team · Security
8	Vendor blocked	Rejected	Vendor Management
9	Unknown vendor and unknown PO	Rejected	AP Team
10	Unknown or inactive vendor	Needs review	Vendor Management
11	Addressed to another company	Rejected	AP Team
12	Missing number/date/total, arithmetic errors, future-dated	Info requested	Vendor (via AP)
13	GSTIN invalid or different from master	Needs review	Tax team
14	PO not found, ambiguous, other vendor's, closed; line not on PO	Needs review	Buyer on the PO
15	Quantity over PO or over received, price or total outside tolerance	Needs review	Buyer on the PO
16	Currency differs from PO	Needs review	AP Team
17	Wrong GST type (CGST+SGST vs IGST) or amount	Needs review	Tax team
18	Key field read with low confidence	Needs review	AP Team
19	Clean but above ₹10 lakh	Needs review	Finance Manager
20	Clean with notes (price within tolerance, PO inferred…)	Approved · note	AP Team
21	Clean	Approved	AP Team

Edge cases

Each is a real AP failure mode with a sample PDF you can run. Highlighted cards are the core demo set.

03Scanned invoice with no PO numberApproved · note

A scanned, slightly skewed invoice with no PO number, only 'your order dated 12 Oct'. Two open GreenLeaf POs both have boxes and tape. The process must OCR it and work out which PO it belongs to.

PO_INFERRED

04Second split invoice on the same POApproved

TechNova bills 6 more laptops on the same PO. Together with invoice 1 that is 16 of 20 ordered and 18 received, so it still passes.

05Third split invoice over-bills the PONeeds review

Another 6 laptops: 22 billed against 20 ordered and only 18 received. Each invoice looks fine alone; only the running PO balance reveals the problem.

QTY_EXCEEDS_PO · QTY_EXCEEDS_RECEIVED

06Disguised duplicate of a paid invoiceOn hold

A re-scanned copy of GreenLeaf invoice GL/2026/0457 (already paid on 20 Oct), now numbered 'GL-2026-457'. Different file, different number format, same bill.

DUPLICATE_SUSPECTED

07Bank details changed on the invoiceOn hold

Everything matches the PO, but the invoice asks for payment to a new bank account. Classic payment-diversion fraud pattern: hold and verify by phone.

BANK_MISMATCH

08Invoice currency differs from the PONeeds review

CloudSpring bills USD 4,800 against a PO raised in INR. At the reference rate it also exceeds the PO value.

CURRENCY_MISMATCH

09Vendor not in the master, PO doesn't existRejected

An invoice from a company Meridian has never onboarded, quoting a PO number that does not exist.

VENDOR_UNKNOWN · PO_NOT_FOUND

10Invoice number and date missingInfo requested

A valid vendor and PO, but the invoice has no invoice number and no date, so it can't be posted. Ask the vendor for a corrected invoice.

MISSING_INVOICE_NUMBER · MISSING_INVOICE_DATE

11Totals don't add upInfo requested

Line items add up to ₹1,71,000 but the invoice states a subtotal of ₹1,76,000, with tax and total computed on the wrong figure.

ARITHMETIC_MISMATCH

12Credit note, not an invoiceNeeds review

A credit note against a September invoice. It must not be paid; AP should apply it against the vendor balance.

CREDIT_NOTE

13Blocked vendorRejected

Pinnacle's payments are suspended pending a GST compliance review. The invoice itself is fine, but the vendor can't be paid.

VENDOR_BLOCKED

14Unit price 8% above PONeeds review

Monitors billed at ₹15,350 against ₹14,200 on the PO (+8.1%), well beyond tolerance.

PRICE_VARIANCE

15Hidden instruction aimed at the AIOn hold

The invoice hides a line telling 'automated systems' it is pre-approved by the CFO. It also bills 6 deep cleans against 4 ordered and received. The instruction must be ignored and flagged.

PROMPT_INJECTION · QTY_EXCEEDS_PO

Assumptions

The buyer is Meridian Consumer Brands Pvt. Ltd., a mid-size Indian company in Karnataka. Invoices are in INR with GST, plus one US SaaS vendor in USD.
Vendors, purchase orders, goods/service receipts and past invoices live in a system of record (seeded here; in production, the ERP).
Tolerances: unit price ±2%; total ±2% or ₹5,000, whichever is lower. Clean invoices above ₹10 lakh still need a finance manager.
3-way match: billed quantity (including earlier invoices) may not exceed ordered or received quantity. Services use service-entry confirmations as receipts.
A PO inferred by the agent is accepted only above 80% confidence and still flagged as a note for the buyer.
Foreign-currency invoices are compared with INR POs at a configured reference rate; line-level price checks are skipped across currencies.
Approved invoices count toward PO consumption; invoices on hold or in review don't, but still block duplicates.
All companies, people, GSTINs and bank accounts are fictional; GSTINs use the real checksum algorithm.

Architecture

Front end + API

Next.js 16 on Vercel (Fluid compute). Runs execute in the background with after(); the UI polls the run and renders each stage live.

OCR + invoice fields

Azure AI Document Intelligence v4, prebuilt-invoice model: works on digital and scanned PDFs, returns field confidences and page coordinates.

Language model

Azure OpenAI GPT-5.1 via the Responses API with strict JSON schemas: extraction, PO inference + line mapping, and the decision note.

System of record

Neon Postgres: vendors, POs, receipts, AP ledger, plus every run, step, event and page render for the audit trail.

Safety

Vercel BotID on endpoints that spend credits, per-IP rate limits and a daily cap; prompt-injection detection; no secrets in the browser.

Typical run: 20–40 s and about $0.02–0.03 of AI cost (≈ ₹2) per invoice.

What I'd build next

Email intake

A dedicated AP inbox: each forwarded invoice starts a run, and request-info emails go back on the same thread.

ERP write-back

Post approved invoices to Tally, SAP or NetSuite and read POs and receipts live instead of from a seed.

Verification APIs

GSTN lookup for registration status, penny-drop bank verification, and a vendor callback workflow for bank changes.

Evaluation set

A labelled set of real invoices to measure extraction accuracy and calibrate confidence thresholds per vendor.

Multi-invoice files

Split PDFs that contain several invoices, and handle multi-page line items beyond ten pages.

SLA + analytics

Ageing of exceptions per owner, early-payment discount capture, and vendor-level error rates.

Design principles

Rules decide, AI reads and explains

Two independent readers

State, not just one document

Never trust the document

Fail safe, not fail silent

Humans own the exceptions

Decision policy

#	When	Decision	Routed to
1	Not a PDF / not an invoice (quote, statement)	Rejected	AP Team
2	Password-protected file	Info requested	Vendor (via AP)
3	Unreadable, too long, or a step failed	Needs review	AP Team
4	Credit note	Needs review	AP Team
5	Duplicate: same file, same or reformatted number, same amount + date	On hold	AP Team
6	Bank details differ from vendor master	On hold	Vendor Management
7	Instructions aimed at automated systems	On hold	AP Team · Security
8	Vendor blocked	Rejected	Vendor Management
9	Unknown vendor and unknown PO	Rejected	AP Team
10	Unknown or inactive vendor	Needs review	Vendor Management
11	Addressed to another company	Rejected	AP Team
12	Missing number/date/total, arithmetic errors, future-dated	Info requested	Vendor (via AP)
13	GSTIN invalid or different from master	Needs review	Tax team
14	PO not found, ambiguous, other vendor's, closed; line not on PO	Needs review	Buyer on the PO
15	Quantity over PO or over received, price or total outside tolerance	Needs review	Buyer on the PO
16	Currency differs from PO	Needs review	AP Team
17	Wrong GST type (CGST+SGST vs IGST) or amount	Needs review	Tax team
18	Key field read with low confidence	Needs review	AP Team
19	Clean but above ₹10 lakh	Needs review	Finance Manager
20	Clean with notes (price within tolerance, PO inferred…)	Approved · note	AP Team
21	Clean	Approved	AP Team

Edge cases

Each is a real AP failure mode with a sample PDF you can run. Highlighted cards are the core demo set.

03Scanned invoice with no PO numberApproved · note

A scanned, slightly skewed invoice with no PO number, only 'your order dated 12 Oct'. Two open GreenLeaf POs both have boxes and tape. The process must OCR it and work out which PO it belongs to.

PO_INFERRED

04Second split invoice on the same POApproved

TechNova bills 6 more laptops on the same PO. Together with invoice 1 that is 16 of 20 ordered and 18 received, so it still passes.

05Third split invoice over-bills the PONeeds review

Another 6 laptops: 22 billed against 20 ordered and only 18 received. Each invoice looks fine alone; only the running PO balance reveals the problem.

QTY_EXCEEDS_PO · QTY_EXCEEDS_RECEIVED

06Disguised duplicate of a paid invoiceOn hold

A re-scanned copy of GreenLeaf invoice GL/2026/0457 (already paid on 20 Oct), now numbered 'GL-2026-457'. Different file, different number format, same bill.

DUPLICATE_SUSPECTED

07Bank details changed on the invoiceOn hold

Everything matches the PO, but the invoice asks for payment to a new bank account. Classic payment-diversion fraud pattern: hold and verify by phone.

BANK_MISMATCH

08Invoice currency differs from the PONeeds review

CloudSpring bills USD 4,800 against a PO raised in INR. At the reference rate it also exceeds the PO value.

CURRENCY_MISMATCH

09Vendor not in the master, PO doesn't existRejected

An invoice from a company Meridian has never onboarded, quoting a PO number that does not exist.

VENDOR_UNKNOWN · PO_NOT_FOUND

10Invoice number and date missingInfo requested

A valid vendor and PO, but the invoice has no invoice number and no date, so it can't be posted. Ask the vendor for a corrected invoice.

MISSING_INVOICE_NUMBER · MISSING_INVOICE_DATE

11Totals don't add upInfo requested

Line items add up to ₹1,71,000 but the invoice states a subtotal of ₹1,76,000, with tax and total computed on the wrong figure.

ARITHMETIC_MISMATCH

12Credit note, not an invoiceNeeds review

A credit note against a September invoice. It must not be paid; AP should apply it against the vendor balance.

CREDIT_NOTE

13Blocked vendorRejected

Pinnacle's payments are suspended pending a GST compliance review. The invoice itself is fine, but the vendor can't be paid.

VENDOR_BLOCKED

14Unit price 8% above PONeeds review

Monitors billed at ₹15,350 against ₹14,200 on the PO (+8.1%), well beyond tolerance.

PRICE_VARIANCE

15Hidden instruction aimed at the AIOn hold

The invoice hides a line telling 'automated systems' it is pre-approved by the CFO. It also bills 6 deep cleans against 4 ordered and received. The instruction must be ignored and flagged.

PROMPT_INJECTION · QTY_EXCEEDS_PO

Assumptions

The buyer is Meridian Consumer Brands Pvt. Ltd., a mid-size Indian company in Karnataka. Invoices are in INR with GST, plus one US SaaS vendor in USD.

Vendors, purchase orders, goods/service receipts and past invoices live in a system of record (seeded here; in production, the ERP).

Tolerances: unit price ±2%; total ±2% or ₹5,000, whichever is lower. Clean invoices above ₹10 lakh still need a finance manager.

3-way match: billed quantity (including earlier invoices) may not exceed ordered or received quantity. Services use service-entry confirmations as receipts.

A PO inferred by the agent is accepted only above 80% confidence and still flagged as a note for the buyer.

Foreign-currency invoices are compared with INR POs at a configured reference rate; line-level price checks are skipped across currencies.

Approved invoices count toward PO consumption; invoices on hold or in review don't, but still block duplicates.

All companies, people, GSTINs and bank accounts are fictional; GSTINs use the real checksum algorithm.

Architecture

Front end + API

Next.js 16 on Vercel (Fluid compute). Runs execute in the background with after(); the UI polls the run and renders each stage live.

OCR + invoice fields

Azure AI Document Intelligence v4, prebuilt-invoice model: works on digital and scanned PDFs, returns field confidences and page coordinates.

Language model

Azure OpenAI GPT-5.1 via the Responses API with strict JSON schemas: extraction, PO inference + line mapping, and the decision note.

System of record

Neon Postgres: vendors, POs, receipts, AP ledger, plus every run, step, event and page render for the audit trail.

Safety

Vercel BotID on endpoints that spend credits, per-IP rate limits and a daily cap; prompt-injection detection; no secrets in the browser.

Typical run: 20–40 s and about $0.02–0.03 of AI cost (≈ ₹2) per invoice.

What I'd build next

Email intake

A dedicated AP inbox: each forwarded invoice starts a run, and request-info emails go back on the same thread.

ERP write-back

Post approved invoices to Tally, SAP or NetSuite and read POs and receipts live instead of from a seed.

Verification APIs

GSTN lookup for registration status, penny-drop bank verification, and a vendor callback workflow for bank changes.

Evaluation set

A labelled set of real invoices to measure extraction accuracy and calibrate confidence thresholds per vendor.

Multi-invoice files

Split PDFs that contain several invoices, and handle multi-page line items beyond ten pages.

SLA + analytics

Ageing of exceptions per owner, early-payment discount capture, and vendor-level error rates.