Agents decide the IBS/CBS treatment of real products and services, repair NF-e and NFS-e invoices, audit batches of them, and check the product codes declared on real invoice lines. Further families cover ICMS on interstate sales, audits of SPED Fiscal files and federal withholding on services. Every answer is graded field by field by a deterministic engine built on the law, the official code tables and the official layout and validation rules. No rubric, no LLM judge.
Potatoes sold for human consumption fall under two annexes of LC 214: Annex XV (zero rate) and Annex VII (60% reduction), and the law says Annex VII yields. Pick an answer to see how it scores.
One item of the invoice carries planted errors: a standard-rate product labelled with a 60% hygiene-products code and its reduction. The agent returns the whole XML; anything changed outside the IBS/CBS group scores zero.
A batch of invoices, some with planted errors (a reduction ignored, a wrong classification code, the 2027 IBS split used in 2026). The agent lists every wrong field; the reward is half item-level F1, half field-level F1.
A real invoice line: the supplier sold "ALHO" (garlic) under 0712.90.20, the code for dried garlic, which takes Annex VII's 60% reduction. Fresh garlic is 0703.20.90, zero-rated under Annex XV. The agent checks the code, gives the right one and both tax outcomes.
| Family | The agent | Answer | Graded by |
|---|---|---|---|
| F1 Decision | Gives the IBS/CBS treatment of one sale: CST, cClassTrib, base, rates, amounts. Seven kinds, from obvious standard-rate goods to annex precedence traps and sales dated after the rules run out. | JSON | Field diff weighted by tax impact; abstention scored both ways |
| F2 Invoice repair | Fills or fixes the IBS/CBS group of a 1–3 item NF-e, including the reduction block (gRed) the layout requires. | Full XML | Layout order and number formats checked; only fixes earn credit |
| F3 Batch audit | Finds every wrong field across 5–8 invoices. | JSON findings | Item-level and field-level F1 |
| F4 NCM audit | Checks the product code declared on a real invoice line against its description, gives the right code and the cClassTrib each code leads to: does the mistake change the tax? | JSON | Exact match on the accepted codes and both cClassTribs |
| F1T, F4T Tool use | F1-style decisions weighted toward annex exceptions, precedence and conditions the agent must infer, and F4-style NF-e batches of 3–8 real lines. The agent looks things up with five offline tools (NCM search and lookup, cClassTrib search and lookup, law search with distractors) under a call budget. | JSON, after tool calls | Same graders; every call logged |
| S1 EFD audit | Audits or repairs a synthetic monthly SPED Fiscal file (EFD ICMS/IPI): lists every line that breaks one of 63 official validation rules, each quoted from the official guide, or returns the corrected file. | JSON findings, or the full file | Audits like F3, on (line, rule) findings; repairs like F2, only fixes earn credit |
| I1 ICMS on goods | Gives the CFOP, the interstate rate, the interstate ICMS and the DIFAL owed to the destination state for an NF-e of one or more items, under each state's sourced rules, imported goods and their import content included; abstains where a deciding fact is missing. | JSON | Field diff weighted by tax impact; abstention scored both ways |
| R1 Federal withholding | Decides whether IRRF, CSRF (PIS/Cofins/CSLL) and INSS are withheld on a service invoice, with base and amount: minimums, Simples Nacional on either side, labour assignment, materials, missing facts. | JSON | Overlap with the retentions due; amounts within R$ 0,01 |
| N1, N2, N3 Services on the NFS-e | F1, F2 and F3 for services on the national NFS-e: the NBS code decides the treatment (LC 214 Annexes II and III, education and health), with the operation indicator and the place of incidence. | JSON, full XML, JSON findings | As F1, F2 and F3 |
Tasks ship as JSONL with a standalone scorer. Each pack is generated from a seed, carries a canary string, and records the engine, ruleset and data versions behind every answer.
Ground truth today: our curated readings (the LC 214 annex lists by NCM and NBS, the F4 product rules, each state's ICMS rules, the withholding rules and the EFD validation rules) are machine-checked by reproducible scripts against the law texts and the official tables. Sign-off by a Brazilian accountant is the next level, and every task records which level it rests on.
| Family | Claude Opus 5.5, effort high | Claude Opus 5.5, effort high, no tools | Expected answers | Simple heuristic | Abstain or do nothing |
|---|---|---|---|---|---|
| F1 Decision | 90.0% (96.5%) n = 30 | — | 100% (100%) | 46.7% (57.3%) | 6.7% (6.7%) |
| F2 Invoice repair | 100% (100%) n = 10 | — | 100% (100%) | 30.0% (52.9%) | 0% (0%) |
| F3 Batch audit | 90.0% (99.5%) n = 10 | — | 100% (100%) | 0% (40.1%) | 0% (0%) |
| F4 NCM audit | 73.0% (89.6%) n = 100 | — | 100% (100%) | 0% (18.0%) | 0% (0%) |
| F1T Decision, with tools | 96.7% (96.7%) n = 30 | 73.3% (82.2%) n = 30 | 100% (100%) | 43.3% (54.7%) | 3.3% (3.3%) |
| F4T NF-e audit, with tools | 93.3% (99.5%) n = 15 | 46.7% (90.3%) n = 15 | 100% (100%) | 0% (25.1%) | 0% (0%) |
| S1 EFD audit | 91.7% (98.5%) n = 24 | — | 100% (100%) | 0% (23.7%) | 0% (0%) |
| I1 ICMS on goods | 74.0% (93.6%) n = 100 | — | 100% (100%) | 6.7% (32.9%) | 10.0% (10.0%) |
| R1 Federal withholding | not run yet | not run yet | 100% (100%) | 33.3% (68.8%) | 0% (2.8%) |
| N1 Service decision | not run yet | not run yet | 100% (100%) | 36.7% (56.3%) | 3.3% (3.3%) |
| N2 NFS-e repair | not run yet | not run yet | 100% (100%) | 50.0% (57.0%) | 0% (0%) |
| N3 NFS-e batch audit | not run yet | not run yet | 100% (100%) | 0% (39.4%) | 0% (0%) |
Sample pack: pass rate, with the mean score in brackets; n is the number of answers graded. Claude Opus 5.5 at high effort handles standard-rate goods, near misses, Simples Nacional and the 2027 abstentions; its misses are mostly annex calls, such as água sanitária coded as sodium hypochlorite instead of the 3808.94.19 that Annex VIII names, or soybean oil and yogurt placed in the wrong annex, and on ICMS the DIFAL base in some destination states. With the lookup tools it closes most of the annex gap. Other frontier models are next.
| Pack | What you get | Price (USD) |
|---|---|---|
| Sample | 279 tasks (30 F1, 10 F2, 10 F3, 50 F4, 30 F1T, 15 F4T, 24 S1, 30 I1, 30 R1, 30 N1, 10 N2, 10 N3) with the scorers and the lookup tools, to evaluate. | Free |
| Pilot | 300 tasks across the families, difficulty mix to your spec, non-exclusive. | $15,000 flat credited toward a first pack |
| Standard pack | From 500 tasks, non-exclusive, any mix of families. | F1 $150 / task F2 $400 / task F3 $600 / task F4 $150 / task F1T $150 / task F4T $150 / task S1, I1, R1, N1–N3: on request |
| Exclusive | Your own generation seeds and held-out splits; tasks never resold. | 4× non-exclusive |
| Refresh | New tasks as rules change through the transition (Senate rates, new Notas Técnicas, cClassTrib updates). | From $5,000 / month |