FinRule-Bench: a new arXiv benchmark testing whether LLMs can apply accounting rules to real financial tables

Large language models are moving into finance. But can they audit a balance sheet while correctly applying explicit accounting principles? A new arXiv paper, "FinRule-Bench: A Benchmark for Joint Reasoning over Financial Tables and Principles" (https://arxiv.org/abs/2603.11339), sets out to answer that question by creating tasks that require models to jointly reason over structured financial statements and the formal rules that govern them.

What FinRule-Bench does

Unlike prior datasets that focus on question answering, numerical arithmetic, or spotting synthetically injected anomalies, FinRule-Bench frames auditing as rule-guided reasoning: models must interpret tables, locate relevant line items, and apply specific accounting principles to decide whether entries comply with those principles or require adjustment. The authors construct test cases that pair realistic financial tables with explicit rule descriptions and decision labels, aiming to measure whether LLMs can perform the kind of procedural, principle-driven checks that human auditors do.

Why it matters

The benchmark is timely. It has been reported that LLMs are increasingly applied to financial analysis, and regulators are paying closer attention to model-driven decisioning in markets. Accurate, auditable reasoning matters for investor protection and for firms that hope to automate parts of audit and compliance workflows. It has also been reported that geopolitics — for example, tightened U.S. export controls on advanced AI chips — could shape which models and toolchains are available to firms in different jurisdictions, making independent, open benchmarks one way to assess cross-border model performance and safety. The FinRule-Bench release on arXiv gives researchers and practitioners a reproducible yardstick to test whether current LLMs can truly bridge table understanding and accounting expertise — or whether significant model and dataset work remains.