What matters in AI.

Subscribe

A check against the first text finds 77% of AI finance errors

A check that uses the same data as the writer found only 35% of the added errors.

Researchers made FinVeriQA. It is an open-source prototype that gives an answer to each finance question. It uses reports of companies in the United States and live market data. A check with the same data as the writer finds only 35% of incorrect numbers. A check against the first text finds 77%. The authors tell that a check must not use the data that the writer saw.

How it works

A small model sends each question to the correct tool. It does not answer questions for advice. Python software does the maths. Mistral-7B writes the answer and shows the sources. A code check tests that each ID that it shows is in the data and that numbers agree.

The numbers

  • Mistral-7B did 29% of 100 calculations correctly. Python did 100%.
  • A live price cut a price error from 210% to 0.00%.
  • The small model got 0.85 for routing. Mistral-7B got 0.52.
  • Hybrid retrieval found the correct text for 76% of 50 questions.

What is not known

In one live test, the check accepted 2 answers with AMD sample data in the names of NVIDIA and Alphabet. The error tests used one benchmark, FinRank. The authors give no result for users.

Insight: A check on the same data repeats the writer error

In an earlier study, researchers found that finance assistants give a number from the data that answers a different question. Checks against the data accept it. This study shows the same weak point: a check on the same data finds 35%. A check on the same data as the writer is weak, and maybe buyers will need a check on different data. Leaders must tell vendors to show the data that the check reads.

Sources

  1. FinVeriQA: Financial Research Agents for Question Answering: Routing, Retrieval, MCP Tool Use, and Provenance-Aware Evidence Verificationarxiv.org
AI MATTER · NEWS · AI MATTER · NEWS ·20 NOV2026

Posted

Tags

More in Research

All Research news