Blog · AI and self-service analytics
A plain explanation of why large language models make arithmetic errors, for people deciding whether to trust AI with business figures. A language model predicts the next piece of text; it does not calculate. It reads numbers as fragments, not quantities, produces answers that look right, and is most likely to be wrong on long numbers, long columns and multi-step calculations, which is exactly what business data involves. This page explains the mechanism without jargon, shows the kinds of error to expect, and sets out the design rule that removes the problem: the model decides what to compute and explains the result, and code does every calculation.
People are surprised that software which writes a legal summary in seconds cannot reliably add a column. It is less surprising once you know what the software is doing.
A language model takes text and predicts what text comes next, one fragment at a time. It was trained on a vast amount of writing, and it has learned, extraordinarily well, what plausible continuations look like.
When the text is "the capital of France is", the plausible continuation is also the true one. When the text is "2 + 2 =", likewise: that string appears everywhere.
When the text is a list of four hundred invoice amounts followed by "the total is", there is no memorised answer. The model produces what a total would look like here: about the right number of digits, a believable leading figure, two decimal places. It is a very good guess at the shape of the answer. It is not a sum.
Numbers are read as fragments. A model does not see 1,284,570.35 as a quantity. It sees a few chunks of characters, split in ways that have nothing to do with place value. Carrying a digit across those chunks is something it has to imitate, not something it does.
Errors compound across steps. Share of wallet needs a sum, another sum, a division and a comparison with a median. If each step is 97 percent reliable, four steps are about 88 percent, and a table of fifty accounts will contain several wrong rows with nothing to mark which.
Length defeats it. Accuracy is high on two small numbers and falls as numbers get longer and lists get longer. Business data is long numbers in long lists.
And one reason the errors are dangerous:
A wrong answer looks identical to a right one. A spreadsheet with a broken formula shows an error. A model with a wrong total shows a confident sentence. There is no visible difference, and the prose around it is just as fluent.
| Kind | Example |
|---|---|
| Near miss | True total 1,284,570; stated 1,284,750. Two digits transposed; passes a glance |
| Right magnitude, wrong value | A plausible 23.4 percent where the answer is 19.8 |
| Dropped rows | A sum over most of the list; long inputs get partially attended to |
| Inconsistent restatement | 412 in one paragraph, 421 in the next |
| Invented precision | A figure to two decimals for something never computed |
| Definition slip | The denominator quietly changes between two answers |
None of these announces itself.
Asking a model to think step by step, or to double-check, improves the odds. It does not change what the model is. A report needs the same answer every time from the same data, and a process that is right 98 percent of the time cannot give that. The remaining 2 percent lands somewhere in the board pack.
Separate the language work from the arithmetic, completely.
| Step | Done by |
|---|---|
| Understand the question; pick the measure and its definition | Model |
| Work out what columns mean; propose the mapping for a person to confirm | Model, then a person |
| Generate or select the query that computes it | Model, or a fixed library |
| Execute the query: every sum, count, ratio, median, join | Deterministic code |
| Check control totals and identities; refuse to show a table that fails | Deterministic code |
| Explain the result, in words, quoting only figures the code returned | Model |
| Attach to every number its source: file, rows, formula, definition version | System |
The model never states a number it did not receive from the engine. If asked for something the engine cannot compute, it says so; see what a refusal should look like. This is the rule this product is built on, set out in why the model must never do the arithmetic and in the deterministic-first architecture page.
The ten questions for an AI analytics vendor extend this into a buying checklist.
A chatbot answers a follow-up from memory. The first figure came from code; the second came from the conversation.
A summary restates the numbers. The table was computed; the paragraph describing it was generated, and one figure in it differs.
Percent changes done in the head. The two values are right; the growth rate between them is not.
Trust transferred from prose to numbers. The writing is excellent, so the figures are assumed to be.
A language model predicts text. It does not calculate, it reads numbers as fragments, its errors compound over steps, and its wrong answers look like right ones. That is a reason to keep it away from the arithmetic, not from analytics: let the model handle meaning and explanation, and let code compute every figure, with checks and a traceable source on each. Covirage is built that way, and the six tests above will show whether any tool is.
Better, and still not reliable in the way a calculator is. Accuracy on short calculations is high; on long numbers, many operands or chained steps it falls, and the failures are silent: the wrong answer looks exactly like a right one. For a report, mostly right is not a usable standard. Modern assistants work around this by calling a code tool, which is the correct approach when it is actually used.
Ask for the code and the rows. A calculated figure comes with a query or script and can be traced to source rows. A generated figure comes with an explanation only. In a well-built tool every number on the screen carries a citation: which file, which rows, which formula, which definition version.
No. Models are very good at the parts of analytics that are language: working out what a column means, turning a question into a precise request, choosing the right measure, explaining a table, noticing what is odd. They are poor at being a calculator. Use each for what it does well.