Blog · AI and self-service analytics
A practical answer to whether a general-purpose AI chatbot can analyse a sales export. It can: given a clean file and a precise question, a chatbot with a code tool will compute correct totals, rankings and simple rates. It goes wrong in predictable places: silent assumptions about what columns mean, duplicated customers, joins across files, definitions that change between sessions, arithmetic done in prose rather than in code, and no check against the ledger. This page lists what works, the seven failure points, five checks anyone can run on an answer, and what a purpose-built tool does differently.
A sales manager uploads an export to a chatbot and asks who the top customers are. The answer is right, fast and nicely explained. The interesting question is when it stops being right.
Given one clean file and a precise question, a chatbot with a code tool does these reliably:
| Task | Why it works |
|---|---|
| Totals, averages, counts by one column | One file, one operation, no definitions needed |
| Top and bottom rankings | Sorting is unambiguous |
| Month-by-month trend | If the date column is clean |
| Simple ratios: margin percent, share of total | Both parts in the same file |
| Charts | Drawn from computed data |
| Explaining a table in plain language | What language models are best at |
| Writing a spreadsheet formula or a query | Usually correct; test it |
| Spotting oddities: negative quantities, blank fields, outliers | Useful first-pass data review |
For a one-off look at a single export, that is a great deal of value for ten minutes.
1. Silent assumptions about columns. Is amount gross or net? Is each row an order or an order line? Does the date mean ordered, shipped or invoiced? The chatbot picks one and proceeds. The arithmetic is right; the answer may not be.
2. Customer identity. Acme Ltd, ACME Limited and Acme Group UK are three customers to a chatbot. Concentration is understated, dormancy overstated, and the top-ten list is wrong. It has no customer master and no parent mapping unless you supply them.
3. Joins across files. Ledger to CRM, quotes to orders, activities to accounts. Joins on names fail quietly; joins on the wrong key multiply rows, and revenue doubles. This is where most wrong answers come from once analysis goes beyond one file.
4. Arithmetic in prose. Asked a follow-up, a chatbot may answer from the numbers it sees in the conversation instead of re-running code. Those figures are generated text and can be wrong in the last digits or entirely. See why the model must never do the arithmetic.
5. Definitions that drift. Ask for win rate on Monday and again on Friday and the denominator may differ: decided deals one day, all created the next. Nothing pins the definition, so the number cannot be compared with itself. See versioned metric definitions.
6. No reconciliation. It does not know the ledger total for the period, so it cannot notice that the export is missing a week or that a filter dropped two thousand rows.
7. No memory of the work. Next month the file is uploaded again and the mapping, the cleaning and the definitions are redone, slightly differently. Trends across months are trends in the method as much as in the business.
Add practical limits: file size caps, time limits on code, and very wide or multi-sheet workbooks that get partially read.
And before any computation: ask it to list its assumptions about each column and wait for the list. The guide to asking an analytics assistant covers how to phrase questions so they have one answer.
| Use a general chatbot for | Do not rely on it for |
|---|---|
| A quick look at a new export | Figures that go to a board or a customer |
| Drafting formulas and queries | Anything joining two or more files, unchecked |
| Explaining a table you already trust | Measures that must mean the same thing every month |
| Finding data problems | Customer-level measures without a clean customer master |
| Drafting the commentary for a report | The numbers in the report |
The difference is not a smarter model. It is everything around the model.
| Need | General chatbot | Purpose-built analytics |
|---|---|---|
| Column meaning | Guessed per session | Mapped once, confirmed by a person, saved |
| Customer identity | Names as they come | Identifier map and parent roll-up, maintained |
| Definitions | Improvised | Fixed, versioned, stated on every table |
| Arithmetic | Sometimes code, sometimes prose | Always code; the model never computes |
| Checks | None unless asked | Control totals and identities on every table; a failing table is not shown |
| Repeatability | Each session starts again | Same files, same result, every period |
| Output | An answer | Ranked lists with owners, each figure traceable to rows |
The deterministic-first architecture page sets out how these pieces fit, and the vendor questions page turns them into a checklist for evaluating any tool, including this one.
A general chatbot is a quick, articulate analyst with no memory, no customer master, no definitions and no habit of checking its totals. Use it for a first look and for drafting; check anything that matters with the five tests; and do not let it join files or define measures unsupervised. Covirage uses a model to read files and explain results, and code to compute every figure, with the definitions fixed and the checks run on every table.
When the chatbot writes and runs code, the arithmetic is as reliable as the code, which you can ask to see. When it answers from reading the numbers in the conversation, it is not: language models predict plausible text, and a plausible-looking total can be wrong. Always ask whether the figure was computed with code, and ask for the code.
A confident, well-written answer built on a silent assumption: that the amount column is net of credits, that each row is one order, that a blank region means head office. The arithmetic is right and the answer is wrong. A human analyst would ask; a chatbot usually proceeds. Ask it to list its assumptions before it computes.
Check your company's policy and the tool's data terms first. Consumer versions of chatbots may use conversations to improve their models unless that is switched off; business versions generally commit not to. Customer names, prices and volumes are commercially sensitive, and in many cases personal data is present in contact fields. Remove what the question does not need.