Blog · AI and self-service analytics
A buyer's checklist for AI sales analytics: where each number comes from, whether the model computes or reads, how definitions are versioned, what the roll-up identity is and whether it is checked, how a wrong answer is caught, what data leaves your building, whether names are needed, what happens when the exports change, what the assistant refuses to do, and what you can take with you. Each with the answer that should worry you.
Every AI sales analytics demo shows a question typed in plain language and a confident answer. The questions below are about what happens between the two, and the answers separate a tool that computes from one that generates.
| # | Ask | A good answer | A worrying answer |
|---|---|---|---|
| 1 | Where did this number come from? | A table, the rows, the period, the definition version | "The AI analysed your data" |
| 2 | Does the model compute or read? | Deterministic code computes; the model reads and cites | "The model is very accurate" |
| 3 | How are definitions versioned? | Written, dated, cited on every output; prior periods marked | "You can configure metrics" |
| 4 | What is the roll-up identity, and is it checked? | Region = team = person = account = ledger, on every load, failures listed | "The dashboard is real-time" |
| 5 | How would a wrong answer be caught? | Citation to the table; identity checks; the assistant declines arithmetic | "Our customers trust it" |
| 6 | What data leaves our building? | Exports you choose, as files; identifiers, not names | "We connect to everything" |
| 7 | Does it need names? | No; identifiers suffice; a pseudonymised export works | "Names help the AI" |
| 8 | What happens when an export changes shape? | Validation fails visibly; the mapping is updated once | "It adapts automatically" |
| 9 | What does the assistant refuse to do? | Compute in prose, estimate what is not in the tables, answer without a citation | "It can answer anything" |
| 10 | Can we take everything it computed? | Yes, every table, on request, in a plain format | "The insights layer is proprietary" |
One and two are the same question from two sides. A number with a table behind it can be checked. A number the model produced cannot, and it will look identical.
Three and four are what make two months comparable. A definition that changed silently, or a roll-up that does not sum, produces a movement that is not one.
Five is the question that reveals whether the vendor has thought about being wrong. Every system is sometimes wrong. The ones worth buying know how they would find out.
Six and seven are the data protection surface. A tool that works on identifiers from files you export has a small one. A tool that connects live and needs names has a large one, and your DPO will ask.
Eight is the operational question. Exports change. A tool that "adapts" silently has guessed at your new column; one that fails validation and asks has not.
Nine is the trust question. An assistant that answers everything is generating some of it. One that declines to add two numbers and points at the total is working correctly.
Ten is the exit question. The measures are yours.
| Vendor | Q1 | Q2 | Q3 | Q4 | Q5 | Q6 | Q7 | Q8 | Q9 | Q10 |
|---|---|---|---|---|---|---|---|---|---|---|
| A | Table cited | Reads | Versioned | Checked | Citations, refusals | Files | Identifiers | Fails visibly | Declines arithmetic | Export all |
| B | "Analysed" | Computes | Configurable | Not mentioned | Trust | Live connectors | Needs names | Adapts | Anything | Proprietary |
Vendor B gave the worrying answer to every question and had the better demo.
Judging on the demo. The demo is the question and the answer; the ten questions are about the middle.
Accepting adjectives. Accurate, powerful, intelligent. Ask for the table.
Skipping question ten. Discovered at renewal.
Asking the salesperson. Ask for the person who built it.
Deterministic code computes from stated, versioned definitions; the assistant reads and cites and declines to compute; the roll-up identity is checked on every upload and the failures are listed; data arrives as files on identifiers; every computed table is yours. The sales intelligence solution describes the setup, and the arithmetic guide is the long answer to question two.
Whether the language model computes numbers or reads them. If the model adds, averages or extrapolates in prose, every figure it produces is plausible and unverifiable. If deterministic code computes and the model reads and cites, the figures can be checked. Everything else follows from that answer.
Because a tool that needs customer names to work has a data protection surface that a tool working on identifiers does not. Coverage, share of wallet and gaps compute on identifiers. If the vendor cannot run on a pseudonymised export, ask why.
That the computed tables are proprietary or only available inside the tool. The measures are yours; they were computed from your data by stated definitions. If you cannot take the tables with you, you have not bought analytics, you have rented a view.