Blog · Board and management reporting · Customer service
Chat bot analytics compares recorded service outcomes from existing session and support exports. Define resolution, handoff, and repeat-contact measures, calculate category rates, and reconcile the totals before changing a service process.
Chat bot analytics compares recorded resolution, handoffs, and repeat contacts from existing session and support exports. Define the eligible population and each outcome before calculating rates, then inspect category differences without assuming that a session ending means the customer's issue was solved.
This is a service-management exercise. Use the categories and outcome fields already recorded in your files, and keep uncertain records visible.
Start with a session-summary export containing one row per eligible session: session ID, start date, recorded category, terminal outcome, and a repeat-contact flag. A separate support export can substantiate handoffs and qualifying later contacts. Use restricted internal identifiers where matching is necessary; this worked example uses only aggregate counts.
The service owner must agree what counts as resolution. For example, a recorded resolution confirmation can qualify, while a session that simply closes remains unresolved or unknown. Decide how test sessions, duplicate export rows, abandoned sessions, and categories with missing values enter the population.
Use mutually exclusive terminal outcomes for the original chatbot session:
Repeat contact is a separate flag, not another terminal outcome. A session initially recorded as resolved may still lead to another contact. The distinction also matters when comparing this measure with first-contact resolution.
The fictional US customer-service team below reviews September 2026 sessions with a seven-day repeat-contact window. Its export includes follow-up through October 7, 2026, so every included session has the full window available. The repeat flag means at least one later contact about the same issue under the team's documented matching rule.
Do not count every later message as a repeat session. Multiple contacts linked to one original session still produce a single yes flag. If matching evidence is missing, mark the flag unknown and report that coverage gap rather than setting it to no. For a detailed counting example, see repeat contacts on ten tickets.
On a Sessions worksheet, put session ID in A, category in B, terminal outcome in C, repeat flag in D, and eligibility in E. Eligibility is Yes only after the agreed population checks. Repeat flags are Yes or No for the complete records used here.
On Summary, put category names in A2:A3. Enter these formulas in B2 through F2 for eligible sessions, resolved sessions, handoffs, remaining outcomes, and sessions with repeats:
=COUNTIFS(Sessions!$B$2:$B$101,$A2,Sessions!$E$2:$E$101,"Yes")
=COUNTIFS(Sessions!$B$2:$B$101,$A2,Sessions!$E$2:$E$101,"Yes",Sessions!$C$2:$C$101,"Resolved")
=COUNTIFS(Sessions!$B$2:$B$101,$A2,Sessions!$E$2:$E$101,"Yes",Sessions!$C$2:$C$101,"Handoff")
=B2-C2-D2
=COUNTIFS(Sessions!$B$2:$B$101,$A2,Sessions!$E$2:$E$101,"Yes",Sessions!$D$2:$D$101,"Yes")
Microsoft documents COUNTIFS as counting rows where all supplied criteria are met. Fill down through row 3; expand all ranges together for a larger export.
| Category | Eligible sessions | Resolved | Handoff | Unresolved or unknown | Sessions with repeats |
|---|---|---|---|---|---|
| Billing | 60 | 36 | 18 | 6 | 8 |
| Delivery | 40 | 20 | 16 | 4 | 6 |
| Total | 100 | 56 | 34 | 10 | 14 |
The remaining-outcomes column intentionally retains all records that do not match a defined resolved or handoff outcome. Review unexpected labels before interpreting it.
Use eligible sessions as the denominator for resolution and handoffs. Every eligible session here has complete repeat follow-up and matching coverage, so repeats use the same denominator. Enter these formulas in G2 through I2, respectively:
=IFERROR(C2/B2,"Review population")
=IFERROR(D2/B2,"Review population")
=IFERROR(F2/B2,"Review population")
Microsoft's IFERROR documentation explains returning a chosen value when a formula errors. The text flag prevents a category with no sessions from appearing to have a measured zero rate. Covirage is not affiliated with Microsoft.
| Category | Resolution calculation | Resolution rate | Handoff calculation | Handoff rate | Repeat calculation | Repeat rate |
|---|---|---|---|---|---|---|
| Billing | 36 / 60 | 60.0% | 18 / 60 | 30.0% | 8 / 60 | 13.3% |
| Delivery | 20 / 40 | 50.0% | 16 / 40 | 40.0% | 6 / 40 | 15.0% |
| Total | 56 / 100 | 56.0% | 34 / 100 | 34.0% | 14 / 100 | 14.0% |
Rates are rounded to one decimal place. Calculate the total from summed counts, rather than averaging category percentages. The terminal outcomes reconcile: 56 resolved plus 34 handoffs plus 10 remaining sessions equals 100 eligible sessions. The repeat count overlaps those outcomes and must not be added to that reconciliation.
For an Excel check, =SUM(B2:B3)-SUM(C2:E3) returns zero. Also reconcile eligible session IDs to the source export after documented exclusions; the identity alone cannot reveal omitted sessions.
Delivery has a lower recorded resolution rate and a higher handoff rate than Billing. Ask the service owner to inspect recorded handoff reasons and the issues in each category. A transfer required by service policy is different from a missing answer, even when both appear in the same rate.
The repeat rate helps test whether an apparent success held after the session ended. It does not prove why a customer returned. Assign a category owner to review the underlying records and propose a process change, then compare a later complete cohort using the same definitions.
Keep elapsed-time questions separate; average resolution time requires its own start, stop, and unresolved-case rules. Use customer care analytics when the decision spans channels beyond the chatbot.
Bring session summaries and the corresponding support export to Covirage with your outcome and follow-up definitions. Covirage maps the columns; its tools compute and check category comparisons, and the external AI model explains the results with every figure traced to its rows. Create a free Covirage account and bring a sample of your own data; creating the account takes about a minute.
See Covirage for customer-service teams for the supporting business context.
Start with eligible sessions, recorded resolution rate, handoff rate, unresolved sessions, and repeat-contact rate. Define the denominator and observation window for each. Add handling time or satisfaction only when the source export contains a usable field and the population is clearly identified.
No. A session that never reaches an agent may end because the customer abandons it or still needs help. Define resolution using recorded evidence appropriate to the service. Keep unresolved sessions visible rather than treating every session without a handoff as a success.
Choose a follow-up window and a documented way to identify the same service issue. Count each original session once when at least one qualifying repeat appears. Exclude sessions without enough follow-up time from that rate's denominator, and disclose records that cannot be matched.
A handoff may be the appropriate outcome for a complex issue or an authorized service requirement. Compare categories and inspect the recorded reason before changing routing. Evaluate whether the customer reached the right support path rather than assuming every transfer should be prevented.