Sign in

Blog · AI and self-service analytics

Setting the threshold per measure so the weekly digest is not noise

How to decide what counts as a movement worth reporting for each measure in a weekly sales digest: thresholds from the measure's own week-to-week variability rather than a round number, the floor that stops small denominators generating alerts, the cap on items per digest, and the review that tightens or loosens each threshold from how many alerts were acted on.

The short answerA movement is worth reporting when it is larger than the measure normally moves week to week. Set each threshold from the measure's own trailing variability, two standard deviations of its weekly change say, with a floor on the denominator so that a rep with four accounts does not generate a coverage alert when one is touched, and a cap on items per digest so that the reader gets the ten largest movements and not eighty. Review quarterly: thresholds on measures whose alerts were never acted on are loosened, and the ones that missed something are tightened.

A weekly digest that lists eighty movements is not read. One that lists none is not trusted. The difference is the threshold per measure, and most digests set one number for everything. This guide sets out thresholds from each measure's own variability, the denominator floor, the cap, and the quarterly review.

The rule

Per measure:

Weekly change = this week − last week Variability = standard deviation of weekly change, trailing 26 weeks Threshold = k × variability, k stated, usually 2 Report if |weekly change| > threshold and denominator ≥ floor

Per digest:

At most n items, largest movements first, n stated, usually 10

The rows you need

  • Measure history: measure, level (rep, region, company), week, value, denominator.
  • Digest log: item, week, measure, acted on.

A worked threshold table

Measure Level Variability Threshold at k=2 Floor Alerts last quarter Acted on
Coverage Rep 2.1 pts 4.2 pts 10 accounts 31 74%
Pipeline coverage Rep 0.4× 0.8× 5 opps 58 22%
Dormant run rate Region $18,000 $36,000 9 89%
Share of wallet Segment 0.9 pts 1.8 pts 20 customers 14 36%

Pipeline coverage generated the most alerts and a fifth were acted on. Its k goes to 3 next quarter. Dormant run rate generated few and almost all were acted on; its k could come down to 1.5 to see whether the extra items are as useful.

The cap

Ten items. When more than ten pass their thresholds, the ten largest in value terms are shown and the rest are counted: "and 14 smaller movements". The reader can open them. The digest stays readable.

The review

Quarterly, per measure:

Acted on Change
Over 70% Loosen k slightly; the measure may be under-reporting
30% to 70% Hold
Under 30% Tighten k; the measure is noisy at this threshold

And a check the other way: movements that were not reported and turned out to matter, found from the account list after the fact. Those tighten the threshold on that measure.

Where it goes wrong

One threshold for all. Silent on the stable measures, constant on the volatile ones.

No floor. The four-account rep is on the digest every week.

No cap. Eighty items; none read.

No review. The thresholds set in month one are the thresholds forever.

Movements without a citation. A threshold breach that does not name the table and row is an alert, not an insight.

Every week, ten movements

Mapped once, the measure history produces the variability, the thresholds and the capped digest every week, and the digest log produces the review every quarter. Covirage builds this from the tables it computes. The sales insights solution describes the setup, and the what makes a change an insight guide covers what the digest says about each movement once the threshold has decided it belongs.

Questions people ask

Why not a fixed threshold like five points?

Because five points is a large move in a stable measure and noise in a volatile one. Coverage per rep might move two points in a normal week; pipeline coverage might move twenty. One threshold makes the digest silent on one and constant on the other.

What is the denominator floor?

A minimum count beneath the ratio before a movement can be reported: ten accounts, twenty tickets, five opportunities. Below the floor, the measure is shown but not alerted, because one event moves it by a large share.

How is 'acted on' measured?

By a mark on the digest item, opened or dismissed, or by whether the account on the item had a logged activity in the following week. The share of items acted on, per measure, is the digest's own coverage figure, and it drives the review.