AI basics: Monitoring for automations
Every automation needs one place where you can see whether it delivered, whether it was uncertain, and whether a human still had to rework the output. Three metrics are enough to start: runs without usable output, uncertain assignment, and rework before acceptance. These are counting metrics for your own process, not benchmark targets from an industry report.
Why monitoring matters
A run that silently produces nothing, or that turns a data gap into a fact, is usually discovered later than a visible error. In appointment analysis, a property address that is not confirmed by transcript or calendar returns as “unclear, please verify.” During the documented test phase, nothing was written to CRM. If appointment type is uncertain, a human checks instead of silently continuing.
In image post-processing, a photo that fails automated checks three times is delivered with a verification marker. The marker appears in status view, result email, and filename. The original stays next to it. This is monitoring in practice: uncertain or failed outcomes stay visible instead of disappearing in a folder.
Without that visibility, you only see that a tool is switched on. Whether it actually saves work is reflected in rework. The prioritization grid for your first process includes control, correction, and maintenance in remaining effort. Monitoring provides the numbers for exactly that calculation.
For CRM processes, monitoring adds the economic view: the article on CRM setup costs for agents explains why data quality and ongoing maintenance are the dominant cost blocks.
The three metrics
Count one week, or a fixed batch of runs. A run is one job accepted by your automation: one recording, one image, one dataset. The three metrics are intentionally non-overlapping.
Three counts, no overlap
Each metric tracks a different remainder. No target threshold.
- 01No usable outputAbort or verification marker
- 02Uncertain assignmentGaps are not guessed
- 03ReworkChange before acceptance
| Metric | What you count | What an increase means in your setup |
|---|---|---|
| Runs without usable output | Aborted jobs, stuck jobs, and results returned only with a verification marker | The flow is not delivering, and without the metric nobody sees it |
| Uncertain assignment | Cases where contact, property, or appointment type is not evidenced and therefore not guessed | A gap would otherwise become an entry that only looks correct |
| Rework before acceptance | Drafts a human still edits before CRM, email, or publication | Control effort can consume the expected efficiency gain |
We do not set a percentage threshold here. You define that after counting the same process for a while using a fixed method. A marker like “unclear” is not a metric failure. The failure would be not having that marker.
Example calculation, not a measured result
Assume one week has 20 runs. Two end without usable output, four are marked as uncertain assignment, and six drafts are edited before acceptance. The three shares are 2 of 20, 4 of 20, and 6 of 20.
This is an example calculation. It is not measured customer data and not a target.
Interpretation: two failures need a root-cause check, otherwise silent loss remains invisible. Four uncertain assignments are acceptable if they stay marked and nobody accepts them as final CRM state. Compare six reworks with the time the same step took manually before. If correction takes roughly as long as the old workflow, this run is not yet relieving work. Then the metric is a trigger to adjust the process, not a reason to add a fourth chart.
One review per week
Once a week, the three numbers, one concrete failure example, and one decision are enough. The decision has three outcomes: keep as is, tighten checks, or pause until assignment is clear.
Three outcomes
One decision per week, not another dashboard.
- 01KeepRework stays shorter
- 02TightenDo not accept plausible-only data
- 03PauseRework replaces the intended savings
Keep fits when failures are rare, uncertain cases stay marked, and rework is clearly shorter than the previous method. Tighten fits when incorrect cases slip through, for example an address that only sounds plausible. Pause fits when rework replaces the old effort instead of reducing it. More views do not change the decision. They only delay it.
Frequently asked questions
Which alerts should I activate first?
Start with alerts for failures, uncertain assignment, and high rework. These three signals show early whether the flow is stable or silently losing quality.
How often should a team review monitoring metrics?
A fixed weekly rhythm is enough to start. What matters is that each review ends in a decision: keep, tighten checks, or pause the flow.
Are many uncertain assignments always a problem?
Not automatically. Uncertain cases are fine if they stay visibly marked and are not accepted unchecked. It becomes a problem when uncertainty is treated as final state.
How does monitoring connect to CRM data quality?
Monitoring shows where data gaps appear before they become seemingly clean CRM facts. This protects downstream workflows from chain errors.