600 Out of 7,801: The Silent Bug That Booked Jobs on Techs Who Were Off
600 out of 7,801.
That's how many calendar events an AI voice bot was actually reading before it decided a technician was "free."
The other 7,201? Invisible.
No error. No crash. Every single run logged "success." It just quietly returned the wrong answer — for months.
This is the story of a silent bug. They're the scariest kind, and if you run any automation on live data, this is the failure mode you should lose sleep over.
What was the bug — and why did nobody catch it?
The setup: a voice AI books repair appointments over the phone. Before it books anything, it checks the technician's calendar for conflicts — existing jobs, time-off, personal blocks. Standard stuff.
The complaint from dispatch: it kept booking jobs on techs who were already off. No error message. No failed run. The logs were spotless.
Here's what was actually happening:
- The scheduling API held 7,801 events across 40 pages of results.
- The workflow was hard-coded to pull only the first 3 pages — sorted by "last updated."
- So it only ever saw the 600 most recently touched events.
An older time-off block? Not in those 600. Never loaded. Never checked. To the bot, the technician looked wide open — so it booked a job right on top of their day off.
The automation wasn't broken in any way a computer could detect. It did exactly what it was told. It was told the wrong thing.
Why did it work at launch and break later?
Because it broke gradually, which is the trap.
At launch there were fewer than 600 events in the system, so "pull the first 3 pages" happened to return everything. The bot was correct — not by design, but by accident.
Then events piled up. Every month, a few more jobs, a few more time-off blocks, and the 600-event window covered a smaller and smaller slice of reality. There was no moment where it "went wrong." It decayed, one booking at a time, while the logs kept saying "success."
That's the defining trait of a silent bug: it doesn't fail when you write it. It fails when the data grows past an assumption you didn't know you'd made.
Why are silent bugs the most expensive kind?
Because everything you rely on to catch problems is watching for the wrong signal — the crash — and a silent bug never crashes.
The numbers here are brutal. Gartner estimates poor data quality costs the average organization $12.9 million a year, and MIT Sloan research puts revenue lost to bad data at 15–25% annually. Almost none of that comes from software that fell over. It comes from software that kept running and confidently produced wrong output.
AI makes this worse, not better. On realistic conversation benchmarks, even the strongest current models hallucinate around 30% of the time, and in one large study 36 of 40 models were more likely to give a confident wrong answer than a correct one when the question got hard. A crash announces itself. A confident wrong answer looks exactly like a right one until someone checks.
Here's the uncomfortable part — standard monitoring catches almost none of it:
| Failure type | Throws an error? | Caught by standard monitoring? | How you actually catch it |
|---|---|---|---|
| Crash / exception | ✅ Yes | ✅ Yes | Alerts fire on their own |
| Timeout / rate limit | ⚠️ Usually | ⚠️ Often | Shows up in logs |
| Silent wrong answer (this bug) | ❌ No | ❌ No | Independent re-check against the source |
| AI hallucination | ❌ No | ❌ No | Verification layer or human review |
The two failure modes that hurt most are exactly the two your dashboard can't see.
How do you catch a silent bug?
You stop trusting the automation's own report and verify its output against the source of truth — separately.
Finding this one wasn't clever debugging. It was a process:
- Pull the complaint — "it booked on a tech who was off." Treat it as data, not noise.
- Find the exact run — the specific booking, the specific timestamp.
- Open the node — read what the workflow actually requested, not what it was supposed to request.
- Compare against the live calendar — 7,801 events in the system, 600 in the bot's view. There's your bug.
The fix itself was small: paginate through all 40 pages, or filter the API call by date instead of relying on "last updated" order. Five minutes of code. Finding it was the real work — and it only got found because a human went looking for the gap between what the bot said and what was true.
What should you actually monitor?
Monitor outputs, not just uptime. The rules I now build into every automation:
- Never take success from the bot's own logs. A separate script re-checks results against the source system until the discrepancy count is zero.
- Test with production-scale data, not launch-day data. The bug that works on 600 records and breaks on 7,801 will never show up in a demo.
- Watch for silent drift. Bookings on unavailable techs, "done" tasks that aren't done, totals that don't reconcile — these are the smoke.
- Keep a human on anything irreversible. The model does the work; the human signs off on the parts you can't undo.
This is the same philosophy behind everything I ship: the AI is a talented junior, and juniors get code review. It's not that you distrust the worker — it's that you verify the work.
The lesson I keep relearning
The scariest bugs don't throw errors. They return confident, wrong answers.
Monitor what your automations output — not just whether they crash. A green dashboard is not the same thing as a correct one.
What's the most expensive silent bug you've run into?
I build and audit supervised AI automations for home service businesses — voice bots, dispatch tools, lead recovery — with verification baked in so they fail loud instead of failing silently. If you've got an automation you're not 100% sure you can trust, let's talk.
Related:
- I Stopped Chasing Startup Ideas and Started Collecting Business Pains
- 3 AM, Database Down, AI Bot Dead: What I Learned From My Worst Night in Automation
- I Analyzed 100 Service Business Calls With AI. What I Found Was Uncomfortable.
- You Don't Have a Lead Problem. You Have a Conversion Problem You Can't See.