The expensive mistake in automated calling is not a bad call. It is four hundred identical bad calls, discovered afterwards, on a list you cannot call again.
Twenty minutes of deliberate testing prevents almost all of it. Not "does it work" testing — it will work. Testing for the specific ways it goes wrong.
Test as a difficult person, not a cooperative one
The instinct is to answer clearly and in order. Nobody does that on a real call. Run these, each as its own test call:
- Interrupt it. Talk over the greeting after about a second. It should stop mid-word, not finish the sentence.
- Answer vaguely. "A few years, I guess." It should ask once for a specific number and then move on — not loop.
- Ask a question back. Salary, location, process. It should answer from the knowledge base, or defer cleanly. It should never invent.
- Ask something it cannot know. "Who is the hiring manager?" A confident wrong answer here is the most damaging failure there is.
- Go silent. Say nothing for ten seconds. It should prompt once and eventually end, not sit there billing minutes.
- Say you are busy. It should offer a callback and end politely, not push on.
- Say no. "I'm not interested." It should accept it. One clarifying question at most.
- Switch languages mid-sentence, if your callers do. This is where transcription quietly falls apart.
Then read the transcripts
Not the audio — the transcripts, because that is what your ratings and extractions are built on. Check three things:
- Did it ask everything the prompt lists? Missing questions are the most common real defect, and they are invisible if you only read your own answers.
- Are the fields you filter on present, in any spelling?
- How long are its turns? Anything over two sentences means the person waited fifteen seconds. Fix it in the prompt.
Then run five, not four hundred
Take five real people from the list and call them. Real people fail in ways you will not think to simulate — bad signal, a child in the background, a wrong number that is now someone else's.
Read all five transcripts. If four are good, run the batch. If two are odd in the same way, you have found a prompt bug that would have repeated four hundred times.
The economics
Twenty test calls cost a few hundred rupees. A four-hundred-call batch that has to be discarded costs the money, the list, and the week. The ratio is not close.