Four hundred transcripts is not an improvement on four hundred phone calls. If the output of automation is a folder of documents nobody reads, you have moved the bottleneck rather than removed it.
The rating template is what closes that gap. It is a rubric applied to every call the moment it ends, producing the same columns with the same meaning for every candidate — so four hundred conversations become a table you can sort.
What a template is made of
Two things, and the second matters more than people expect.
Column headers — one per line, each becoming a column in the ratings table. For a technical screen: Self Intro, Communication, ReactJS, NodeJS, Overall Fit.
The prompt — free-text instructions to the model doing the scoring. This is where the rubric actually lives, and a vague prompt produces scores that look precise and mean nothing.
Designing columns that survive contact
Score things the call can actually observe. "Communication" works — the transcript is evidence of it. "Culture fit" does not; nothing in a five-minute screening call supports that judgement, and a number against it is false precision that will mislead whoever reads the table.
Keep it to five or six. Twelve columns is a form, and nobody scans a twelve-column table. Every column should be one you would actually filter on.
Separate the hard filters from the judgements. Notice period and location are facts — extract them as fields, do not score them. Reserve the rating columns for things that genuinely need an assessment.
Writing the prompt
The difference between a useful rubric and a random number generator is whether the prompt says what each score means. Compare:
Rate the candidate on each column from 1 to 5.
against
You are evaluating a backend screening call. Score each column 1–5 based only on what the candidate actually said. 5 = clear, specific, evidenced with examples. 3 = answered but general, no specifics. 1 = did not answer or contradicted themselves. If a topic never came up, score it 1 and say "not discussed". Keep every reason under 25 words.
The second produces scores two different reviewers would agree with. Three rules do most of that work:
- Anchor the ends. Say what a 5 and a 1 look like. Without anchors everything drifts to 3 and 4.
- Say what to do with silence. Otherwise the model guesses — sometimes charitably, sometimes not, and never consistently.
- Cap the reason length. A 25-word reason is scannable. A paragraph is another transcript.
Calibrating it
Do not trust a new rubric on 400 calls. Run it on ten you have already read and formed an opinion about, and compare. Where the rubric disagrees with you, one of you is wrong — and about half the time it is the rubric's prompt, which you can fix and re-run.
Re-scoring is cheap and non-destructive: change the template, regenerate, and the existing calls are scored again with the new rubric. It costs nothing to iterate before you commit to a batch.
Using the output
Sort by overall, read the top thirty, and spot-check ten from the middle band. The middle is where a rubric earns or loses your trust — the extremes are usually obvious to anyone.
What you should never do is treat the score as a decision. It is a reading order. The point is that your best recruiter reads the thirty most promising transcripts first instead of dialling four hundred numbers to find them.