Set pass-fail standards for calls and WhatsApp triage

Automated call grading software is only useful when the rubric defines what a good conversation looks like in observable terms. “Be helpful” is not a test. “Confirm the appointment date before ending the call” is. The same principle applies to WhatsApp conversation QA rubrics: score what the thread shows, not what a reviewer guesses the agent intended.

Start with one narrow triage task, such as answering an inbound service question or qualifying a new lead. Write the rubric before looking at scores. That reduces the temptation to excuse a failure because the conversation sounded polished, or to mark a pass because the outcome happened to be favorable.

1. Pick the job and the evidence

Define the conversation type first. A call that qualifies a lead and a WhatsApp thread that confirms an appointment may need different resolution checks, even if both share policy and tone standards. Keep the first rubric small enough that a supervisor can explain every item without a manual.

For each criterion, identify evidence a reviewer can find in the recording, transcript or message thread. “Did the agent verify the caller’s identity before discussing account details?” is testable. “Did the agent show good judgment?” is not, unless you spell out the behaviors that count.

Use a simple structure for each check: the required behavior, the evidence that proves it, and the condition that makes it fail. For example, a policy check might require the agent to avoid promising an unconfirmed refund. The evidence is a promise in the conversation; the failure condition is any statement that guarantees the refund before approval.

2. Separate policy, tone and resolution

Build three categories. Policy covers required steps and prohibited claims. Tone covers behaviors customers can hear or read, such as acknowledging the question and avoiding dismissive language. Resolution checks whether the conversation reached the next agreed step: an answer, a handoff, a booked appointment or a clear follow-up plan.

Keep serious policy failures visible. Do not let several successful tone checks average out a privacy breach or an unauthorized commitment. Mark critical checks as pass or fail, and make the overall result fail if any defined critical check fails. For noncritical checks, a separate tally can show patterns without disguising a high-risk miss.

For calls, specify what must happen in spoken conversation, including any required confirmation before the call ends. For WhatsApp, define what counts across a thread: a response may arrive in more than one message, so judge the complete exchange rather than treating each text as an isolated answer. If voice and messaging use different approved language or steps, keep those criteria channel-specific.

3. Make “pass” and “fail” reproducible

Write examples for the edge cases. If a customer asks a question the agent cannot answer, does a clear handoff count as resolution? If the customer leaves before confirming an appointment, should the agent pass for offering a next step? State the rule. Otherwise two reviewers—or an automated evaluator and a manager—can apply different standards to the same exchange.

Then test the rubric on a small, varied set of real interactions. Include clear successes, clear failures and awkward cases: interruptions, incomplete requests, corrections and channel changes. Have a manager compare the grade with the conversation evidence. Track disagreements by criterion, not just by total score. When reviewers disagree, tighten the wording or add an example before broadening the rollout.

This is also where human review earns its keep. An automated grade can make routine sampling more consistent, but a surprising fail or a borderline case deserves a look at the underlying interaction. Kulissa’s account of replacing subjective manager reviews with rubric scoring is relevant here: the rubric, not the aura of objectivity around automation, is what makes scores comparable.

4. Use the result to improve the workflow

Review failures by category and by outcome. Repeated misses on a required disclosure suggest a different fix from repeated failures to book the right appointment. Check whether the instruction is unclear, the approved information is missing, or the workflow leaves the agent without a reliable next step. Do not revise a criterion simply because it produces an inconvenient result.

For a team handling calls and WhatsApp, preserving context matters when investigating a failure. Voicetta describes its product as handling voice, SMS and WhatsApp conversations in a shared timeline, and says teams can define behaviors for grading calls and WhatsApp threads as pass or fail, with a reason. Its setup is done-for-you with a specialist, so a team can bring the rubric and channel-specific examples into that setup rather than assuming a particular configuration screen or control.

Keep a record of the rubric version, the examples used to test it and the reasons for any changes. That gives managers a way to distinguish a genuine performance trend from a changed standard. For related operational context, the practical case for reliable conversation records and handoffs is worth considering alongside grading.

5. Keep the rubric small and reviewable

Begin with the few behaviors that affect safety, accuracy and whether the customer gets a clear next step. Add criteria only when reviews reveal a meaningful gap. A long checklist can make a score look precise while obscuring the failures that matter most.

The working standard is simple: every pass or fail should point to evidence, every critical failure should remain visible, and every rubric change should be explainable. That makes AI conversation performance grading useful to the people managing triage—not merely another number in a dashboard.