We set up a receptionist for a fictional London firm, “Brightwater Plumbing & Heating”, on xAI’s realtime voice API, and called it 25 times with five scripted callers: a callout booking, a refund that needs approval, an angry customer whose plumber didn’t show, someone leaving a callback number, and a legal question it shouldn’t answer. Then we ran 20 more calls comparing Grok Voice 2.0 and 3.0 head to head. Every transcript, audio file and timing is published.
What it got right
- It never acted without a yes. No booking, refund or message was saved before the caller confirmed, apart from one angry-caller run (1 of 5) where it saved a message before the caller confirmed the read-back.
- Phone numbers: 10 of 10 correct, each read back digit by digit first.
- Out of scope: 5 of 5 declined the legal question politely and pointed to a solicitor or Citizens Advice, with no tool calls.
- Refunds: it told the caller the refund needed approval every time, instead of promising money.
What it got wrong: names
18 of 25 calls passed our strict checks. All seven failures had the same cause: a surname saved wrong. “Clarke” became “Clark” in 5 of 5 angry-caller runs. “Shah” became “Shaw” in 2 of 5. It never asked the caller to spell either name.
That isn’t a 3.0 regression. In our head-to-head runs, both 2.0 and 3.0 wrote “Clark” in every angry-caller run.
For a UK business this matters more than it sounds. Surnames like Clarke, Shah, Reid or Thomson have common sound-alike spellings. If your CRM or job system matches customers by name, a confident misspelling creates a duplicate record or a missed match. The fix is cheap: tell the agent to ask for the spelling of any surname it’s about to save, and match on phone number first.
Latency: close to a second, not under it
The chart above shows every turn’s time from the end of the caller’s audio to the first byte of the agent’s reply. Across 54 turns on 3.0, the median was 1.11s, and 23 of 54 came in under a second. The first turn of each call was slower (median 1.41s) than later turns (0.87s).
Head to head, 3.0 was not faster to first audio than 2.0 on our calls: medians of 1.00s and 0.97s across 22 turns each. That’s within noise. 3.0 did finish turns sooner (2.77s vs 3.34s median), mostly because its replies were shorter. xAI’s “40% lower latency” claim compares 3.0 with a different model, not with 2.0; we’d assumed otherwise at first and corrected it.
Caution has a cost too
In 4 of 10 booking and refund calls, the caller said “Yes, please go ahead” and the agent read the details back again, acting only on a second yes. That’s safe, and on our scoring it still counts as resolved. On a real phone line it’s an extra turn for a caller who has already said yes.
What we’d tell a UK business trying one
- Make it ask for the spelling of surnames, and match on phone number.
- Budget for about a second of silence after the caller stops, more on the first turn, and more again on a real phone line with codecs and voice-activity detection (our test sent clean audio).
- Write the confirmation rule once, clearly, so it acts on the first clear yes, and test it with your own scripts.
- Keep refunds and anything involving money as “pending approval” in the tool itself, as we did.
Limits
45 calls, synthetic text-to-speech callers, clean audio, no line noise or interruptions, one UK network location, run on 15 Oct 2026 (BST). Treat this as a careful spot check, not a ranking. Cost: about $1.27 for the 25 calls and $1.07 for the comparison, on an audio-minutes basis (ledger).