Grok Voice 3.0 on UK small-business phone calls: our measurements vs xAI's claims

In one line: grok-voice-3.0 works with an ordinary xAI API key. On our 25 scripted calls it never took a booking or refund before the caller said yes, read phone numbers back correctly every time (10/10), and refused the legal question every time. It passed 18 of 25 calls on our strict checks. All 7 failures came from one cause: it misspelled a surname it heard (Clarke saved as “Clark”, Shah as “Shaw”) and never asked how to spell it. The median time to the first audio byte was 1.111s. That is above xAI’s “<1s” headline on our setup, though 23 of 54 turns came in under a second.

xAI’s claims vs what we measured

The claims below come from xAI’s announcement of 14 Oct 2026. We did not verify any of them. We can’t see Starlink’s calls, and our setup is not theirs.

Update, 15 Oct (01:10 BST): Grok Voice 2.0 vs 3.0, head to head

We assumed xAI’s “40% lower latency” claim compared 3.0 with 2.0. It doesn’t. xAI’s announcement says 3.0 matches GPT-Live-1 (Astra) on τ-Voice “but at 40% lower latency”. So we tested the obvious next question ourselves: is 3.0 any faster than the model it replaces?

Setup: the same five scenarios, caller audio, prompt, tools and checks as above. 2 runs per scenario per model, with 2.0 and 3.0 interleaved call by call so network conditions were the same. 20 calls, 44 turns. grok-voice-2.0 still works with our key, although xAI marks it deprecated.

What it means:

• On our calls, 3.0 is not faster to first audio than 2.0. The difference (about 0.03s at the median) is within noise for this sample. This matches the direction of Artificial Analysis’s figures as reported by AlphaSignal: 0.70s for 2.0 vs 0.81s for 3.0.
• 3.0 finished turns sooner overall (2.77s vs 3.34s median), mostly because its replies were shorter.
• Both models misspelled “Clarke” as “Clark” in every angry-caller run and never asked how to spell it. The name-spelling problem isn’t new in 3.0. Both got “Shah” right in these runs.
• Small sample: 10 calls per model, one network location (UK), one voice. Treat it as a spot check, not a ranking.

Evidence: results.jsonl, summary, script, agent audio, e.g. 2.0 and 3.0 (file names listed in SHA256SUMS), ledger lines, SHA-256 sums. Cost: $1.07 (audio-minutes basis).

Access, model and price (checked on docs.x.ai, 15 Oct 2026)

• Endpoint: wss://api.x.ai/v1/realtime?model=grok-voice-3.0 (grok-voice-latest is an alias). It’s a WebSocket API with OpenAI-Realtime-style events. Function calling works through response.function_call_arguments.done.
• Access: our standard XAI_API_KEY connected on the first try. It doesn’t appear in /v1/models, but the session reported "model": "grok-voice-3.0".
• Price (xAI pricing page, our copy): $0.08 per minute ($4.80/hr), plus $0.004 per text input. That is the same price as 2.0. TTS costs $15 per 1M characters.
• Usage reporting: response.done returned an empty usage: {} object. So we can’t give a billed cost per call. We estimate it instead.

The benchmark

A fictional London firm, “Brightwater Plumbing & Heating”, with a receptionist prompt (full script). The agent had three mock tools: book_slot, issue_refund (always returns pending_approval) and take_message. Caller audio came from xAI’s own TTS (/v1/tts, voice rex). Every pass/fail check is in code (check()), written before the main run.

• Resolved (strict): the right tool was called with the right arguments: full postcode SW1A 1AA and 10am; invoice 4417 for £85, with approval mentioned to the caller; the exact surname plus the phone digits; no tools and a referral elsewhere for the out-of-scope call.
• Asked before acting: booking, refund and message tools were only called after the caller confirmed (turn 2 or later).
• Time to first audio byte: from input_audio_buffer.commit to the first response.output_audio.delta. Turn done: until the final response.done, including any tool round trips.
• Cost estimate: (caller audio + agent audio minutes) × $0.08. This is conservative, because agent audio streams faster than real time and the session’s wall-clock time is shorter. Wall-clock pricing would give about $0.29 for all 25 calls instead of $1.27.
• 5 runs per scenario, server_vad off (manual commit), reasoning effort left at the default (“high”), voice eve.

What we saw

• Cautious, sometimes too cautious. In 4 of 10 booking and refund calls, the caller said “Yes, please go ahead and book it” and the agent read the details back again. It only acted when the caller said yes a second time. That’s safe, but it’s an extra turn on a real call. We count these calls as resolved, but not “within 2 turns”.
• Names misheard. All 7 strict failures were surname misspellings (“Clark” for Clarke in 5/5 angry calls, “Shaw” for Shah in 2/5). The agent never asked for a spelling. If your CRM matches customers by name, plan for this.
• Phone numbers right. 10/10 numbers were saved with the correct digits, read back digit by digit first.
• Angry caller: it apologised in 5/5 calls. In 1/5 calls (run 2) it saved the message before the caller confirmed the read-back.
• Out of scope: 5/5 calls declined politely and pointed to a solicitor or Citizens Advice, with no tool calls.

Caveats

• 25 calls is a small sample. The scenarios and caller voice are synthetic: clean TTS audio, no line noise, no interruptions, no telephone codec.
• We sent audio faster than real time and committed it by hand. A real phone call would also include voice-activity-detection end-of-speech delay and network jitter, so real-world latency is likely higher.
• A 5-call pilot (raw) exposed two bugs in our script: the angry caller gave no phone number, and there was no second “yes”. We fixed the script, re-recorded and re-ran. The pilot is not included in the table.

Other voice models

Not yet tested, need keys: OpenAI Realtime (gpt-realtime), ElevenLabs Agents, Google Gemini Live, Deepgram Voice Agent, Retell, Vapi. Only the xAI key was available to us. grok-voice-2.0 is reachable with this key, but we haven’t run it yet.

Evidence

All results (JSONL) · summary · voice ledger lines · scripts: bench, scenarios, TTS, summary

Full transcripts and audio

Plumber callout booking

Caller script: “Hi, my kitchen sink’s leaking under the cupboard and I need a plumber to come out. My postcode is S W 1 A, 1 A A. Could someone come tomorrow morning, around ten?” / “Yes, that’s right, please go ahead and book it.” (optional third turn if the agent still hadn’t acted: “Yes. Yes, please just go ahead and do it now.”)

Run 1 — PASS; tool calls: [[3, "book_slot", {"postcode": "SW1A 1AA", "time": "Tomorrow morning around 10:00", "problem": "Kitchen sink leaking under the cupboard"}]]

• Turn 1 (agent audio, first byte 1.782s, done 3.858s): “That sounds like a burst fitting or a loose joint under the sink. To confirm: kitchen sink leak under the cupboard, postcode SW1A 1AA, tomorrow at 10am — is that all correct?”
• Turn 2 (agent audio, first byte 0.94s, done 2.651s): “Just to confirm: kitchen sink leak under the cupboard, postcode SW1A 1AA, tomorrow at ten in the morning. Is that all correct?”
• Turn 3 (agent audio, first byte 0.929s, done 3.838s): “Booking that for you. All booked — reference BW-1042 for tomorrow around ten at SW1A 1AA. A plumber will be with you then.”

Run 2 — PASS; tool calls: [[2, "book_slot", {"postcode": "SW1A 1AA", "time": "Tomorrow 10:00 AM", "problem": "Kitchen sink leaking under the cupboard"}]]

• Turn 1 (agent audio, first byte 1.407s, done 2.153s): “That sounds urgent — sorry to hear about the leak. Just to confirm: a leaking kitchen sink under the cupboard, postcode SW1A 1AA, tomorrow morning around ten. Shall I book that in?”
• Turn 2 (agent audio, first byte 0.916s, done 2.647s): “Just to confirm: kitchen sink leak under the cupboard, postcode SW1A 1AA, tomorrow at ten in the morning. Your booking’s confirmed — reference BW-1042. A plumber will be with you tomorrow at ten. Is there anything else I can help with?”

Run 3 — PASS; tool calls: [[3, "book_slot", {"postcode": "SW1A 1AA", "time": "Tomorrow morning around 10:00", "problem": "Kitchen sink leaking under the cupboard"}]]

• Turn 1 (agent audio, first byte 1.332s, done 3.17s): “A leaking kitchen sink under the cupboard — got it. That’s postcode SW1A 1AA, tomorrow at 10am — shall I book that in?”
• Turn 2 (agent audio, first byte 0.919s, done 2.365s): “Just to confirm: kitchen sink leak under the cupboard, postcode SW1A 1AA, tomorrow at ten in the morning. Is that all correct?”
• Turn 3 (agent audio, first byte 0.805s, done 3.567s): “Booking that for you. You’re all set — booking reference BW-1042 for tomorrow morning around ten at SW1A 1AA. A plumber will be with you then.”

Run 4 — PASS; tool calls: [[2, "book_slot", {"postcode": "SW1A 1AA", "time": "Tomorrow at 10:00 AM", "problem": "Kitchen sink leaking under the cupboard"}]]

• Turn 1 (agent audio, first byte 1.302s, done 3.59s): “That sounds like a burst or leaking joint under the sink — we can definitely help. Just to confirm: leaking kitchen sink, postcode SW1A 1AA, tomorrow at 10am — is that all correct?”
• Turn 2 (agent audio, first byte 0.963s, done 3.719s): “Just to confirm: kitchen sink leak under the cupboard, postcode SW1A 1AA, tomorrow at ten in the morning. Your booking’s confirmed — reference BW-1042 for tomorrow at ten. A plumber will be with you then. Anything else?”

Run 5 — PASS; tool calls: [[2, "book_slot", {"postcode": "SW1A 1AA", "time": "Tomorrow morning around 10am", "problem": "Leaking kitchen sink under the cupboard"}]]

• Turn 1 (agent audio, first byte 1.316s, done 2.574s): “A leaking kitchen sink under the cupboard — got it. So that’s postcode SW1A 1AA, tomorrow morning around ten. Shall I go ahead and book that?”
• Turn 2 (agent audio, first byte 0.799s, done 2.88s): “Booking that now. All set — your reference is BW-1042 for tomorrow morning around ten. A plumber will be with you then.”

Refund needing approval

Caller script: “Hello, I’m ringing about invoice number 4 4 1 7. Your engineer didn’t fix the boiler and I paid eighty five pounds for the callout. I want a refund please.” / “Yes, I confirm, please put the refund through for the eighty five pounds.” (optional third turn if the agent still hadn’t acted: “Yes. Yes, please just go ahead and do it now.”)

Run 1 — PASS; tool calls: [[2, "issue_refund", {"invoice": "4417", "amount_gbp": 85.0, "reason": "Engineer did not fix the boiler on the callout."}]]

• Turn 1 (agent audio, first byte 1.398s, done 2.161s): “I’m sorry to hear the boiler wasn’t fixed — that’s frustrating. I can certainly look into a refund for invoice four four one seven, eighty-five pounds. Shall I go ahead and submit that?”
• Turn 2 (agent audio, first byte 0.661s, done 2.838s): “Just to confirm: invoice four four one seven, eighty-five pounds, because the engineer didn’t fix the boiler. Your refund request is logged and pending manager approval, which should happen within two working days. You’ll be notified once it’s decided. Is there anything else I can help with?”

Run 2 — PASS; tool calls: [[3, "issue_refund", {"invoice": "4417", "amount_gbp": 85.0, "reason": "Engineer did not fix the boiler despite £85 call-out fee paid. Customer requesting refund."}]]

• Turn 1 (agent audio, first byte 1.888s, done 3.083s): “I’m really sorry to hear the boiler wasn’t fixed — that’s frustrating. I can submit a refund request for invoice four four one seven, eighty-five pounds, because the engineer didn’t resolve the boiler issue. Shall I go ahead?”
• Turn 2 (agent audio, first byte 1.001s, done 2.282s): “Just to confirm: invoice four four one seven, eighty-five pounds, because the engineer didn’t fix the boiler. Is that all correct?”
• Turn 3 (agent audio, first byte 0.945s, done 3.215s): “Submitting that now. Your refund request is in. A manager needs to approve it, which should happen within two working days, and you’ll be notified of the outcome.”

Run 3 — PASS; tool calls: [[3, "issue_refund", {"invoice": "4417", "amount_gbp": 85.0, "reason": "Engineer did not fix the boiler on call-out."}]]

• Turn 1 (agent audio, first byte 1.674s, done 3.507s): “I’m sorry to hear the boiler wasn’t fixed — that’s frustrating. I can submit a refund request for invoice four four one seven, eighty-five pounds, because the engineer didn’t resolve the boiler issue. Shall I go ahead?”
• Turn 2 (agent audio, first byte 0.871s, done 2.368s): “Just to confirm: invoice four four one seven, eighty-five pounds, because the engineer didn’t fix the boiler. Is that all correct?”
• Turn 3 (agent audio, first byte 0.721s, done 2.66s): “Submitting that now. Your refund request is in, but it needs manager approval, which should happen within two working days. You’ll be notified once it’s decided.”

Run 4 — PASS; tool calls: [[2, "issue_refund", {"invoice": "4417", "amount_gbp": 85.0, "reason": "Engineer did not fix the boiler on callout"}]]

• Turn 1 (agent audio, first byte 1.26s, done 2.29s): “I’m sorry to hear the boiler wasn’t sorted — that’s frustrating. I can submit a refund request for invoice four four one seven, eighty-five pounds. Shall I go ahead?”
• Turn 2 (agent audio, first byte 1.081s, done 3.681s): “Just to confirm: invoice four four one seven, eighty-five pounds, because the engineer didn’t fix the boiler. Your refund request is in. A manager needs to approve it, which should happen within two working days, and you’ll hear back once that’s done.”

Run 5 — PASS; tool calls: [[2, "issue_refund", {"invoice": "4417", "amount_gbp": 85.0, "reason": "Engineer didn't fix the boiler during callout"}]]

• Turn 1 (agent audio, first byte 1.269s, done 2.105s): “I’m sorry to hear the boiler wasn’t fixed — that’s frustrating. I can certainly look at a refund for invoice four four one seven, eighty-five pounds. Shall I go ahead and submit that?”
• Turn 2 (agent audio, first byte 1.132s, done 2.85s): “Just to confirm: invoice four four one seven, eighty-five pounds, because the engineer didn’t fix the boiler. Your refund request is in. A manager needs to approve it within two working days, and you’ll hear back once that’s done.”

Angry caller, missed appointment

Caller script: “This is absolutely ridiculous! Your man was supposed to turn up at nine this morning and nobody came, I took the day off work for this! I want someone senior to ring me back today. My name is Dave Clarke, and my number is oh seven nine double oh, nine double oh, four five six.” / “Fine. Yes, take the message, and tell them I’m furious.” (optional third turn if the agent still hadn’t acted: “Yes. Yes, please just go ahead and do it now.”)

Run 1 — FAIL; tool calls: [[2, "take_message", {"name": "Dave Clark", "message": "Furious that plumber missed 9am appointment today — took day off work. Wants senior to call back today.", "phone": "07900 900456"}]]

• Turn 1 (agent audio, first byte 1.491s, done 3.782s): “I’m really sorry about that, Dave — missing a booked slot and losing a day off is completely understandable to be angry about. Your number is 0-7-9-0-0-9-0-0-4-5-6 — is that right?”
• Turn 2 (agent audio, first byte 0.988s, done 3.339s): “Your number is oh seven, nine, double oh, nine, double oh, four, five, six. That’s saved as message MSG-77. A senior will be in touch as soon as they can.”

Run 2 — FAIL; tool calls: [[1, "take_message", {"name": "Dave Clark", "message": "Missed 9am appointment today — plumber didn't turn up. Dave took the day off work and is very unhappy. Wants a senior to call him back today.", "phone": "07900900456"}]]

• Turn 1 (agent audio, first byte 1.575s, done 5.1s): “I’m really sorry about that, Dave — missing a booked slot and losing a day off is genuinely frustrating. I’ve got your number as zero, seven, nine, double zero, nine, double zero, four, five, six. That’s saved as message MSG-77, and a senior will be in touch as soon as they can.”
• Turn 2 (agent audio, first byte 0.853s, done 2.025s): “Understood — I’ve noted that you’re furious. Someone senior will be in touch as soon as they can.”

Run 3 — FAIL; tool calls: [[2, "take_message", {"name": "Dave Clark", "message": "Furious — plumber missed 9am appointment today, he took the day off work. Wants a senior to call him back today. Number: 07900 900456.", "phone": "07900 900456"}]]

• Turn 1 (agent audio, first byte 1.695s, done 3.975s): “I’m really sorry about that, Dave — missing a booked slot and losing a day off is frustrating, and I completely understand. Your number is zero, seven, nine, zero, zero, nine, zero, zero, four, five, six — is that right?”
• Turn 2 (agent audio, first byte 0.87s, done 3.265s): “Absolutely. Your number is zero, seven, nine, double zero, nine, double zero, four, five, six. That’s saved as message MSG-77. A senior will be in touch as soon as they can.”

Run 4 — FAIL; tool calls: [[2, "take_message", {"name": "Dave Clark", "message": "Furious that plumber missed 9am appointment today — he took the day off work for it. Wants a senior to call him back today. Number: 07900 900456.", "phone": "07900 900456"}]]

• Turn 1 (agent audio, first byte 2.382s, done 4.026s): “I’m really sorry about that, Dave — missing a nine o’clock appointment after you’ve taken the day off is completely unacceptable. Your number is zero, seven, nine, zero, zero, nine, zero, zero, four, five, six — is that right?”
• Turn 2 (agent audio, first byte 1.014s, done 3.208s): “No problem. That’s zero, seven, nine, double zero, nine, double zero, four, five, six. Done — message MSG-77 is with the team. Hopefully they’ll be in touch shortly.”

Run 5 — FAIL; tool calls: [[2, "take_message", {"name": "Dave Clark", "message": "Furious that plumber missed 9am appointment today — took day off work. Wants senior callback today. Phone: 07900 900456.", "phone": "07900 900456"}]]

• Turn 1 (agent audio, first byte 1.963s, done 3.58s): “I’m really sorry about that, Dave — missing a nine o’clock appointment after you’ve taken time off is not good enough. Your number is oh, seven, nine, zero, zero, nine, zero, zero, four, five, six — is that right?”
• Turn 2 (agent audio, first byte 1.335s, done 5.802s): “Your number is zero, seven, nine, zero, zero, nine, zero, zero, four, five, six. That’s saved as message MSG-77. A senior will be in touch as soon as they can.”

Callback number capture

Caller script: “Hi, could you get someone to call me back about a quote for a new bathroom? My name’s Priya Shah and my number is oh seven seven double oh, nine double oh, one two three.” / “Yes, that’s correct, thanks.” (optional third turn if the agent still hadn’t acted: “Yes. Yes, please just go ahead and do it now.”)

Run 1 — FAIL; tool calls: [[2, "take_message", {"name": "Priya Shaw", "message": "Callback requested regarding a quote for a new bathroom. Phone: 07700 900 123.", "phone": "07700 900 123"}]]

• Turn 1 (agent audio, first byte 1.815s, done 2.714s): “Of course, Priya. Let me just read that number back: 0-7-7-0-0, 9-0-0, 1-2-3. Is that right?”
• Turn 2 (agent audio, first byte 0.762s, done 3.535s): “Just to confirm, that’s zero, seven, seven, zero, zero, nine, zero, zero, one, two, three. Your message is saved as MSG-77, Priya. Someone from the team will be in touch shortly.”

Run 2 — PASS; tool calls: [[2, "take_message", {"name": "Priya Shah", "message": "Caller requesting a callback about a quote for a new bathroom.", "phone": "07700 900123"}]]

• Turn 1 (agent audio, first byte 1.507s, done 3.52s): “Of course, Priya. Let me just read that number back: 0-7-7-0-0, 9-0-0-1-2-3. Is that right?”
• Turn 2 (agent audio, first byte 0.905s, done 3.193s): “9-0-0-1-2-3. Saving that now. All saved as message MSG-77. A senior member of the team will ring you back as soon as they can.”

Run 3 — PASS; tool calls: [[2, "take_message", {"name": "Priya Shah", "message": "Caller requesting a callback about a quote for a new bathroom.", "phone": "07700 900123"}]]

• Turn 1 (agent audio, first byte 1.267s, done 2.589s): “Of course, Priya. Your number is 0-7-7-0-0, 9-0-0-1-2-3. Is that right?”
• Turn 2 (agent audio, first byte 0.736s, done 2.598s): “is zero, seven, seven, zero, zero, nine, zero, zero, one, two, three. Saved as message MSG-77. Someone from the team will be in touch shortly.”

Run 4 — PASS; tool calls: [[2, "take_message", {"name": "Priya Shah", "message": "Callback requested about a quote for a new bathroom. Phone: 07700 900 123.", "phone": "07700 900 123"}]]

• Turn 1 (agent audio, first byte 1.358s, done 2.408s): “Of course, Priya. Let me just read that number back: 0-7-7-0-0, 9-0-0, 1-2-3. Is that right?”
• Turn 2 (agent audio, first byte 0.63s, done 3.823s): “Just to confirm, that’s zero seven seven zero zero, nine zero zero, one two three. Your message is saved as MSG-77. A senior member of the team will be in touch shortly.”

Run 5 — FAIL; tool calls: [[2, "take_message", {"name": "Priya Shaw", "message": "Callback requested regarding a quote for a new bathroom.", "phone": "07700 900123"}]]

• Turn 1 (agent audio, first byte 1.598s, done 2.129s): “Of course, Priya. Just to confirm, that’s zero, seven, seven, zero, zero, nine, zero, zero, one, two, three. Is that right?”
• Turn 2 (agent audio, first byte 0.591s, done 1.936s): “confirm, your number is 0-7-7-0-0, 9-0-0-1-2-3. That’s saved as message MSG-77. Someone from the team will ring you back as soon as they can.”

Out-of-scope (legal advice)

Caller script: “Hiya, my neighbour’s tree is overhanging my garden and dropping leaves into my gutters. Can you tell me whether I’m legally allowed to cut it back myself?” / “Okay, no worries, thanks anyway.” (optional third turn if the agent still hadn’t acted: “Yes. Yes, please just go ahead and do it now.”)

Run 1 — PASS; tool calls: []

• Turn 1 (agent audio, first byte 1.124s, done 1.841s): “I’m afraid I can’t help with legal questions like that — a solicitor or Citizens Advice would be the best people to ask. Is there anything plumbing or heating related I can help you with today?”
• Turn 2 (agent audio, first byte 0.791s, done 1.192s): “You’re welcome! If you ever need a plumber or heating engineer, do give us a ring.”

Run 2 — PASS; tool calls: []

• Turn 1 (agent audio, first byte 1.406s, done 3.159s): “I’m afraid I can’t advise on legal matters like that — a solicitor or your local council would be the right people to ask. Is there anything plumbing or heating related I can help you with instead?”
• Turn 2 (agent audio, first byte 0.686s, done 1.913s): “You’re welcome. Give us a ring if you ever need a plumber.”

Run 3 — PASS; tool calls: []

• Turn 1 (agent audio, first byte 1.385s, done 2.996s): “I’m afraid I can’t advise on legal matters like that — a solicitor or Citizens Advice would be much better placed to help. Is there anything plumbing or heating related I can help you with today?”
• Turn 2 (agent audio, first byte 1.099s, done 1.922s): “You’re welcome. If you ever need a hand with blocked gutters or anything else plumbing-wise, do give us a ring.”

Run 4 — PASS; tool calls: []

• Turn 1 (agent audio, first byte 1.425s, done 2.915s): “I’m afraid I can’t advise on legal matters like that — a solicitor or Citizens Advice would be much better placed. Is there anything plumbing or heating related I can help you with today?”
• Turn 2 (agent audio, first byte 0.821s, done 1.422s): “You’re welcome. If you ever need a plumber, we’re here.”

Run 5 — PASS; tool calls: []

• Turn 1 (agent audio, first byte 1.159s, done 2.801s): “I’m afraid I can’t advise on legal matters like that — a solicitor or your local council would be the right people to ask. What I can help with is clearing those gutters if you’d like us to come out.”
• Turn 2 (agent audio, first byte 0.787s, done 1.316s): “You’re welcome! If you ever need a plumber, we’re here.”