If you've built an AI voice agent using Twilio, Vapi, Retell, or the OpenAI Realtime API, you've made a choice about your transport stack — even if you didn't know you were making it.
The voice infrastructure underneath your agent is a layered stack of protocols that were each designed for a different era, a different problem, and a different definition of "a phone call." Understanding which layer you're on — and why it matters — is the difference between shipping a demo and running something in production that doesn't embarrass you.
Start with the actual problem
A conversational AI voice agent needs to do three things simultaneously:
The "noticeable delay" threshold for conversational speech is around 150ms. Once your total round-trip exceeds 1.5 seconds, the interaction starts to feel robotic. Your transport layer is the first place latency accumulates — before the STT model has heard a single phoneme.
That's why the protocol choice matters.
PSTN: The backbone nobody thinks about
The Public Switched Telephone Network is the reason phone calls work at all. It's the physical and logical infrastructure of copper wires, fiber, switching equipment, and carrier interconnects that has been building since Alexander Graham Bell's first call in 1876.
A PSTN call is circuit-switched: when you dial a number, a dedicated circuit is reserved for your call for its entire duration. No packets, no internet, no sharing of that bandwidth with other traffic. The circuit is yours until you hang up.
This design choice — made before packet-switched networking existed — gives PSTN its best quality: extreme reliability. Carriers target 99.999% ("five nines") uptime. The network was built to survive nuclear attacks. It degrades gracefully. It doesn't have jitter.
The codec used over PSTN is G.711 — a narrowband standard that samples at 8kHz and transmits at 64kbps. It sounds like a phone call because it is the sound of a phone call. Good enough for speech recognition, but not particularly high fidelity.
When your AI agent calls a human's mobile phone, the final segment of that call almost certainly traverses PSTN — even if your platform is entirely IP-based. The carrier converts at the edge.
PSTN reach: any phone number in the world.
PSTN latency to your AI stack: 250–400ms depending on carrier hops.
SIP: PSTN's IP-native successor
SIP (Session Initiation Protocol) is an IETF standard (RFC 3261, published 2002) for setting up, modifying, and tearing down real-time communication sessions over IP networks. SIP is a signaling protocol — it handles the handshake, the negotiation, the "I'm calling, are you ready?" exchange.
The actual voice data travels separately, carried by RTP (Real-time Transport Protocol) over UDP. SDP (Session Description Protocol) is the third piece — it negotiates which codec to use, which port to listen on, and what format the audio will arrive in.
SIP is how most enterprise phone systems work today. Your office PBX is almost certainly SIP. Twilio Voice, Bandwidth, Vonage, and Telnyx all expose SIP trunks. When you build a voice agent on Twilio, you're using SIP — Twilio translates between SIP/RTP and their own API, then bridges to PSTN at the carrier layer.
SIP's advantage over raw PSTN: it's programmable. You can intercept calls, fork audio streams, redirect mid-call, play audio, receive DTMF input. Every cloud telephony API you've used is a SIP abstraction.
SIP's latency problem: each hop between SIP nodes (caller → originating carrier → Twilio → your SIP endpoint → your AI stack) adds 20–50ms. A path with three to five hops accumulates 250–400ms of latency before your model has heard a single word.
SIP also introduces jitter — variable packet arrival times that require buffering to smooth out, adding another 20–60ms of de-jitter buffer delay.
WebRTC: Built for the browser, adopted by AI
WebRTC (Web Real-Time Communication) is a W3C/IETF open standard that landed in Chrome in 2012 and is now supported natively by every major browser. It was designed so that two browsers could establish a direct, low-latency audio/video connection without a plugin, without a media server in the middle.
WebRTC is not a single protocol — it's a stack:
The Opus codec alone is meaningful for AI voice agents. Wideband audio gives your STT model significantly more signal to work with — fewer transcription errors on accented speech, overlapping words, and background noise.
WebRTC's latency to your AI stack: 60–120ms in the median case. That's two to four times lower than a SIP path, which is the difference between a conversation that feels fluid and one that feels like you're talking to a satellite phone.
The catch: WebRTC requires that the endpoint runs WebRTC. A regular phone number — calling from a mobile phone, a landline, a desk phone — does not speak WebRTC. To reach standard phone numbers, you need to bridge WebRTC back to SIP and then to PSTN.
How they fit together in practice
Almost no production voice AI stack runs on a single protocol in isolation. The architecture that has crystallized in 2025–2026 looks like this:
Ingress path 1 — Phone numbers (PSTN via SIP):
Human dials a phone number → PSTN → SIP trunk (Twilio, Bandwidth, etc.) → Media gateway → AI stack
Ingress path 2 — Browser/app (WebRTC):
Human opens a web or mobile app → WebRTC direct connection → AI stack
Same agent logic, same prompts, same tools — two ingress paths with different latency profiles. A media gateway in the middle translates between SIP and WebRTC so both paths feed the same audio pipeline.
The OpenAI Realtime API made this concrete: it now supports both WebRTC and SIP as connection methods. WebRTC for browser-based agents, SIP for phone-number-based agents. You can wire the same model to both paths.
The latency budget
Here's what a full round-trip looks like for a conversational AI voice agent:
| Stage | Typical latency |
|---|---|
| Transport (WebRTC) | 60–120ms |
| Transport (SIP/PSTN) | 250–400ms |
| STT (speech-to-text) | 200–400ms |
| LLM inference (first token) | 300–800ms |
| TTS (text-to-speech) | 150–300ms |
| Total (WebRTC path) | ~800ms–1.6s |
| Total (SIP/PSTN path) | ~1.0s–1.9s |
Full Round-Trip Latency by Stage
milliseconds — hover bars to inspect each stage
The human conversational comfort threshold is roughly 1–1.5 seconds of response latency. Both paths can hit it, but the WebRTC path has more headroom for LLM variance. When your model takes 800ms to generate the first token on a complex query, the WebRTC path stays inside the comfortable range; the SIP/PSTN path may not.
This is why production teams optimize the transport layer first, before touching the model or the STT provider.
What each protocol is actually good for
Protocol Comparison by Dimension
score out of 100 — higher is better for each axis
Use PSTN (via SIP trunk) when:
Use SIP when:
Use WebRTC when:
The real answer: use both. WebRTC for in-app conversations, SIP/PSTN for outbound dialing. The media gateway that bridges them is the piece most teams don't want to build.
What this means if you're building on a platform
If you're building on Twilio Programmable Voice, you're on SIP/PSTN. That's fine — it reaches every phone number and it's operationally reliable. But you're starting with 250–400ms of transport latency before your AI layer.
If you're building on the OpenAI Realtime API over WebRTC, you're getting 60–120ms transport latency for browser-based conversations, with SIP trunking available for phone number connectivity.
Platforms like Vapi and Retell abstract all of this — they handle the SIP trunking, the WebRTC bridging, the media gateway — so you don't have to think about which protocol is carrying your audio. Until something breaks.
When something breaks in a voice AI system — dropped audio, one-sided conversations, echoing, extreme latency spikes — the root cause is almost always in the transport layer: a misconfigured jitter buffer, a TURN server under load, a SIP re-INVITE that failed silently, a codec negotiation that fell back from Opus to G.711.
Knowing which protocol you're on is the starting point for debugging any of it.