WebRTC testing: what breaks at each layer and how to test it

WebRTC testing is six tests, not one. A call passes through signalling, ICE/STUN/TURN connectivity, media encoding under packet loss and jitter, the browser and device matrix, concurrency on the media server, and the production network after deploy. Each layer fails on its own, leaves its own evidence (a signalling log, an ICE candidate pair, a getStats() counter) and needs its own class of tool. The table below maps all six; the sections after it show how to run each test and where the usual approach stops working.

Which layers of a WebRTC call need testing?

A session succeeds only if every layer succeeds in order, and a test of one layer says nothing about the next: a green unit test on the signalling server does not prove the TURN server relays UDP from a corporate network, and a clean lab call does not prove the same call survives 2% loss on a mobile link.

How do you test WebRTC signalling?

Signalling is the one layer WebRTC does not standardise, so the test is a functional test of your own protocol: a client asks to join, receives whatever your server returns (token, SDP answer, ICE server list) and ends up with an RTCPeerConnection in the connected state. The assertion is the state transition and the time it took, not the HTTP 200 on the join request.

The test client is a real browser driven by a script. Playwright's and Puppeteer's launch() both take an args array of command-line flags for the browser instance. Two Chromium flags make the join work on a machine with no camera and no microphone: --use-fake-device-for-media-stream, described in Chromium's source as "Use fake device for Media Stream to replace actual camera and microphone", and --use-fake-ui-for-media-stream, which webrtc.org lists for CI servers. --use-file-for-fake-audio-capture=<file.wav> plays a WAV file as the microphone, so the publisher sends known speech instead of a tone. Firefox uses preferences instead: media.navigator.streams.fake: true and media.navigator.permission.disabled: true, passed through Playwright's firefoxUserPrefs.

Assert, in order: the join response carried ICE servers with unexpired credentials; setRemoteDescription succeeded; iceConnectionState reached connected inside the budget; the first inbound RTP packet arrived. Join-to-first-packet is the connect time, the number that regresses silently when a signalling server adds one database round trip per join.

How do you test ICE, STUN and TURN reachability?

ICE (RFC 8445) gathers three candidate types that matter here: a host candidate bound to a local address, a server-reflexive (srflx) candidate learned from a STUN server (RFC 8489), and a relayed candidate "obtained from a relay server, such as a TURN server" (RFC 8656). Defaults that surface in failures: port 3478 for UDP and TCP, 5349 for TLS and DTLS, a 10-minute allocation lifetime and a 5-minute permission lifetime the client must refresh.

The quickest manual check is the WebRTC project's Trickle ICE sample page: enter a STUN or TURN URI with credentials, press "Gather candidates", read the candidate table. Its own success criteria: a STUN server works if it yields an srflx candidate, a TURN server works if it yields a relay candidate, and with a single TURN/UDP server the page reveals wrong credentials.

The automated check creates an RTCPeerConnection with iceTransportPolicy: "relay", which the W3C WebRTC spec defines as "The ICE Agent uses only media relay candidates such as candidates passing through a TURN server". If the connection still reaches connected, the whole TURN path works: allocation, permission, relay and credentials. In getStats(), the selected RTCIceCandidatePairStats reports currentRoundTripTime "computed from both STUN connectivity checks", and the local RTCIceCandidateStats reports relayProtocol, "the protocol used by the endpoint to communicate with the TURN server": UDP, or a fallback to TCP or TLS.

Three TURN failures a reachability test must cover, because each one passes a plain port check:

• Expired time-limited credentials. coturn's REST API mode (--use-auth-secret) builds the username as timestamp:username and the password as an HMAC with a shared secret; the README states these are valid only "while the timestamp is not expired". A signalling server that caches its ICE server list hands out dead credentials once the window closes.
• Expired TLS certificate. coturn's --cert and --pkey point at PEM files; the TLS listener on 5349 is the only path for users on networks that block UDP, so a lapsed certificate removes exactly the users who need TURN most.
• UDP blocked, TCP open. The relay works, but over TCP; relayProtocol is the only place you see it.

Outside the browser, coturn's turnutils_stunclient and turnutils_uclient check the server itself over UDP, TCP, TLS or DTLS; neither checks the credential path your app uses.

How do you test media quality under packet loss and jitter?

A media test needs a way to damage the network on purpose and a way to read what the receiver did about it. Linux tc netem adds delay with jitter and correlation (delay 100ms 10ms 25%), independent random loss (loss 0.1%), duplication, corruption, reordering and a rate limit. RTCPeerConnection.getStats() returns the counters defined in the W3C WebRTC Statistics API (Candidate Recommendation Draft, 25 September 2025). The inbound RTP counters that answer "did the receiver suffer":

• jitter: "Packet Jitter measured in seconds for this SSRC", the RFC 3550 interarrival estimate updated per packet as J(i) = J(i−1) + (|D(i−1,i)| − J(i−1))/16.
• packetsLost: "Total number of RTP packets lost for this SSRC"; it can go negative when duplicates arrive, so assert on the delta over a window.
• jitterBufferDelay / jitterBufferEmittedCount: their ratio is the average delay the receiver chose to add.
• concealedSamples: samples "replaced with synthesized samples generated locally before being played out"; concealmentEvents counts each time concealment starts after real audio. These are the receiver's own record of audible gaps.

On the sending side, RTCOutboundRtpStreamStats.qualityLimitationReason reports why resolution or frame rate was reduced: bandwidth ("congestion cues during bandwidth estimation"), cpu, other or none. A test that only watches the receiver misses this: the sender drops from 720p to 360p and every inbound counter stays clean.

A loss test is only meaningful with the codec named. RFC 7874 requires WebRTC endpoints to implement Opus and G.711. Opus (RFC 6716) carries in-band forward error correction: perceptually important frames are re-encoded at a lower bitrate and attached to a later packet, so a single lost packet is often recovered rather than concealed. G.711 has nothing of the kind; ITU-T G.113 gives it a packet-loss robustness factor Bpl of 4.3 without concealment and 25.1 with the G.711 Appendix I concealer. The same 2% loss sounds different in each of the three cases.

To fold loss, delay and jitter into one number, the ITU-T G.107 E-model computes a rating factor R and maps it to an estimated MOS; the MOS calculator on this site runs that model and the MOS score guide covers what the number can and cannot say. For a test assertion the raw counters are better: a threshold on concealedSamples per second is reproducible, while a MOS estimate depends on which impairment table the tool used.

How do you automate WebRTC tests across browsers and devices?

The matrix test is the signalling test repeated per engine with that engine's fake-media setup. The Chromium flags are Chromium-only; Firefox takes the media.navigator.* preferences; WebKit takes neither, so Safari coverage needs a real device or a virtual camera driver. Fake video in Chromium is a .y4m file via --use-file-for-fake-video-capture; fake audio is a .wav and, per the Chromium source comment, must be combined with --use-fake-device-for-media-stream.

What the matrix catches: a constraint (frameRate, echoCancellation) accepted by one engine and rejected by another; a codec one engine does not offer; a permission flow that blocks headless runs. Compare the same getStats() fields across engines and expect the field set itself to differ.

Jitsi's open-source suite, jitsi-meet-torture (Apache-2.0), runs conference scenarios against a Jitsi Meet instance and includes a PSNR check that scores received video frame by frame against a stamped .y4m source: an assertion on what the receiver rendered, not on what the counters claim.

How do you load test a WebRTC service?

A WebRTC load test launches many real media publishers against one or many rooms and watches two sides at once: per-participant stats (jitter, loss, concealment, qualityLimitationReason) and server metrics (CPU, forwarded bitrate). The question is not "did the server stay up" but "at what participant count did the first stream degrade, and which counter moved first".

The tool class is a browser farm. testRTC, acquired by Spearline in November 2021 and now sold by Cyara (which bought Spearline in March 2023) as CyaraRTC, schedules "tests from real browsers" and covers load and regression testing. The self-built alternative is Playwright or Puppeteer on a fleet of cloud instances with the fake-media flags, every instance publishing the same WAV and Y4M so streams are comparable. Each headless Chromium encoding video costs real CPU, so qualityLimitationReason: cpu appears on the test rig before it appears on the server.

Keep two outputs per run: the participant count at which median inbound concealedSamples per second first crossed your threshold, and the count at which forwarded bitrate stopped scaling linearly.

How do you catch WebRTC regressions in production?

Everything above runs before deploy, on networks you control. Production breaks for reasons those tests cannot see: a TURN certificate expires on a Sunday, a cloud provider changes NAT behaviour in one region, a carrier adds 80 ms. Two tool classes cover this and answer different questions.

Real-user monitoring (RUM) collects getStats() from your users' browsers. rtcstats (Jitsi, MIT licence) wraps getUserMedia and RTCPeerConnection, calls getStats on a timer and streams the result to a server over a websocket; ObserveRTC's client-monitor-js (Apache-2.0) polls getStats() and emits a ClientSample per sampling period for a server-side observer. RUM reports on the users who connected. A user who never got past ICE sent no sample, and at 03:00 with no callers there is nothing to report.

Synthetic monitoring is the other half. Datadog describes its synthetic product as "simulated requests and actions from around the globe" run "from managed locations"; Grafana describes its version as black-box checks of "the externally visible behavior of your systems" from global probe locations. For WebRTC the simulated action is a scheduled canary call: a headless client outside your network joins a real room on a timer, publishes known media, runs the ICE and relay checks above, reads the same getStats() counters and alerts when a threshold trips. That is the design behind how RTCanary's canary calls work. A canary catches the expired certificate before the first user does, in every region it runs from; it does not see what one user on one hotel Wi-Fi saw.

Per incident the two reduce to one question: did the canary fail too? If yes, the fault is in your stack or its network path. If the canary passed and RUM shows a cluster of bad sessions, the fault is on the users' side.

Which WebRTC testing tools cover which layer?

When the standard WebRTC testing advice fails

• "Just use fake media." A fake device publishes a clean signal with no echo or room noise, so echo-cancellation and noise-suppression bugs never appear. Those need a real loudspeaker-to-microphone path or recorded impaired audio through --use-file-for-fake-audio-capture.
• "Assert on MOS." The E-model was built for narrowband transmission planning; the G.113 impairment tables list G.711, G.729 and G.723.1, not Opus. A MOS from a WebRTC stats tool is an estimate with an undocumented codec mapping. Assert on the counters; report MOS as context.
• "1% loss is fine." Random 1% and bursty 1% are different impairments; G.107's loss term carries a burst ratio for this reason. netem's loss random is independent by definition and understates bursty real-world loss unless you set a correlation.
• "The TURN server is up." A TCP connect to 3478 proves a listener exists. It does not prove allocation succeeds with the credentials your signalling server hands out right now, or that the 5349 certificate is valid. Test allocation, not the port.
• "RUM covers production." A user who fails ICE sends no stats, and a region with no traffic sends no stats. RUM measures survivors; synthetic calls measure availability.

Frequently asked questions

Can I test WebRTC without a camera and microphone?

Yes. Chromium's --use-fake-device-for-media-stream replaces both with synthetic sources, --use-fake-ui-for-media-stream suppresses the permission prompt, and the --use-file-for-fake-*-capture flags feed a WAV or Y4M file. Firefox uses media.navigator.streams.fake and media.navigator.permission.disabled.

Is a connected peer connection enough to say the call worked?

No. connected means a candidate pair passed STUN checks and DTLS completed. Media can still fail to flow or arrive unusable. The check that matters is inbound packets increasing and concealedSamples staying under your threshold over a window of several seconds.

Do I need a TURN server to run WebRTC tests?

On one machine or one LAN, no: host candidates connect directly. For any test that stands in for users behind NATs and firewalls, yes, and it should force iceTransportPolicy: "relay" at least once so the relay path is proven rather than assumed.

Sources

1. RFC 8445, Interactive Connectivity Establishment (ICE), candidate types and connectivity checks.
2. RFC 8489, Session Traversal Utilities for NAT (STUN), Binding transaction and default ports.
3. RFC 8656, Traversal Using Relays around NAT (TURN), allocations, lifetimes, transports and ports.
4. RFC 3550, RTP: A Transport Protocol for Real-Time Applications, section 6.4.1.
5. RFC 6716, Definition of the Opus Audio Codec, in-band FEC (LBRR).
6. RFC 7874, WebRTC Audio Codec and Processing Requirements.
7. W3C, Identifiers for WebRTC's Statistics API, Candidate Recommendation Draft, 25 September 2025.
8. W3C, WebRTC: Real-Time Communication in Browsers, RTCIceTransportPolicy.
9. ITU-T G.107 (06/2015), The E-model, equation 7-29; ITU-T G.113 (09/2024), Appendix I Table I.4.
10. webrtc.org, Testing; Chromium, media/base/media_switches.cc.
11. WebRTC samples, Trickle ICE.
12. Playwright, BrowserType.launch; Puppeteer, LaunchOptions.
13. coturn, README.turnserver; README.turnutils.
14. tc-netem(8) manual page.
15. jitsi/rtcstats; observertc/client-monitor-js; jitsi/jitsi-meet-torture.
16. Spearline acquires testRTC (30 November 2021); Cyara acquires Spearline (27 March 2023); Cyara, CyaraRTC.
17. Datadog, Synthetic Monitoring; Grafana, Synthetic Monitoring.