Documentation — RTCanary checks, configuration, alerting

Concepts

Target — the thing being called: a room join URL, a TURN URI, a SIP trunk, a phone number or a WebRTC endpoint of a voice agent.
Check — a target plus a probe region, a schedule, a scenario and thresholds.
Run — one execution of a check. Produces a result (pass, warn, fail, error), a metric summary and a per-second trace.
Scenario — what the probe does once connected: publish and subscribe for a room, a relay round-trip for TURN, a scripted multi-turn dialogue with expectations for a voice agent.
Alert rule — how many consecutive non-passing runs from how many regions trigger a notification, and where it goes.

Check configuration

Checks are declared in YAML and applied through the API or the dashboard. A minimal WebRTC room check against a LiveKit-style join:

version: v1
checks:
  - id: prod-room-join
    type: webrtc_room
    regions: [eu-west, us-east]
    interval: 60s
    target:
      join_url: https://meet.example.com/join?room=canary
      token: ${{ secrets.CANARY_TOKEN }}   # stored encrypted, never logged
    scenario:
      publish: { audio: speech-sample-1, video: test-pattern-720p }
      subscribe: true
      duration: 20s
    thresholds:
      time_to_connected_ms:   { warn: 2000, fail: 5000 }
      time_to_first_audio_ms: { warn: 1500, fail: 4000 }
      jitter_ms:              { warn: 30,   fail: 60 }
      packet_loss_pct:        { warn: 1,    fail: 3 }
      mos_estimate:           { warn: 3.8,  fail: 3.3, direction: below }
      candidate_type:         { expect: [host, srflx, relay] }
    alert: default

A TURN check and a SIP OPTIONS check in the same file:

  - id: turn-eu
    type: turn
    regions: [eu-west, us-east, ap-southeast]
    interval: 120s
    target:
      uris:
        - turn:turn.example.com:3478?transport=udp
        - turn:turn.example.com:3478?transport=tcp
        - turns:turn.example.com:5349?transport=tcp
      credentials:
        mode: rest            # RFC-style time-limited credentials
        secret: ${{ secrets.TURN_SECRET }}
        ttl: 600
    thresholds:
      allocate_ms:        { warn: 400, fail: 1500 }
      relay_rtt_ms:       { warn: 150, fail: 400 }
      cert_days_left:     { warn: 14,  fail: 3, direction: below }
      allocate_success:   { expect: all_transports }

  - id: trunk-options
    type: sip_options
    regions: [us-east]
    interval: 30s
    target:
      uri: sip:trunk.example.net:5061;transport=tls
    thresholds:
      response_ms:    { warn: 500, fail: 2000 }
      status_code:    { expect: [200] }

A voice-AI agent check with a scripted dialogue. Expectations are checked per turn; judge: rubric enables the rubric judge for that turn and stores its verdict and reasoning with the run:

  - id: support-agent
    type: voice_agent
    regions: [us-east]
    interval: 300s
    target:
      kind: phone
      number: "+1xxxxxxxxxx"
    scenario:
      turns:
        - say: "Hi, I need to change the email on my account."
          expect: { contains_any: ["verify", "confirm", "account number"] }
          expect_reply_within_ms: 2500
        - say: "It's 4 4 8 1 2 0."
          expect: { intent: "asks_for_new_email", judge: rubric }
        - say: "Actually, never mind."
          interrupt_after_ms: 800        # start speaking while the agent is still talking
          expect_agent_stops_within_ms: 600
      end_state: { any_of: [hangup_by_agent, transfer_offered] }
    thresholds:
      time_to_first_audio_ms: { warn: 1500, fail: 3000 }
      turn_latency_p95_ms:    { warn: 2000, fail: 4000 }
      scenario_result:        { expect: pass }

A realtime API check against an OpenAI Realtime-style endpoint or your own gateway:

  - id: realtime-gateway
    type: realtime_api
    regions: [us-east, eu-west]
    interval: 120s
    target:
      url: wss://realtime.example.com/v1/session
      transport: websocket          # or webrtc
      auth: { header: Authorization, value: ${{ secrets.RT_TOKEN }} }
    scenario:
      prompt: speech-sample-1
      expect_response: true
    thresholds:
      session_setup_ms:       { warn: 800,  fail: 2000 }
      first_audio_delta_ms:   { warn: 1200, fail: 2500 }
      response_done_ms:       { warn: 6000, fail: 12000 }

Run results

Every run is stored as JSON and retrievable through the API. Abbreviated example of a WebRTC room run:

{
  "check_id": "prod-room-join",
  "region": "eu-west",
  "started_at": "2026-10-19T09:00:00Z",
  "result": "warn",
  "summary": {
    "time_to_connected_ms": 1420,
    "time_to_first_audio_ms": 611,
    "candidate_pair": { "local": "relay", "remote": "host", "protocol": "udp" },
    "rtt_ms": { "p50": 71, "p95": 88 },
    "jitter_ms": { "p50": 12, "p95": 38 },
    "packet_loss_pct": 0.4,
    "concealment_events": 7,
    "mos_estimate": 4.1
  },
  "violations": [ { "metric": "jitter_ms", "level": "warn", "value": 38, "threshold": 30 } ],
  "trace_url": "https://app.rtcanary.com/runs/…"
}

Alert rules

alerts:
  default:
    require:
      consecutive_failures: 2
      regions: 2
    notify:
      - slack: "#rtc-oncall"
      - webhook: https://hooks.example.com/rtcanary
      - pagerduty: ${{ secrets.PD_ROUTING_KEY }}
    recover_notify: true

consecutive_failures counts warn only if alert_on_warn: true is set; by default warnings are visible on the dashboard and the status page but do not page anyone.

REST API and MCP server

Everything the dashboard shows is available at https://api.rtcanary.com/v1 with a bearer token created in the workspace settings. Tokens are scoped read-only or read-write.

Method and path	Returns
`GET /v1/checks`	All checks with their current state and last run per region
`PUT /v1/checks`	Applies a configuration file; the response lists created, changed and removed checks
`GET /v1/checks/{id}/runs?region=&since=`	Run summaries, newest first
`GET /v1/runs/{run_id}`	One run with its summary, violations, attribution and scenario verdicts
`GET /v1/runs/{run_id}/trace`	The per-second metric trace as JSON or CSV
`POST /v1/checks/{id}/mute`	Mutes alerts for a duration, with a reason that appears in the alert history
`GET /v1/metrics`	Prometheus exposition of the latest run per check and region (Scale tier)

The MCP server exposes the same operations as tools, so an on-call agent can investigate a page without a person pasting JSON into it. Streamable HTTP endpoint: https://api.rtcanary.com/mcp, same bearer token.

Tool	What it does
`list_checks`	Checks with state, last result and region
`get_run`	One run: summary, violations, attribution, scenario verdicts and judge reasoning
`get_trace`	Per-second trace of a run, optionally one metric
`compare_runs`	Diff of two runs' summaries, by default the failing run and the last passing one from the same region
`explain_failure`	The attribution with its evidence lines, in the order the probe established them
`mute_check`	Mute with a duration and a reason (read-write token)

Adding it to Claude Code:

claude mcp add --transport http rtcanary https://api.rtcanary.com/mcp \
  --header "Authorization: Bearer ${RTCANARY_TOKEN}"

For Cursor and other MCP clients, declare the same URL and header in the client's MCP configuration. OpenTelemetry export (Scale tier) sends each run as a span with the summary as attributes to the OTLP endpoint you configure, so canary runs sit on the same timeline as your traces.

How the MOS estimate is computed

The audio score follows the ITU-T G.107 E-model: a transmission rating R is reduced by delay impairment (from one-way mouth-to-ear delay) and by equipment impairment (codec impairment adjusted for packet loss with the codec's robustness factor), then mapped to a MOS. The inputs are the measured round-trip time, jitter-buffer delay, codec and packet loss. The exact formulas and the published codec values are on the MOS calculator page, which uses the same implementation, and the background is in the guide on what a MOS score means.

Security notes

Tokens and secrets are stored encrypted and are never written to run traces or logs.
Probe IP ranges are published per region so you can allow-list them on firewalls and TURN servers.
Recorded media from a run is not stored unless you enable it for a check; the per-second metric trace is stored for the retention of your tier.

Checks, configuration and alerting