Testing and observability

Verification must cover authorization, transport fallback, device capture/input and real network behavior. A green unit suite alone does not prove that a signed guest artifact has the required TCC permissions or that ICE works from the target region.

Automated tests

API

Run the scoped Jest suite from the repository root:

npm --prefix api test -- --runInBand \
  test/computer-access.service.spec.ts \
  test/connector-token.service.spec.ts \
  test/machine-health.service.spec.ts \
  test/machine-metadata.spec.ts \
  test/machine-pool.service.spec.ts \
  test/machines.role-access.spec.ts \
  test/machines.service.spec.ts \
  test/remote-desktop-connect.spec.ts \
  test/remote-desktop-tailnet-connector.spec.ts \
  test/remote-desktop-webrtc-signaling.spec.ts \
  test/remote-desktop.spec.ts \
  test/turn-credentials.service.spec.ts \
  test/webrtc-ice.controller.spec.ts \
  test/webrtc-telemetry.spec.ts

These cover role and entitlement gates, machine pool transitions, credential redaction, direct token behavior, raw relay authorization, tailnet SOCKS, signaling isolation/cross-pod behavior, TURN credentials and bounded telemetry.

Frontend

npm --prefix frontend test -- --runInBand \
  src/app/workspace/computer/__tests__/connect-mode.test.ts \
  src/app/workspace/computer/__tests__/useComputerWebrtcVideo.test.ts \
  src/app/workspace/computer/__tests__/webrtcIceCandidateQueue.test.ts \
  src/app/workspace/computer/__tests__/webrtcStats.test.ts \
  src/app/workspace/computer/__tests__/webrtcTelemetry.test.ts \
  src/app/workspace/computer/adaptive-quality.test.ts

These cover server-plan selection, ICE candidate ordering, native video setup and recovery, stats/telemetry shaping and adaptive quality loss calculation.

Connector and native encoder

cd remote-desktop-agent
go test ./...

cd ..
swift test --package-path remote-desktop-agent/native-video

Connector tests cover configuration, input parsing/rate limiting/gating, session teardown, Annex-B handling, ICE fetch and TURN integration. macOS native tests cover configuration bounds, malformed AVCC, hardware-required H.264 ABR and CBR, keyframe forcing with SPS/PPS/IDR, and VideoToolbox decode.

The native test suite needs macOS hardware VideoToolbox. Portable CGO_ENABLED=0 builds retain the RFB fallback but do not prove native capture.

Device provisioning and direct guard

python3 -m pytest \
  devices/mac-mini/tests/test_enroll_desktop_connector.py \
  devices/mac-mini/tests/test_connect_guard.py

These tests validate secret handling, idempotent enrollment, launchd rendering, token validation, WebSocket framing and health/session calls. They do not replace a real GUI/TCC smoke test.

Manual acceptance matrix

Test with an Expert who has org access, an entitled assigned machine, and a client user without access.

Scenario	Expected result
Expert, `webrtc`, healthy connector	Native H.264 connects; input works; path, RTT and quality display
Force `p2p`	P2P RFB connects through the connector; no API media relay
Disable native video or fail capture	Automatic P2P RFB fallback, then relay descriptor if needed
`direct_wss`, healthy tunnel and Redis	Direct WSS connects; token cannot be replayed
Direct handshake failure	One new-token retry, then API relay with a visible path change
`proxy`	noVNC connects through the raw API relay
Connector disconnect while holding input	Held keys/buttons release and input gate closes
Expert loses access or entitlement	Machine disappears; HTTP/signaling/relay connect is rejected
Client role attempts any path	Rejected before machine/network access
Machine enters maintenance/retired/unassigned	Not listed or dialable
Guest reboot	Auto-login starts Aqua LaunchAgent; no new TCC prompt; presence returns online

Existing detailed manual records live under docs/test-cases/:

Computer-LatencyBadge-CloseSession-test.md
Computer-P2P-Transport-Toggle-test.md
Computer-P2P-WebRTC-Fixes-Regression-test.md
Computer-WebRTC-CursorDisplay-test.md
Expert-Workspace-Computer-SessionLogs-test.md
Expert-Workspace-RemoteDesktop-VNCPort-test.md

Those are test cases/evidence, not the architecture source of truth.

Regional latency harness

The Playwright harness in e2e/latency/ compares native H.264 and P2P RFB against the same guest, input action and frame threshold. Run from real network egress in each named region; a VPN label or injected delay is not equivalent.

Start the deterministic responder in the guest GUI session:

swiftc e2e/latency/guest-responder.swift -o /tmp/hwork-latency-responder
/tmp/hwork-latency-responder

Then, from the regional runner:

cd e2e
npm ci
npx playwright install --with-deps chromium
npm run latency:remote-desktop -- \
  --region eu --transport both --trials 12 --fps-seconds 5 \
  --output latency-eu.json

Repeat serially for EU, US and HK so two regions never control the same guest. The JSON output records raw input-to-frame samples, p50/p95, decoded/visual fps, resolution, selected candidates, direct versus TURN, codec, RTT and p50 - RTT added latency. Preserve raw JSON with release evidence.

On a restricted network, --ice-config may provide an uncommitted short-lived ICE array and --force-relay may force TURN. A forced relay result must be labeled TURN, not direct P2P. See e2e/latency/README.md for the measurement definition and all runner switches.

The target is sustained at least 30 fps and roughly 40-60 ms or less added beyond RTT, with native WebRTC measurably faster than P2P RFB. Regional numbers are evidence; they are not independent hard-coded product thresholds.

Browser diagnostics

The native receiver samples RTCPeerConnection.getStats() about every 1.5 seconds into a rolling 60-second buffer. It records selected candidate pair, RTT, available incoming bitrate, relay protocol, jitter, packet loss, frame rate/size, decoded/dropped/key frames, freezes, jitter buffer, NACK and PLI counts.

Every 15 seconds the browser posts a compact rollup. On an abnormal disconnect, it sends a final report and creates a Sentry message with a bounded reason and the last approximately 30 seconds of the local blackbox. Supported reason groups are:

SIGNALING_TIMEOUT
SIGNALING_CLOSED
ICE_FAILED
ICE_DISCONNECTED
PEER_CONNECTION_FAILED
DATA_CHANNEL_CLOSED
MEDIA_STALLED
UNKNOWN

Use the page's path badge before interpreting RTT. A low RTT on relay_udp is not a direct peer connection, and direct WSS/API relay RTT is measured by the noVNC path rather than WebRTC getStats().

Prometheus metrics

Browser telemetry feeds bounded server-side series:

Metric	Labels / value
`webrtc_client_telemetry_total`	`kind=periodic
`webrtc_client_disconnect_total`	bounded `reason` and `path`
`webrtc_client_rtt_ms`	histogram by `direct
`webrtc_client_jitter_ms`	histogram by path
`webrtc_client_session_seconds`	disconnect-time duration histogram
`webrtc_client_session_freeze_count`	disconnect-time cumulative freeze histogram

Candidate types host, srflx, and prflx map to direct; relay candidates map by relay protocol. Client-supplied strings cannot create unbounded labels, and numeric inputs are clamped before observation.

Release evidence checklist

Before promoting a connector/platform release, retain:

Scoped API, frontend, Go, Swift and device test results.
Connector -version output matching the intended commit and feature list.
Codesign identifiers, Team IDs/designated requirements and the golden-image TCC smoke result for the exact deployed binaries.
A successful clone reboot/enrollment test with no privacy prompt.
Native, P2P and relay fallback smoke results, including role/entitlement negative cases.
Regional raw latency JSON and selected ICE/codec evidence.
Prometheus and Sentry confirmation that a controlled abnormal disconnect is classified without exposing credentials or signed URLs.

Automated tests​

API​

Frontend​

Connector and native encoder​

Device provisioning and direct guard​

Manual acceptance matrix​

Regional latency harness​

Browser diagnostics​

Prometheus metrics​

Release evidence checklist​