Skip to main content

Deployment and operations

The supported deployment is a macOS guest with a dedicated auto-login GUI user, an assigned org_machines row, and the desktop connector running as a managed Aqua LaunchAgent. Provision the golden image first, clone it while stopped, and enroll each clone separately.

Platform configuration

Required services and secrets

  • JWT_SECRET verifies Expert/Superadmin platform sessions.
  • RUNNER_JWT_SECRET signs and verifies device connector tokens. Connector token minting fails closed when it is absent.
  • PostgreSQL stores machine inventory, assignment audit and session audit.
  • Redis is required for active-active WebRTC signaling and direct WSS single-use tokens. Without Redis, WebRTC signaling is single-pod only and direct WSS falls back to the API relay.

API environment variables

VariableDefaultPurpose
REMOTE_DESKTOP_CONNECTOR_TOKEN_TTL30dMachine connector JWT lifetime
REMOTE_DESKTOP_CONNECT_TOKEN_TTL60 secondsDirect WSS single-use token; maximum accepted value is 3600
CLOUDFLARE_TURN_TOKEN_IDunsetCloudflare Realtime TURN key ID
CLOUDFLARE_TURN_API_TOKENunsetCloudflare Realtime TURN API secret
REMOTE_DESKTOP_WEBRTC_STUN_URLSCloudflare STUNWhitespace/comma/semicolon-separated STUN URLs
REMOTE_DESKTOP_TURN_URLSunsetSelf-hosted coturn URLs used when managed TURN is not configured/available
REMOTE_DESKTOP_TURN_SHARED_SECRETunsetcoturn REST HMAC secret
REMOTE_DESKTOP_TURN_TTL_SECONDS3600TURN credential lifetime; maximum 24 hours
TS_SOCKS_PROXYunsetTailscale userspace SOCKS5 proxy for relay dials to tailnet addresses
MACHINE_HEALTH_TIMEOUT_MS3000Per-machine raw TCP health timeout
MAX_MACHINE_HEALTH_PER_TICK20Machines probed by each leader-only minute tick
REMOTE_DESKTOP_TARGET_HOST / REMOTE_DESKTOP_TARGET_PORTunsetLocal test-only relay target override

The frontend may use NEXT_PUBLIC_WEBRTC_ICE_SERVERS only as a local fallback. Normal application flow uses the short-lived ICE list from /connect.

TURN selection

Configure CLOUDFLARE_TURN_TOKEN_ID and CLOUDFLARE_TURN_API_TOKEN together to prefer managed Cloudflare Realtime TURN. Credentials are generated with a five-second API timeout and cached until five minutes before expiry. If the request fails, the API degrades to configured STUN plus optional self-hosted coturn rather than rejecting a direct ICE attempt.

For coturn, set both REMOTE_DESKTOP_TURN_URLS and REMOTE_DESKTOP_TURN_SHARED_SECRET. The API generates REST-style HMAC-SHA1 credentials whose username contains the expiry and a bounded identity.

Create and assign a machine

Use the Superadmin fleet UI or the operations API. A minimal sequence is:

curl -fsS -X POST "$API_URL/admin/machines" \
-H "Authorization: Bearer $ADMIN_TOKEN" \
-H 'Content-Type: application/json' \
-d '{
"tailscaleHostname": "guest-tailnet-name",
"vncPort": 5900,
"vmName": "EU Expert Computer 01",
"jurisdiction": "EU",
"region": "eu-west",
"city": "Rome",
"labels": {"country": "IT"},
"connectMode": "proxy",
"metadata": {
"model": "Mac",
"osName": "macOS",
"osVersion": "26",
"vncUsername": "expert",
"vncPassword": "<secret>"
}
}'

curl -fsS -X POST "$API_URL/admin/machines/$MACHINE_ID/assign" \
-H "Authorization: Bearer $ADMIN_TOKEN" \
-H 'Content-Type: application/json' \
-d "{\"orgId\":\"$ORG_ID\",\"reason\":\"new Expert Computer\"}"

curl -fsS -X POST "$API_URL/admin/machines/$ORG_ID/computer-access" \
-H "Authorization: Bearer $ADMIN_TOKEN" \
-H 'Content-Type: application/json' \
-d '{"enabled":true}'

Do not put real credentials in shell history. In an operator shell, use a protected temporary file or the fleet UI for the metadata body. Assignment and entitlement are separate so a pooled machine cannot become visible by accident.

Build the guest image

Requirements

  • macOS 14 or later for the native ScreenCaptureKit/VideoToolbox package.
  • A dedicated GUI user, normally expert, with automatic login so the Aqua launchd domain exists after boot.
  • macOS Screen Sharing enabled for the guest-local RFB fallback.
  • Final, signed connector and video encoder at stable paths: /opt/humanwork/bin/hwork-desktop-connector and /opt/humanwork/bin/hwork-video-encoder.
  • No legacy /Library/LaunchDaemons/com.humanity.hwork-desktop-connector.plist.

The managed launch file belongs at /Library/LaunchAgents/com.humanity.hwork-desktop-connector.plist, is root-owned, has LimitLoadToSessionType=Aqua, and starts the connector as the logged-in user. Do not convert it to a system daemon.

TCC approval

Apple does not permit a PPPC profile to silently grant Screen Recording. A golden guest therefore needs one manual approval for the exact production code identities before it is sealed. Granting Terminal, a probe, or a wrapper does not grant the connector.

Verify the final artifacts first:

CONNECTOR=/opt/humanwork/bin/hwork-desktop-connector
ENCODER=/opt/humanwork/bin/hwork-video-encoder

test -x "$CONNECTOR" && codesign --verify --strict --verbose=2 "$CONNECTOR"
test -x "$ENCODER" && codesign --verify --strict --verbose=2 "$ENCODER"
codesign -dvvv "$CONNECTOR" 2>&1 | egrep 'Identifier=|TeamIdentifier=|CDHash='
codesign -dvvv "$ENCODER" 2>&1 | egrep 'Identifier=|TeamIdentifier=|CDHash='
codesign -dr - "$CONNECTOR"
codesign -dr - "$ENCODER"

From the guest console, while logged in as the production GUI user:

  1. Start the managed LaunchAgent so it performs capture and Accessibility preflight.
  2. In System Settings -> Privacy & Security -> Screen & System Audio Recording, add /opt/humanwork/bin/hwork-video-encoder and enable it.
  3. In Privacy & Security -> Accessibility, add /opt/humanwork/bin/hwork-desktop-connector and enable it.
  4. Relaunch the connector if macOS requests it.
  5. Verify changing video and a harmless pointer/key input through the final connector. A successful screencapture command does not prove the encoder's grant.

Optional read-only evidence:

sudo sqlite3 '/Library/Application Support/com.apple.TCC/TCC.db' \
"SELECT service,client,client_type,auth_value,auth_reason,last_modified
FROM access
WHERE service IN ('kTCCServiceScreenCapture','kTCCServiceAccessibility');"

auth_value=2 is allowed, but the end-to-end capture/input test remains the acceptance check. Never edit or transplant TCC.db, disable SIP, or automate the privacy UI.

Stop and shut down the guest before cloning. The clone retains TCC only while the local user and executable code requirements remain unchanged. Revalidate after any macOS upgrade, path/signature/identifier change, user/UID replacement, or image process that installs a fresh system.

Enroll every clone

Never put a machine token in the golden image. Boot the clone into its GUI session, ensure its unique machine row is already active and assigned, then run the enrollment script as that GUI user, not with sudo:

cd ~/humanwork
read -rs HUMANWORK_ADMIN_TOKEN && export HUMANWORK_ADMIN_TOKEN
python3 devices/mac-mini/enroll_desktop_connector.py \
--machine-id "$MACHINE_ID" \
--api-url "$API_URL"
unset HUMANWORK_ADMIN_TOKEN

Enrollment mints a machine-bound token, stores it in the login Keychain under service com.humanity.hwork-desktop-connector and account machine-token, builds and installs the Go connector and Swift encoder, installs/reloads the LaunchAgent, requires the process to remain running, and only then changes the machine's connection mode to webrtc.

Production/CI can avoid building on the guest:

python3 devices/mac-mini/enroll_desktop_connector.py \
--machine-id "$MACHINE_ID" \
--api-url "$API_URL" \
--connector-binary /secure/artifacts/hwork-desktop-connector \
--video-encoder-binary /secure/artifacts/hwork-video-encoder

Re-running enrollment rotates the Keychain token and reloads launchd idempotently. A non-dev API URL automatically derives its matching signaling WebSocket. Use --signal-url only if signaling is intentionally on another origin.

Connector configuration

VariableDefaultNotes
HWDC_SIGNAL_URLwss://api.h852.work/v1/remote-desktop/signalOutbound broker URL
HWDC_MACHINE_TOKENlogin KeychainDevelopment override only
HWDC_VNC_HOST / HWDC_VNC_PORT127.0.0.1 / 5900Guest-local RFB fallback
HWDC_ICE_SERVERSCloudflare STUNStatic fallback when authenticated ICE fetch is unavailable
HWDC_ICE_SERVERS_URLDerived from signaling originRunner-authenticated ICE endpoint
HWDC_VIDEO_ENCODER_PATH/opt/humanwork/bin/hwork-video-encoderNative encoder child
HWDC_VIDEO_ENABLEDtrueSet false for RFB-only recovery
HWDC_VIDEO_SOURCEscreensynthetic tests encode without Screen Recording
HWDC_VIDEO_FPS60Supported native range is 30-60
HWDC_VIDEO_BITRATE8000000Initial target in bits/second
HWDC_VIDEO_WIDTH / HWDC_VIDEO_HEIGHT1920 / 1080Base/high tier size
HWDC_VIDEO_SHOW_CURSORtrueSet false in production when using the browser-local cursor

The managed enrollment plist writes only HWDC_SIGNAL_URL and the non-secret static ICE fallback. It must not contain the connector token.

Build provenance

Release builds should stamp the connector:

cd remote-desktop-agent
CGO_ENABLED=1 go build \
-ldflags "-X main.version=$(git describe --tags --always) \
-X main.gitCommit=$(git rev-parse --short HEAD) \
-X main.buildTime=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
-o hwork-desktop-connector .

./hwork-desktop-connector -version emits JSON with version, commit, build time and feature list. dev/unknown, or no provenance output, identifies an unstamped or old binary and should block a production rollout.

Tailscale-backed API relay

The API relay needs a route to the machine's raw VNC endpoint. In a userspace Tailscale deployment, run tailscaled --tun=userspace-networking --state=mem: outside the NestJS process and expose its local SOCKS5 listener through TS_SOCKS_PROXY. In-memory state makes each short-lived Railway container an ephemeral Tailscale node, even with a reusable tagged auth key, so Tailscale removes it after disconnect. Never replace it with a persisted state directory, commit the key, or start tailscaled from application code.

The inventory target must be a raw VNC port, normally 5900. A 100.64.0.0/10 target is routed through the SOCKS proxy; other targets dial directly. For a local fake RFB server only, use the REMOTE_DESKTOP_TARGET_* override.

Optional direct WSS fallback

Direct WSS is a noVNC fallback, not the native WebRTC path:

browser -> wss://machine.example/vnc?token=...
-> Cloudflare edge -> cloudflared
-> http://127.0.0.1:8901 connect_guard.py
-> raw guest VNC

On the device:

  1. Install connect_guard.py with the provided LaunchDaemon and a mode-600 /opt/humanwork/env/connect-guard.env containing PLATFORM_API_URL, the org-scoped runner token, and HUMANWORK_MACHINE_ID.
  2. Install cloudflared, create a named tunnel, and protect its credential JSON with mode 600. The tunnel credential is the only additional device secret.
  3. Copy devices/mac-mini/cloudflared/config.yml.example to a protected runtime path and set tunnel UUID, credential path, hostname, and http://127.0.0.1:8901 ingress.
  4. Route DNS to the tunnel and install com.cloudflare.cloudflared.plist under /Library/LaunchDaemons.
  5. Set the machine publicHostname and connectMode: "direct_wss". The guard heartbeat sets tunnel health automatically.

Representative Cloudflare commands:

cloudflared tunnel login
cloudflared tunnel create "$TUNNEL_NAME"
sudo install -m 600 "$HOME/.cloudflared/$TUNNEL_UUID.json" \
"/opt/humanwork/env/$TUNNEL_NAME.tunnel.json"
cloudflared tunnel route dns "$TUNNEL_NAME" "$PUBLIC_HOSTNAME"
sudo launchctl bootstrap system \
/Library/LaunchDaemons/com.cloudflare.cloudflared.plist

Verify a missing token is rejected locally and the tunnel is connected:

curl -i http://127.0.0.1:8901/vnc # expected 401
cloudflared tunnel info "$TUNNEL_NAME"

Operational verification

LABEL=com.humanity.hwork-desktop-connector
UID_NOW="$(id -u)"

# Confirm the secret exists without printing it.
security find-generic-password -s "$LABEL" -a machine-token >/dev/null

# Must report running in gui/<uid>, not system/.
launchctl print "gui/$UID_NOW/$LABEL" | egrep 'state =|pid =|program ='

tail -n 50 /opt/humanwork/var/log/hwork-desktop-connector.err.log

Then verify through the platform:

  1. /admin/machines reports connectMode=webrtc and current reachability.
  2. The entitled Expert sees the machine in /workspace/computer.
  3. Auto connects on WebRTC video; the path and RTT appear.
  4. Pointer, keyboard and scrolling work; disconnect releases held state.
  5. Relay and P2P operator overrides both connect.
  6. The session appears in the page's session log with the correct transport and closes with an expected reason.
  7. Reboot the guest, allow auto-login, and repeat the checks without another privacy prompt.

Troubleshooting

SymptomCheck and recovery
Connector exits immediatelyInspect stderr log, Keychain item, signaling URL and RUNNER_JWT_SECRET; re-enroll to rotate an expired token
Machine remains unreachableConfirm the connector signaling socket is connected and launchd is in gui/<uid>; do not rely on an API-to-tailnet TCP probe for WebRTC presence
Black or absent videoVerify Screen Recording for the exact encoder identity, encoder path, H.264 output and browser codec negotiation; use HWDC_VIDEO_SOURCE=synthetic to isolate VideoToolbox from TCC
Input does nothingVerify Accessibility for the exact connector identity and that the input channel is open; revoked TCC fails closed
WebRTC always uses TURNInspect selected ICE candidates and NAT/firewall policy; TURN relay is valid but should be labeled as relay
WebRTC fails across API podsConfigure Redis and confirm pub/sub; in-process relay is single-pod only
Direct WSS is never offeredCheck Redis, public_hostname, tunnel_healthy, guard heartbeat and Cloudflare tunnel state
Direct returns 401Token may be expired, replayed or machine/org mismatched; reconnect to request a new plan, never reuse the URL
API relay failsConfirm raw vnc_port, VNC service, Tailscale SOCKS listener and TS_SOCKS_PROXY; do not point at websockify
VNC connects then closesVerify both vncUsername and vncPassword metadata for Apple ARD authentication

If a connector update changes its code requirement, quarantine affected clones, repeat manual TCC approval and rerun the complete smoke test before release.