fix: close the 20 findings of the security and reliability audit #56

Merged
marti merged 20 commits from feature/audit-fixes into develop 2026-11-13 01:27:58 +01:00
Owner

Summary

Fixes the 20 findings of the whole-app audit, one commit per finding (related ones share a commit).

sandbox      F1  host had an address on the runner's "internal" network
api          F2  percent-encoded path skipped the session and CSRF hook
             F3  one undecryptable value broke whole pages; no bulk re-encrypt   (npm run db:reencrypt)
             F4  key id collision silently kept the old key
                 an unreadable upstream OAuth token now reads as needs_reauth
             F15 OIDC auto-provision trusted an unverified email                 (OIDC_ALLOW_UNVERIFIED_EMAIL)
             F16 no warning for https without TRUST_PROXY
             F18 pool had no connect or statement timeout
data         F9  sessions never purged; no retention for tool_calls/audit_events (TOOL_CALL_RETENTION_DAYS, AUDIT_RETENTION_DAYS)
             F17 deleting a user deleted their tool-call history                 (migration 0012)
runner       F8  homes of uninstalled servers and deleted users were kept        (DELETE /users/:userId)
                 and of users who lost access, by reconciliation                 (GET /keys, POST /keys/forget)
             F7  uid isolation could switch off silently                         (ISOLATE_UIDS=true)
compose      F5  NODE_ENV from .env.example crash-looped the prod profile
             F6  prod profile ran on the dev database password
             F10 postgres had no restart policy    F11 no log size cap
web          F12 consent screen showed a brand logo for any client claiming the name
             F13 admin forms lost typed input on background refetch
             F14 one-time secret dialogs closed on a stray click
             F19 logs tab and paged lists re-rendered and refetched everything
             F20 open tabs broke after a deploy

The one change in shape is the sandbox network:

 networks:
   sandbox:
     internal: true            # no route out
+    driver_opts:              # and no host address on the bridge (Docker 28+)
+      com.docker.network.bridge.gateway_mode_ipv4: isolated

The NixOS module does the same on Docker 28+, creates the podman network with --disable-dns (netavark then adds no gateway), and drops everything from runner.subnet in the host firewall (iptables or nftables).

Runner homes follow the database by reconciliation, every 15 minutes and 10 s after an access, server or user change:

reconcileRunnerKeys
  runner.listKeys                  # first, so nothing new looks orphaned
  findOrphans (one read-only snapshot)
    server gone or not stdio  -> runner.uninstall(serverId)
    shared key, server per-user / probe or user key, server shared
    user deleted or without access (disabled users keep theirs)
                              -> runner.forgetKeys(keys)

Evidence

  • Before (F1): on Docker 29.8.1, from a container on a plain --internal network, connect 10.231.99.1:22 (the host's sshd on the bridge gateway) -> open. The new VM test probe against the previous module: AssertionError: open 10.231.71.1.
    After: all seven VM tests pass (nix build .#vm-test-docker, -podman, -docker-gvisor, -podman-gvisor, -docker-nftables, -podman-nftables, -docker25); each boots mcphub, the runner and the egress proxy and probes the sandbox. Each also removes the firewall rule and shows the probe then gets in, so the rule is what closes the host. docker25 covers a Docker too old for the option, where only the rule protects.
  • Before (F2): probe on Fastify 5.12.8: POST /%61pi/auth/login with Origin: https://evil.example reached the /api/auth/login handler while the hook saw request.url = /%61pi/auth/login and returned early.
    After: auth.test.ts "applies the CSRF check and session to a percent-encoded path": 403, no cookie set.
  • Before (F3): with one garbage ciphertext, GET /api/me/servers -> 500.
    After: encryption-rotation.test.ts: 200 with the other servers listed; reencryptAll moves values to the current key and reports the unreadable row.
  • Engine table in the docs: each row of "Requirements" in docs/deployment.md comes from running that engine in a throwaway VM: Docker 29.9.2 (no host address), 27.5.1 (up fails), 26.1.5 and 25.0.16 (option ignored, host keeps the address), Podman 6.1.4 with podman-compose 1.6.0 (up fails) and with docker-compose on its socket (option dropped silently, host keeps the address).
  • Whole suite: nix flake check passes; nix develop -c ./scripts/ci.sh passes: lint, typecheck, tests (shared 7, api 366, runner 53 + 2 skipped, web 170), build, npm audit 0 vulnerabilities. docker compose --profile prod config -q passes, also with compose.gvisor.yaml.
  • Not run: the compose prod profile itself was not started (it would have rebuilt the local mcphub:latest images); the migrate guard's shell logic was run on its own. Docker 28.x and 24 or older were not run (not in the nixpkgs pin), nor Docker 25 together with nftables. trail check reports two conflicts, both keyword matches on unrelated text: a comment in governance.ts ("a deleted user's calls stay") against the tool-kinds learning, and the VM-test sentence in docs/deployment.md against the "pure Nix base image" learning; neither decision is touched.

Merge Danger

Door: one-way

Migration 0012_tool_calls_keep_history makes tool_calls.user_id nullable with SET NULL and adds a backfilled user_email; rolling the code back needs the column changes undone by hand. The runner now deletes home directories: on uninstall, on user deletion, and for every key the reconciliation finds unbacked, which includes uninstalling a server whose row is gone or no longer stdio.

Blast Radius: deployments

Compose installs must act before upgrading:

  • POSTGRES_PASSWORD still mcphub: the migrate service now refuses to run. Change the role's password first (steps in docs/deployment.md).
  • Docker Engine older than 28: 27.x refuses to create the sandbox network; 26 and older ignore the option and keep the hole. podman-compose refuses the option, and docker-compose on Podman's socket drops it silently. The existing sandbox network is recreated on the next up.
  • MCPHUB_BIND now defaults to 127.0.0.1: a reverse proxy on another host needs MCPHUB_BIND=0.0.0.0.
  • The runner refuses to start without the capabilities for per-user uids (ISOLATE_UIDS=true), in compose and the NixOS module.

Behaviour changes inside the app: OIDC_AUTO_PROVISION no longer creates an account for an unverified email unless OIDC_ALLOW_UNVERIFIED_EMAIL=true (IdPs that never send email_verified, such as Entra by default, need it). Statements on the app pool are cancelled after 30 s. Retention is off by default (0 keeps rows forever), so the two log tables keep growing until the settings are set; an old API talking to a new runner, or the reverse, only loses the home cleanup.

🤖 Generated with Claude Code

## Summary Fixes the 20 findings of the whole-app audit, one commit per finding (related ones share a commit). ```text sandbox F1 host had an address on the runner's "internal" network api F2 percent-encoded path skipped the session and CSRF hook F3 one undecryptable value broke whole pages; no bulk re-encrypt (npm run db:reencrypt) F4 key id collision silently kept the old key an unreadable upstream OAuth token now reads as needs_reauth F15 OIDC auto-provision trusted an unverified email (OIDC_ALLOW_UNVERIFIED_EMAIL) F16 no warning for https without TRUST_PROXY F18 pool had no connect or statement timeout data F9 sessions never purged; no retention for tool_calls/audit_events (TOOL_CALL_RETENTION_DAYS, AUDIT_RETENTION_DAYS) F17 deleting a user deleted their tool-call history (migration 0012) runner F8 homes of uninstalled servers and deleted users were kept (DELETE /users/:userId) and of users who lost access, by reconciliation (GET /keys, POST /keys/forget) F7 uid isolation could switch off silently (ISOLATE_UIDS=true) compose F5 NODE_ENV from .env.example crash-looped the prod profile F6 prod profile ran on the dev database password F10 postgres had no restart policy F11 no log size cap web F12 consent screen showed a brand logo for any client claiming the name F13 admin forms lost typed input on background refetch F14 one-time secret dialogs closed on a stray click F19 logs tab and paged lists re-rendered and refetched everything F20 open tabs broke after a deploy ``` The one change in shape is the sandbox network: ```diff networks: sandbox: internal: true # no route out + driver_opts: # and no host address on the bridge (Docker 28+) + com.docker.network.bridge.gateway_mode_ipv4: isolated ``` The NixOS module does the same on Docker 28+, creates the podman network with `--disable-dns` (netavark then adds no gateway), and drops everything from `runner.subnet` in the host firewall (iptables or nftables). Runner homes follow the database by reconciliation, every 15 minutes and 10 s after an access, server or user change: ```text reconcileRunnerKeys runner.listKeys # first, so nothing new looks orphaned findOrphans (one read-only snapshot) server gone or not stdio -> runner.uninstall(serverId) shared key, server per-user / probe or user key, server shared user deleted or without access (disabled users keep theirs) -> runner.forgetKeys(keys) ``` ## Evidence - **Before (F1):** on Docker 29.8.1, from a container on a plain `--internal` network, `connect 10.231.99.1:22` (the host's sshd on the bridge gateway) -> `open`. The new VM test probe against the previous module: `AssertionError: open 10.231.71.1`. **After:** all seven VM tests pass (`nix build .#vm-test-docker`, `-podman`, `-docker-gvisor`, `-podman-gvisor`, `-docker-nftables`, `-podman-nftables`, `-docker25`); each boots mcphub, the runner and the egress proxy and probes the sandbox. Each also removes the firewall rule and shows the probe then gets in, so the rule is what closes the host. `docker25` covers a Docker too old for the option, where only the rule protects. - **Before (F2):** probe on Fastify 5.12.8: `POST /%61pi/auth/login` with `Origin: https://evil.example` reached the `/api/auth/login` handler while the hook saw `request.url = /%61pi/auth/login` and returned early. **After:** `auth.test.ts` "applies the CSRF check and session to a percent-encoded path": 403, no cookie set. - **Before (F3):** with one garbage ciphertext, `GET /api/me/servers` -> 500. **After:** `encryption-rotation.test.ts`: 200 with the other servers listed; `reencryptAll` moves values to the current key and reports the unreadable row. - **Engine table in the docs:** each row of "Requirements" in `docs/deployment.md` comes from running that engine in a throwaway VM: Docker 29.9.2 (no host address), 27.5.1 (`up` fails), 26.1.5 and 25.0.16 (option ignored, host keeps the address), Podman 6.1.4 with podman-compose 1.6.0 (`up` fails) and with docker-compose on its socket (option dropped silently, host keeps the address). - **Whole suite:** `nix flake check` passes; `nix develop -c ./scripts/ci.sh` passes: lint, typecheck, tests (shared 7, api 366, runner 53 + 2 skipped, web 170), build, `npm audit` 0 vulnerabilities. `docker compose --profile prod config -q` passes, also with `compose.gvisor.yaml`. - **Not run:** the compose prod profile itself was not started (it would have rebuilt the local `mcphub:latest` images); the migrate guard's shell logic was run on its own. Docker 28.x and 24 or older were not run (not in the nixpkgs pin), nor Docker 25 together with nftables. `trail check` reports two conflicts, both keyword matches on unrelated text: a comment in `governance.ts` ("a deleted user's calls stay") against the tool-kinds learning, and the VM-test sentence in `docs/deployment.md` against the "pure Nix base image" learning; neither decision is touched. ## Merge Danger **Door:** one-way Migration `0012_tool_calls_keep_history` makes `tool_calls.user_id` nullable with `SET NULL` and adds a backfilled `user_email`; rolling the code back needs the column changes undone by hand. The runner now deletes home directories: on uninstall, on user deletion, and for every key the reconciliation finds unbacked, which includes uninstalling a server whose row is gone or no longer stdio. **Blast Radius:** deployments Compose installs must act before upgrading: - `POSTGRES_PASSWORD` still `mcphub`: the `migrate` service now refuses to run. Change the role's password first (steps in `docs/deployment.md`). - Docker Engine older than 28: 27.x refuses to create the sandbox network; 26 and older ignore the option and keep the hole. podman-compose refuses the option, and docker-compose on Podman's socket drops it silently. The existing sandbox network is recreated on the next `up`. - `MCPHUB_BIND` now defaults to `127.0.0.1`: a reverse proxy on another host needs `MCPHUB_BIND=0.0.0.0`. - The runner refuses to start without the capabilities for per-user uids (`ISOLATE_UIDS=true`), in compose and the NixOS module. Behaviour changes inside the app: `OIDC_AUTO_PROVISION` no longer creates an account for an unverified email unless `OIDC_ALLOW_UNVERIFIED_EMAIL=true` (IdPs that never send `email_verified`, such as Entra by default, need it). Statements on the app pool are cancelled after 30 s. Retention is off by default (`0` keeps rows forever), so the two log tables keep growing until the settings are set; an old API talking to a new runner, or the reverse, only loses the home cleanup. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Settings for the audit fixes that follow: TOOL_CALL_RETENTION_DAYS and
AUDIT_RETENTION_DAYS (0 keeps rows forever, the behaviour until now), and
OIDC_ALLOW_UNVERIFIED_EMAIL for IdPs that never send email_verified.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The onRequest hook chose by the raw URL whether a request is under /api/ or /oauth/, but the router percent-decodes before matching: POST /%61pi/auth/login reached the login route with no Origin check and no session. A cross-site form post could sign a visitor into someone else's account or sign them out.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Linking an existing account needed email_verified, creating one did not, so on an IdP where users set their own address someone could claim a colleague's before their first sign-in. OIDC_ALLOW_UNVERIFIED_EMAIL restores the old behaviour for IdPs that never send the claim.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Three gaps in rotating ENCRYPTION_KEY:

- Nothing re-encrypted values in bulk, so the previous key could never be removed with confidence. npm run db:reencrypt (dist/db/reencrypt.js in the image) moves every encrypted column to the current key and lists the rows it cannot read.
- One undecryptable value made decrypt throw for a whole list: a user's /api/me/servers answered 500 and their tools/list failed. Such a value now counts as not set and is logged with its ids.
- The one-byte key id let a new key with the same id as the previous one be replaced by it (1 in 256), so everything was encrypted with the old key. Startup now refuses that pair.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Without them a stuck Postgres hung every request and /readyz instead of failing them. Migrations and db:reencrypt keep running without a statement timeout.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Behind a reverse proxy with the default TRUST_PROXY=false every client has the proxy's address, so the per-IP login limit is one bucket: ten bad sign-ins a minute from anyone lock everybody out.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
purgeExpiredSessions was never called, and tool_calls and audit_events had no retention. The 15-minute maintenance pass now removes expired sessions, and rows older than TOOL_CALL_RETENTION_DAYS and AUDIT_RETENTION_DAYS when those are set.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
tool_calls.user_id cascaded, so offboarding an account erased the record of what its agents had called. The column is now nullable with SET NULL and each row stores the caller's email; the admin list shows it with a deleted-user hint. The retention settings are described in the same Governance paragraph.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A process key's HOME and uid-map entry were never removed, so whatever a server cached with a user's credentials stayed on the runner volume forever. Uninstalling a server now forgets all of its keys, and deleting a user calls the new DELETE /users/:userId on the runner.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Following the README left a prod stack that either did not start or was open to the host:

- NODE_ENV=development from a copied .env.example made the API load pino-pretty, which is not in the image, and crash-loop. Compose now pins production.
- The migrate service refuses the dev database password (Postgres is published on loopback, so any local user could log in as the owner).
- MCPHUB_BIND defaults to 127.0.0.1 instead of every interface.
- The runner runs with ISOLATE_UIDS=true (also in the NixOS module), so it refuses to start rather than run every user's process as one uid.
- Postgres has a restart policy and every service a log size cap.

BREAKING CHANGE: an install that still uses POSTGRES_PASSWORD=mcphub must change the role's password before upgrading, and a reverse proxy on another host needs MCPHUB_BIND=0.0.0.0; see docs/deployment.md.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The server configuration and instance-values forms were keyed on updatedAt, which changes on every install step and connection error while the page polls, so typed input (a secret, mostly) was wiped. They are keyed on their content now.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A click outside or Escape closed the dialogs that show a new API token or temporary password exactly once; after a password reset that meant resetting again.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A tab opened before an upgrade asked for chunks that no longer exist and Retry asked for the same file.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Registration is open, so a client could call itself Claude and get the bundled logo on the consent screen. There a brand now needs the redirect or the verified metadata URL on the brand's hosts, or a loopback redirect with a matching name.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The Logs tab rendered every process buffer on each 3 s poll and remounted all rows once a buffer was full; paged lists refetched every loaded page on each window focus.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
fix(sandbox): keep the host off the runner's network
All checks were successful
CI / check (pull_request) Successful in 2m2s
20075d41e3
An internal bridge has no route out, but the host still held its gateway address, so a sandboxed process that ignored the proxy variables could connect to host services the firewall lets in. Confirmed on Docker 29: sshd on the gateway address answered from a container on such a network.

Compose gives the sandbox network Docker's gateway mode isolated (no host address). The NixOS module does the same on Docker 28+, creates the podman network without DNS (netavark then adds no gateway), and drops everything from runner.subnet in the host firewall. The VM test probes the gateway address; it fails against the previous module.

BREAKING CHANGE: the compose prod profile needs Docker Engine 28 or later; podman-compose refuses the network option (see docs/deployment.md).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
docs: state the Docker Engine 28 requirement of the prod profile
All checks were successful
CI / check (pull_request) Successful in 1m19s
643daf9cb0
It was a sentence inside the network explanation. It is now a Requirements section with what each engine does (28+, 27.x, 26 and older, podman-compose) and what to do, an upgrade note, and a line in the README quick start.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A token whose key is gone still read as connected, so the server stayed ready and every call failed without telling the user to sign in again. The state now checks that the app holds the ciphertext's key, and a token that fails to decrypt when used is marked.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A per-user home outlived its reason whenever a user merely lost access, a server changed between shared and per-user, or a delete happened while the runner was unreachable. The API now lists the runner's keys (GET /keys), uninstalls servers that are gone and forgets the keys the database no longer backs (POST /keys/forget), every 15 minutes and shortly after access, server and user changes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
test(nix): cover nftables and an old Docker in the sandbox VM tests
All checks were successful
CI / check (pull_request) Successful in 2m5s
caaddc388c
The module's nftables rule and its fallback for a Docker too old for gateway mode isolated had never run. Three more VM tests boot them (docker-nftables, podman-nftables, docker25), and every variant now shows the firewall rule is what closes the host: closed with it, open once it is removed.

The engine table in the deployment guide is now from runs of Docker 29, 27, 26 and 25 and of Podman 6 with podman-compose and with docker-compose on its socket. The last one is new: Podman's Docker API drops the option silently and the host keeps its address.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
marti merged commit 0086f0eac6 into develop 2026-11-13 01:27:58 +01:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
marti/mcphub!56
No description provided.