Logs, request ids and error tracking

Every response from the hub API carries an X-Request-ID header. The same id is on every log line the backend writes while handling that request, and on the error event sent to error tracking. When a customer quotes a Reference from an error message, that value is the request id.

Request ids

• The backend accepts an X-Request-ID sent by the caller (for example a proxy) only if it is 8–64 characters of letters, digits, ., _ or -. Anything else is ignored and the backend generates a new id (32 hex characters). This stops a crafted header from injecting text into the logs.
• The id is set on every response the application answers: successful ones, 4xx refusals (including ones answered by the rate limiter or upload validation), CORS preflights and 5xx errors. The one exception is the 503 uvicorn itself returns when a worker is at its UVICORN_LIMIT_CONCURRENCY limit, which is answered before the application runs.
• 4xx response bodies are unchanged. The id is only in the header.
• 5xx JSON bodies also include it as request_id.

What an unexpected error looks like

Any exception the code does not handle becomes:

HTTP/1.1 500
X-Request-ID: 3f6c0c1b9d0e4f7a8c2d5e6f7a8b9c0d

{"detail": "Something went wrong on our side. Quote the reference if you contact support.",
 "request_id": "3f6c0c1b9d0e4f7a8c2d5e6f7a8b9c0d"}

The exception text and type are never sent to the caller. The backend writes them to the log once, as a single ERROR line from the app.errors logger with the full traceback, and sends one event to error tracking (when configured). Database errors keep their "Database error. Please try again." message and also carry request_id.

Finding a request in the logs

With JSON logs (the production default), each line is one JSON object:

{"ts": "2026-09-13T21:40:02.114Z", "level": "ERROR", "logger": "app.errors",
 "msg": "Unhandled exception on POST /api/deviations",
 "request_id": "3f6c0c1b9d0e4f7a8c2d5e6f7a8b9c0d", "tenant_id": 12, "user_id": 348,
 "exc_type": "RuntimeError", "exc": "Traceback (most recent call last): ..."}

Field	Meaning
ts	UTC timestamp
level, logger, msg	Standard log level, logger name and message
request_id	The request id (empty outside a request, for example scheduled jobs)
tenant_id, user_id	Set once the request is authenticated
exc_type, exc	Exception class and traceback, when the line records an exception
http_method, path, status_code, client_addr	On access lines (logger: uvicorn.access)

Other structured fields a log call attaches appear as extra keys.

To find everything about one reference on the server:

docker logs innoqualis-backend 2>&1 | grep 3f6c0c1b9d0e4f7a8c2d5e6f7a8b9c0d

Request headers and request bodies are never logged. Access lines include the path and query string, as they did before.

Configuration

These variables are read by the backend container. docker/docker-compose.prod.yml passes only the variables it lists, so each one is mapped there.

Variable	Default (production compose)	Effect
LOG_FORMAT	json	json writes one JSON object per line; text writes the readable time LEVEL logger: message [request_id=…] format. When unset outside compose: json if ENVIRONMENT=production, otherwise text. Any other value stops the backend at startup.
LOG_LEVEL	info	debug, info, warning, error or critical. Also applies to uvicorn’s startup and access lines, so warning hides access lines. Any other value stops the backend at startup.
SENTRY_DSN	empty	Sentry-compatible DSN (self-hosted GlitchTip). Empty means error tracking is off.
SENTRY_TRACES_SAMPLE_RATE	0.1	Share of requests traced for performance (0–1).
BACKEND_RELEASE_SHA	empty	Git SHA of the deployed build, attached to error events as the release.

If JSON lines are hard to read during an incident, set LOG_FORMAT=text in the live .env and recreate the backend container. No code change is needed.

What error tracking receives

Before an event leaves the backend it is scrubbed:

• Request bodies are never attached, and cookies are removed.
• The Authorization, Cookie, Set-Cookie headers are dropped, along with any header whose name contains token, secret, api-key and similar.
• Query strings on /api/auth/... requests are removed. On every other request, and in URLs recorded in breadcrumbs and spans, the value of any query parameter whose name matches the list below is replaced with [Filtered] (for example ?code= on an integration’s sign-in callback).
• Stack traces are sent without the values of local variables.
• In extra data, contexts, breadcrumbs and user data, the value of any key containing password, token, secret, otp, code, authorization, api_key, email or cookie is replaced with [Filtered] (status_code is kept).
• Each event is tagged with request_id.

If scrubbing fails, the event is dropped rather than sent unscrubbed.