Skip to content
InnoQualis InnoQualis Docs

Runbook — InnoQualisFrontendDown

Alert

Section titled “Alert”

InnoQualisFrontendDown

Severity

Section titled “Severity”

critical

Audience

Section titled “Audience”

both

Symptom

Section titled “Symptom”

up{job="innoqualis-frontend"} == 0 for 2 minutes. Customers see browser-level “This site can’t be reached” or nginx 502.

Impact

Section titled “Impact”
  • Full outage: users cannot load hub.innoqualis.com at all.
  • API endpoints may still work if hit directly (e.g. integrations), but the UI is dead.

Diagnosis

Section titled “Diagnosis”

  1. Is the container running?

    Terminal window
    docker ps --filter name=innoqualis-frontend --format 'table {{.Names}}\t{{.Status}}'

  2. If running, hit it locally:

    Terminal window
    docker exec innoqualis-frontend curl -fsS http://localhost:3000/

  3. If crashing: docker logs --tail 200 innoqualis-frontend and chase the same path as PodCrashLoopBackOff.

Mitigation

Section titled “Mitigation”
  • Container down: docker compose -f docker/compose.yml up -d frontend
  • Build failure in container: check docker logs innoqualis-frontend for next build errors; usually a missing env var or unbuilt page.
  • Nginx routing: docker compose restart nginx.

Resolution

Section titled “Resolution”
  • Bad deploy → rollback (git revert HEAD && make prod-up).
  • Build failure → fix the build (often pnpm tsc --noEmit would have caught it; add to CI gate).

Prevention

Section titled “Prevention”
  • Same as InnoQualisBackendDown: health-check tuning, auto-restart, multi-instance post-cloud-migration.
  • CI gate on pnpm build for every PR (already in place; verify it’s not being bypassed).

Last reviewed

Section titled “Last reviewed”

2026-05-29 — ops