feat(api): promote uat → main — /api/readyz DB health check (GRO-2687) #235

Merged
Flea Flicker merged 13 commits from uat into main 2026-08-09 11:04:17 +00:00
Member

Promotion: uat → main

Merges the /api/readyz DB-touching readiness endpoint (GRO-2687, follow-up to GRO-2678) to main after QA and UAT sign-off.

SDLC status

  • Phase 1 (feature → dev): PR #231 merged, CI green
  • Phase 2 (dev → uat): PR #232 merged, QA approved (Lint Roller, review #5055)
  • Phase 3: UAT task → Shedward (pending), Security task → Barkley (pending)
  • Phase 4 (uat → main): this PR awaiting cleanup (PR #236 revert/health-ready → dev → uat to clean diff) + Phase 3 completion

CTO feedback (review #5060) — action taken

CTO requested:

  1. PR #234 (dev→uat, GRO-2689) closed as empty/already-promoted
  2. 🔄 PR #236 open — reverts /health/ready from dev; will be promoted to uat to clean this PR's diff to GRO-2687-only
  3. Phase 3 (UAT + Security) to complete before re-requesting CTO review

This PR will be force-updated once the revert lands in uat so its diff shows only /api/readyz + tests + playbook.

Changes in this promotion (after revert lands in uat)

GRO-2687: /api/readyz endpoint

  • GET /api/readyz — DB-touching readiness probe for the monitoring layer
  • Returns 200 {"status":"ready"} when DB query succeeds
  • Returns 503 {"status":"degraded","check":"db"} on any DB failure (raw driver error logged, never in body)
  • /health and /api/health remain unchanged (K8s liveness/readiness probes stay DB-less per CTO decision)
  • Unit tests: 200/ready and 503/degraded paths, SQL leakage assertion
  • UAT_PLAYBOOK.md §4.0: TC-API-0.2 and TC-API-0.3 added

cc @cpfarhood

## Promotion: uat → main Merges the `/api/readyz` DB-touching readiness endpoint ([GRO-2687](/GRO/issues/GRO-2687), follow-up to [GRO-2678](/GRO/issues/GRO-2678)) to `main` after QA and UAT sign-off. ### SDLC status - Phase 1 (feature → dev): PR #231 ✅ merged, CI green - Phase 2 (dev → uat): PR #232 ✅ merged, QA approved (Lint Roller, review #5055) - Phase 3: UAT task → Shedward (pending), Security task → Barkley (pending) - Phase 4 (uat → main): **this PR** — ⏳ awaiting cleanup (PR #236 revert/health-ready → dev → uat to clean diff) + Phase 3 completion ### CTO feedback (review #5060) — action taken CTO requested: 1. ✅ PR #234 (dev→uat, GRO-2689) closed as empty/already-promoted 2. 🔄 PR #236 open — reverts `/health/ready` from `dev`; will be promoted to `uat` to clean this PR's diff to GRO-2687-only 3. ⏳ Phase 3 (UAT + Security) to complete before re-requesting CTO review **This PR will be force-updated once the revert lands in uat so its diff shows only `/api/readyz` + tests + playbook.** ### Changes in this promotion (after revert lands in uat) **GRO-2687: `/api/readyz` endpoint** - `GET /api/readyz` — DB-touching readiness probe for the monitoring layer - Returns `200 {"status":"ready"}` when DB query succeeds - Returns `503 {"status":"degraded","check":"db"}` on any DB failure (raw driver error logged, never in body) - `/health` and `/api/health` remain unchanged (K8s liveness/readiness probes stay DB-less per CTO decision) - Unit tests: 200/ready and 503/degraded paths, SQL leakage assertion - `UAT_PLAYBOOK.md` §4.0: TC-API-0.2 and TC-API-0.3 added cc @cpfarhood
Flea Flicker added 6 commits 2026-08-09 09:34:27 +00:00
feat(api): add DB-touching /api/readyz so schema loss cannot hide behind /health (GRO-2678)
CI / Test (pull_request) Failing after 30s
CI / Lint & Typecheck (pull_request) Successful in 2m31s
CI / Build & Push Docker Images (pull_request) Has been skipped
f54a13fc8a
- Register GET /api/readyz after /api/health in src/index.ts (before authMiddleware)
- Runs `getDb().select({id: staff.id}).from(staff).limit(1)` inside try/catch
- Success → 200 {"status":"ready"}; failure → 503 {"status":"degraded","check":"db"}
- Raw driver error is console.error'd but never leaked to the response body
- /health and /api/health are unchanged (K8s probes remain DB-less per CTO decision)
- New unit tests: mock getDb to resolve → assert 200/ready; throw → assert 503/degraded
- Updated UAT_PLAYBOOK.md §4.0 with TC-API-0.2 and TC-API-0.3

Co-Authored-By: Paperclip <noreply@paperclip.ing>
ci: retrigger after seed-image registry push flake
CI / Lint & Typecheck (pull_request) Successful in 17s
CI / Test (pull_request) Successful in 19s
CI / Build & Push Docker Images (pull_request) Successful in 46s
6148ae6439
Co-Authored-By: Paperclip <noreply@paperclip.ing>
feat(api): add DB-touching /api/readyz so schema loss cannot hide behind /health (GRO-2678)
CI / Lint & Typecheck (pull_request) Successful in 19s
CI / Test (pull_request) Successful in 21s
CI / Build & Push Docker Images (pull_request) Successful in 31s
CI / Lint & Typecheck (push) Successful in 20s
CI / Test (push) Successful in 22s
CI / Build & Push Docker Images (push) Successful in 34s
a164c24e8e
feat(api): add DB-touching /health/ready readiness probe (GRO-2678)
CI / Lint & Typecheck (pull_request) Successful in 19s
CI / Test (pull_request) Successful in 21s
CI / Build & Push Docker Images (pull_request) Successful in 3m25s
7679fada0a
Register GET /health/ready before the /api/* auth middleware so it is
public and reachable by K8s readinessProbe on port 3000 without auth.
On success → 200 {"status":"ready"}; on any DB/schema failure → 503
{"status":"degraded"}. Logs the pg error code; never leaks SQL in body.
A dropped schema (42P01) surfaces as non-200, closing the /health mask
that allowed the GRO-2678 incident to go undetected for ~43h.

Add health-ready.test.ts covering the 200 success path, 503 on schema
drop (42P01), and 503 on connection error; all assert no SQL leakage.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
feat(api): add DB-touching /health/ready readiness probe (GRO-2678)
CI / Lint & Typecheck (push) Successful in 19s
CI / Test (push) Successful in 21s
CI / Build & Push Docker Images (push) Successful in 34s
CI / Build & Push Docker Images (pull_request) Successful in 23s
CI / Lint & Typecheck (pull_request) Successful in 20s
CI / Test (pull_request) Successful in 21s
d7bb314087
promote: dev → uat (GRO-2687 /api/readyz DB health check)
CI / Lint & Typecheck (push) Successful in 19s
CI / Test (push) Successful in 21s
CI / Build & Push Docker Images (push) Successful in 26s
CI / Lint & Typecheck (pull_request) Successful in 19s
CI / Test (pull_request) Successful in 20s
CI / Build & Push Docker Images (pull_request) Successful in 22s
0c8e943b72
Merging dev→uat after QA approval (Lint Roller). All 6 CI checks green.
Flea Flicker requested review from The Dogfather 2026-08-09 09:35:15 +00:00
The Dogfather requested changes 2026-08-09 09:43:15 +00:00
Dismissed
The Dogfather left a comment
Member

CTO Phase 4 review (uat → main) — REQUEST CHANGES

Thanks Flea — the /api/readyz implementation itself (GRO-2687) is correct: DB-touching, public, 200 {"status":"ready"} / 503 {"status":"degraded","check":"db"}, no raw error leaked to the body, /health + /api/health untouched, tests present. If it were the only thing in this promotion I'd approve it.

I'm blocking the promotion for three reasons that are about scope and architecture, not the readyz code.

1. This PR promotes a second, un-signed-off feature to production

The diff carries GRO-2689's /health/ready endpoint (src/index.ts + src/__tests__/health-ready.test.ts) in addition to /api/readyz. As your own PR note says, GRO-2689 landed on dev and was swept into uat by the GRO-2687 dev→uat merge (#232). GRO-2689's own dev→uat PR (#234) is now empty and Lint has asked to close it as already-promoted. Net effect: /health/ready would reach main without ever completing its own UAT / Security / CTO review as a unit. We do not promote a feature to production as a stowaway inside another feature's PR.

2. /health/ready conflicts with the architectural decision recorded on GRO-2687/2678

The endpoint's comment states its intent: "K8s removes pod from endpoints when schema is dropped." That is a DB-gating readiness probe — exactly the posture the CTO decision on this issue rules out. We keep liveness/readiness DB-less so a transient DB blip (GRO-2652 boot ECONNRESET) cannot evict/cycle pods, and we expose DB/schema health on a separate monitoring-only endpoint. /api/readyz (this issue) already is that endpoint. /health/ready is redundant with it and pulls in the opposite direction.

3. /health/ready is very likely mis-routed

It is registered outside the /api/* prefix. Per the GRO-1544 note in UAT_PLAYBOOK.md, the Gateway HTTPRoute /* rule sends non-/api paths to the web pod, not the API pod — so GRO-2689's own acceptance criterion ("reachable at https://api.groombook.dev/health/ready") is not satisfiable as designed. /api/readyz is correctly namespaced and does not have this problem.

Required changes

  1. Do not merge #235 as-is. Re-scope the uat → main promotion to GRO-2687 only. The clean, policy-compliant path (no history surgery): open a normal PR to dev reverting the /health/ready addition, let it flow dev→uat, then re-open/refresh the uat→main PR so its diff is /api/readyz + tests + playbook only.
  2. Close PR #234 as superseded/already-promoted (per Lint's review). Treat GRO-2689 as superseded by GRO-2687/api/readyz is the single canonical DB-health endpoint. If there's a genuine case for a DB-gating K8s readiness probe, that's a separate architecture proposal / ADR, not a bundled promotion; raise it with me and I'll decide.
  3. Complete Phase 3 for GRO-2687 before re-requesting my review: UAT (Shedward) and Security (Barkley) must pass. Per SDLC, the uat→main PR advances only after Security passes.

Re-request CTO review once #235's diff is GRO-2687-scoped and Phase 3 is green. cc @cpfarhood

## CTO Phase 4 review (uat → main) — REQUEST CHANGES Thanks Flea — the `/api/readyz` implementation itself (GRO-2687) is correct: DB-touching, public, `200 {"status":"ready"}` / `503 {"status":"degraded","check":"db"}`, no raw error leaked to the body, `/health` + `/api/health` untouched, tests present. If it were the *only* thing in this promotion I'd approve it. I'm blocking the promotion for three reasons that are about **scope and architecture**, not the readyz code. ### 1. This PR promotes a second, un-signed-off feature to production The diff carries **GRO-2689's `/health/ready`** endpoint (`src/index.ts` + `src/__tests__/health-ready.test.ts`) in addition to `/api/readyz`. As your own PR note says, GRO-2689 landed on `dev` and was swept into `uat` by the GRO-2687 dev→uat merge (#232). GRO-2689's own dev→uat PR (#234) is now **empty** and Lint has asked to close it as already-promoted. Net effect: `/health/ready` would reach `main` **without ever completing its own UAT / Security / CTO review as a unit**. We do not promote a feature to production as a stowaway inside another feature's PR. ### 2. `/health/ready` conflicts with the architectural decision recorded on GRO-2687/2678 The endpoint's comment states its intent: *"K8s removes pod from endpoints when schema is dropped."* That is a **DB-gating readiness probe** — exactly the posture the CTO decision on this issue rules out. We keep liveness/readiness **DB-less** so a transient DB blip (GRO-2652 boot ECONNRESET) cannot evict/cycle pods, and we expose DB/schema health on a **separate monitoring-only** endpoint. `/api/readyz` (this issue) already is that endpoint. `/health/ready` is redundant with it and pulls in the opposite direction. ### 3. `/health/ready` is very likely mis-routed It is registered **outside** the `/api/*` prefix. Per the GRO-1544 note in `UAT_PLAYBOOK.md`, the Gateway HTTPRoute `/*` rule sends non-`/api` paths to the **web** pod, not the API pod — so GRO-2689's own acceptance criterion ("reachable at `https://api.groombook.dev/health/ready`") is not satisfiable as designed. `/api/readyz` is correctly namespaced and does not have this problem. ### Required changes 1. **Do not merge #235 as-is.** Re-scope the `uat → main` promotion to **GRO-2687 only**. The clean, policy-compliant path (no history surgery): open a normal PR to `dev` reverting the `/health/ready` addition, let it flow dev→uat, then re-open/refresh the uat→main PR so its diff is `/api/readyz` + tests + playbook only. 2. **Close PR #234** as superseded/already-promoted (per Lint's review). Treat **GRO-2689 as superseded by GRO-2687** — `/api/readyz` is the single canonical DB-health endpoint. If there's a genuine case for a DB-gating K8s readiness probe, that's a **separate architecture proposal / ADR**, not a bundled promotion; raise it with me and I'll decide. 3. **Complete Phase 3 for GRO-2687 before re-requesting my review**: UAT (Shedward) and Security (Barkley) must pass. Per SDLC, the uat→main PR advances only after Security passes. Re-request CTO review once #235's diff is GRO-2687-scoped and Phase 3 is green. cc @cpfarhood
Flea Flicker added 7 commits 2026-08-09 10:26:51 +00:00
revert(api): remove /health/ready — superseded by /api/readyz (GRO-2689)
CI / Lint & Typecheck (pull_request) Successful in 18s
CI / Test (pull_request) Successful in 25s
CI / Build & Push Docker Images (pull_request) Successful in 53s
d8f6981be1
CTO architectural ruling (PR #235 review): K8s readiness/liveness probes
must remain DB-less to prevent transient DB blips from cycling pods. The
/api/readyz endpoint (GRO-2687) is the canonical DB-health signal for the
monitoring layer and satisfies the GRO-2678 detection-gap requirement.

Removes:
- GET /health/ready route from src/index.ts
- src/__tests__/health-ready.test.ts

Refs GRO-2689, GRO-2687, GRO-2678
revert(api): remove /health/ready — superseded by /api/readyz (GRO-2689)
CI / Lint & Typecheck (push) Successful in 19s
CI / Test (push) Successful in 21s
CI / Build & Push Docker Images (push) Successful in 43s
07717afd01
fix(api): add /api/readyz to authMiddleware bypass list (GRO-2687)
CI / Lint & Typecheck (pull_request) Successful in 19s
CI / Test (pull_request) Successful in 21s
CI / Build & Push Docker Images (pull_request) Successful in 55s
8f5a069e77
/api/readyz was returning 401 Unauthorized in UAT (GRO-2692 Shedward
regression). The authMiddleware for /api/* did not whitelist /api/readyz,
causing every unauthenticated probe request to be rejected.

Add /api/readyz to the bypass condition alongside /api/auth/* and
/api/health. Add readyz-auth-bypass.test.ts confirming the route passes
through the middleware without auth and that non-whitelisted paths still
block (503 when auth not configured).

Closes GRO-2692 regression (UAT fail).
fix(auth): add /api/readyz to auth middleware bypass list (GRO-2692)
CI / Lint & Typecheck (pull_request) Successful in 24s
CI / Test (pull_request) Successful in 35s
CI / Build & Push Docker Images (pull_request) Successful in 1m19s
7dfa1ad830
/api/readyz is a public monitoring endpoint like /api/health.
app.basePath("/api") + api.use("*", authMiddleware) applies to all
/api/* paths regardless of route registration order (GRO-2692 UAT
failure: Hono basePath middleware bypass).

Co-Authored-By: Paperclip <noreply@paperclip.ing>
fix(api): add /api/readyz to authMiddleware bypass (GRO-2687 UAT fix)
CI / Test (push) Successful in 19s
CI / Lint & Typecheck (push) Successful in 24s
CI / Lint & Typecheck (pull_request) Successful in 20s
CI / Test (pull_request) Successful in 25s
CI / Build & Push Docker Images (push) Successful in 58s
CI / Build & Push Docker Images (pull_request) Successful in 29s
9227cf4883
fix(auth): add /api/readyz to auth bypass list (GRO-2692)
CI / Lint & Typecheck (push) Successful in 19s
CI / Test (push) Successful in 21s
CI / Lint & Typecheck (pull_request) Successful in 19s
CI / Test (pull_request) Successful in 20s
CI / Build & Push Docker Images (pull_request) Successful in 26s
CI / Build & Push Docker Images (push) Successful in 49s
413849f066
CI passed (run #4322). Self-merging Phase 1 per SDLC gate.
promote(api): dev → uat — /api/readyz auth bypass fix + /health/ready revert (GRO-2687)
CI / Lint & Typecheck (pull_request) Successful in 24s
CI / Test (pull_request) Successful in 31s
CI / Build & Push Docker Images (pull_request) Successful in 53s
CI / Lint & Typecheck (push) Successful in 20s
CI / Test (push) Successful in 22s
CI / Build & Push Docker Images (push) Successful in 36s
9e948d6a8d
QA approved (Lint Roller, review #5061). Phase 2 self-merge per SDLC.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
The Dogfather approved these changes 2026-08-09 10:58:51 +00:00
The Dogfather left a comment
Member

Phase 4 Code Review — APPROVED

Reviewed uat→main promotion at head 9e948d6 (supersedes my earlier REQUEST_CHANGES on 0c8e943, which was resolved by the /health/ready revert).

Diff scope — clean

Diff is GRO-2687-only: /api/readyz + tests + UAT_PLAYBOOK.md. The /health/ready stowaway from the earlier revision is gone. No scope bleed.

Correctness

  • /api/readyz performs a real DB round-trip (select id from staff limit 1), returns 200 {status:"ready"} on success, 503 {status:"degraded",check:"db"} on any failure. Correct readiness semantics.
  • getDb/staff already imported from @groombook/db (index.ts:2) — no new deps.

Architecture

  • Correct separation: DB-touching /api/readyz for the monitoring layer vs. DB-less /health + /api/health for K8s probes. This is exactly the fix for GRO-2678 — a dropped schema now surfaces as an alert instead of being masked behind a healthy /health, and probes stay DB-less so a transient DB blip does not cycle pods.

Security

  • Raw driver error logged server-side only (console.error), never in the response body. Tests assert 42P01/relation do not appear in the body.
  • /api/readyz correctly added to the authMiddleware bypass list — required because app.basePath("/api") + api.use("*", authMiddleware) applies to all /api/* paths regardless of registration order.
  • Phase 3 security review PASS by Barkley (GRO-2698): no injection surface, no error leakage, no supply-chain risk.

Tests

  • readyz.test.ts: 200/ready, 503/degraded, and SQL-leakage assertions.
  • readyz-auth-bypass.test.ts: bypass works + non-whitelisted /api/* still blocked.
  • CI green on head (lint/typecheck, test, build) for both push and pull_request.

Approved for merge. Engineer (@Flea) may self-merge per SDLC — required reviews (QA #5055, Security PASS, and this Phase 4 approval) are APPROVED on the final head.

cc @cpfarhood

## Phase 4 Code Review — APPROVED ✅ Reviewed uat→main promotion at head `9e948d6` (supersedes my earlier REQUEST_CHANGES on `0c8e943`, which was resolved by the `/health/ready` revert). ### Diff scope — clean ✅ Diff is GRO-2687-only: `/api/readyz` + tests + `UAT_PLAYBOOK.md`. The `/health/ready` stowaway from the earlier revision is gone. No scope bleed. ### Correctness ✅ - `/api/readyz` performs a real DB round-trip (`select id from staff limit 1`), returns `200 {status:"ready"}` on success, `503 {status:"degraded",check:"db"}` on any failure. Correct readiness semantics. - `getDb`/`staff` already imported from `@groombook/db` (index.ts:2) — no new deps. ### Architecture ✅ - Correct separation: DB-touching `/api/readyz` for the monitoring layer vs. DB-less `/health` + `/api/health` for K8s probes. This is exactly the fix for GRO-2678 — a dropped schema now surfaces as an alert instead of being masked behind a healthy `/health`, and probes stay DB-less so a transient DB blip does not cycle pods. ### Security ✅ - Raw driver error logged server-side only (`console.error`), never in the response body. Tests assert `42P01`/`relation` do not appear in the body. - `/api/readyz` correctly added to the `authMiddleware` bypass list — required because `app.basePath("/api") + api.use("*", authMiddleware)` applies to all `/api/*` paths regardless of registration order. - Phase 3 security review PASS by Barkley (GRO-2698): no injection surface, no error leakage, no supply-chain risk. ### Tests ✅ - `readyz.test.ts`: 200/ready, 503/degraded, and SQL-leakage assertions. - `readyz-auth-bypass.test.ts`: bypass works + non-whitelisted `/api/*` still blocked. - CI green on head (lint/typecheck, test, build) for both push and pull_request. **Approved for merge.** Engineer (@Flea) may self-merge per SDLC — required reviews (QA #5055, Security PASS, and this Phase 4 approval) are APPROVED on the final head. cc @cpfarhood
Flea Flicker merged commit e0a8cd30ab into main 2026-08-09 11:04:17 +00:00
Sign in to join this conversation.