fix(db): wait for/retry DB DNS resolution before drizzle-kit migrate (GRO-2163)
CI / Test (pull_request) Successful in 10s
CI / Lint & Typecheck (pull_request) Successful in 16s
CI / Build & Push Docker Images (pull_request) Failing after 30m28s

A fresh migrate-schema pod occasionally hits a transient CoreDNS miss
(EAI_AGAIN) on groombook-postgres-rw.<ns>.svc on its first attempt.
With backoffLimit: 2 the retry pod usually wins, but three unlucky
attempts in a row trips BackoffLimitExceeded and the Job is recreated
on every Flux reconcile (3+ Completed events observed in 8 min in uat).

Add packages/db/scripts/wait-for-db.mjs: a tiny no-deps Node 22 script
that parses DATABASE_URL, resolves the hostname via node:dns.promises
with exponential backoff (12 attempts, ~30s total) and only exits 0
once a real IP is returned. EAI_AGAIN / ENOTFOUND / EAI_NODATA are
retried; any other DNS error is surfaced so drizzle-kit gets a clear
message instead of being starved by retries.

Wire it as a pnpm `pre-migrate` (and `pre-seed` / `pre-reset`) hook
in @groombook/db so pnpm auto-runs it before any of the data-plane
commands. Mirrors the belt-and-braces pattern used in GRO-1985
(disable Corepack download fallback): do not try to outsmart CoreDNS,
just do not ask drizzle-kit to perform the very first DNS lookup of a
freshly-scheduled pod.

Defaults are env-tunable (WAIT_FOR_DB_MAX_ATTEMPTS, _BASE_DELAY_MS,
_MAX_DELAY_MS, _SKIP) so a future uat-debug pod can sidestep the
wait if needed.

Refs: GRO-2163, GRO-1985.
This commit is contained in:
Flea Flicker
2026-06-08 00:31:38 +00:00
parent f67b96ddfe
commit 323f6d6bcb
2 changed files with 107 additions and 0 deletions
+3
View File
@@ -18,7 +18,10 @@
"scripts": {
"build": "tsc --project .",
"generate": "drizzle-kit generate",
"pre-migrate": "node ./scripts/wait-for-db.mjs",
"migrate": "drizzle-kit migrate",
"pre-seed": "node ./scripts/wait-for-db.mjs",
"pre-reset": "node ./scripts/wait-for-db.mjs",
"seed": "tsx src/seed.ts",
"reset": "tsx src/reset.ts && drizzle-kit migrate && tsx src/seed.ts",
"studio": "drizzle-kit studio",