fix(GRO-2672): use drizzle-kit migrate in reset.ts to bypass HWM bug #228

2026-08-06T09:06:56Z

Flea Flicker commented

2026-08-06 09:06:56 +00:00

Root cause

drizzle-orm/postgres-js/migrator's migrate() function has the known high-water-mark (HWM) bug. When reset.ts drops the drizzle schema and calls migrate() on a fresh DB, it applies migration 0000 first — setting the HWM to when = 1773771452946 (2026-03-17). Migrations with stale 2025-era when timestamps are silently skipped:

Migration	`when`	Date	Effect when skipped
0001	1742241600000	2025-03-17	`pets.health_alerts` missing
0003	1742169600000	2025-03-17	`recurring_series` table + `appointments.series_id`/`series_index` missing
0010	1742500800000	2025-03-20	`impersonation_sessions` table missing
0011	1742587200000	2025-03-21	impersonation indexes missing

Migration 0003 is the blocker. It creates the recurring_series table and adds series_id/series_index columns to appointments. Since migrate() wraps all SQL in a single Postgres transaction, any downstream applied migration that depends on those objects fails — causing a full rollback that takes migration 0000's staff and services tables with it.

After reset exits 1, the DB has zero tables and no drizzle ledger. The next reset hits the same failure. The standalone seed-test-data K8s Job also fails because tables don't exist.

The K8s migrate-schema-552a4d9 Job uses drizzle-kit migrate (hash-based), finds the (also-missing) ledger empty, and applies all 42 migrations correctly — but reset.ts was not using the same approach.

Fix

Replace migrate() from drizzle-orm/postgres-js/migrator with pnpm exec drizzle-kit migrate (hash-based). drizzle-kit checks each migration's hash against the ledger — it applies every unhashed migration regardless of when timestamp ordering. This matches the K8s migrate Job's behaviour exactly.

One-line diff (conceptually):

-await migrate(db, { migrationsFolder: MIGRATIONS_FOLDER });
+execSync("pnpm exec drizzle-kit migrate", { stdio: "inherit", env: { ...process.env }, cwd: resolve(__dirname, "..") });

drizzle-kit reads packages/db/drizzle.config.ts (schema, migrations folder, DATABASE_URL) from the cwd. pnpm is already in PATH in the Docker reset image (the CMD is pnpm --filter @groombook/db reset).

Recovery path

Once this fix ships to prod (via the standard dev→uat→main pipeline), the next reset-demo-data CronJob run will apply all 42 migrations on a fresh DB (hash-based, no HWM skip) and re-seed successfully.

Test plan

CI passes (lint, typecheck, build)
UAT: reset-demo-data CronJob runs clean at next :45 mark — verify via pod logs that all 42 migrations are applied and seed completes (Shedward)
UAT: seed-test-data job completes successfully after reset

cc @cpfarhood

## Root cause `drizzle-orm/postgres-js/migrator`'s `migrate()` function has the known high-water-mark (HWM) bug. When `reset.ts` drops the drizzle schema and calls `migrate()` on a fresh DB, it applies migration 0000 first — setting the HWM to `when = 1773771452946` (2026-03-17). Migrations with stale 2025-era `when` timestamps are silently skipped: | Migration | `when` | Date | Effect when skipped | |-----------|--------|------|---------------------| | 0001 | 1742241600000 | 2025-03-17 | `pets.health_alerts` missing | | **0003** | 1742169600000 | 2025-03-17 | **`recurring_series` table + `appointments.series_id`/`series_index` missing** | | 0010 | 1742500800000 | 2025-03-20 | `impersonation_sessions` table missing | | 0011 | 1742587200000 | 2025-03-21 | impersonation indexes missing | **Migration 0003 is the blocker.** It creates the `recurring_series` table and adds `series_id`/`series_index` columns to `appointments`. Since `migrate()` wraps all SQL in a **single Postgres transaction**, any downstream applied migration that depends on those objects fails — causing a full rollback that takes migration 0000's `staff` and `services` tables with it. After reset exits 1, the DB has zero tables and no drizzle ledger. The next reset hits the same failure. The standalone `seed-test-data` K8s Job also fails because tables don't exist. The K8s `migrate-schema-552a4d9` Job uses `drizzle-kit migrate` (hash-based), finds the (also-missing) ledger empty, and applies all 42 migrations correctly — but `reset.ts` was not using the same approach. ## Fix Replace `migrate()` from `drizzle-orm/postgres-js/migrator` with `pnpm exec drizzle-kit migrate` (hash-based). drizzle-kit checks each migration's hash against the ledger — it applies every unhashed migration regardless of `when` timestamp ordering. This matches the K8s migrate Job's behaviour exactly. **One-line diff (conceptually):** ```diff -await migrate(db, { migrationsFolder: MIGRATIONS_FOLDER }); +execSync("pnpm exec drizzle-kit migrate", { stdio: "inherit", env: { ...process.env }, cwd: resolve(__dirname, "..") }); ``` drizzle-kit reads `packages/db/drizzle.config.ts` (schema, migrations folder, DATABASE_URL) from the `cwd`. pnpm is already in PATH in the Docker reset image (the CMD is `pnpm --filter @groombook/db reset`). ## Recovery path Once this fix ships to prod (via the standard dev→uat→main pipeline), the next `reset-demo-data` CronJob run will apply all 42 migrations on a fresh DB (hash-based, no HWM skip) and re-seed successfully. ## Test plan - [ ] CI passes (lint, typecheck, build) - [ ] UAT: `reset-demo-data` CronJob runs clean at next :45 mark — verify via pod logs that all 42 migrations are applied and seed completes (Shedward) - [ ] UAT: `seed-test-data` job completes successfully after reset cc @cpfarhood

Flea Flicker added 1 commit 2026-08-06 09:06:57 +00:00

fix(GRO-2672): use drizzle-kit migrate in reset.ts to bypass HWM bug

CI / Lint & Typecheck (pull_request) Failing after 19s

Details

CI / Test (pull_request) Successful in 20s

Details

CI / Build & Push Docker Images (pull_request) Has been skipped

Details

48de6ec7c4

drizzle-orm's migrate() has a high-water-mark (HWM) bug: on a fresh DB,
migration 0000 sets the watermark to 2026-03-17. Migrations 0001, 0003,
0010, 0011 have stale 2025-era `when` timestamps and are silently
skipped. Migration 0003 (recurring_series) is critical -- its skip
leaves `recurring_series`, `appointments.series_id`, and
`appointments.series_index` missing. A downstream migration inside
migrate()'s single Postgres transaction then fails, rolling back
everything including 0000's `staff` and `services` tables.

Replace the drizzle-orm migrate() call with `pnpm exec drizzle-kit
migrate` (hash-based). drizzle-kit applies every unhashed migration
regardless of `when` ordering, matching the K8s migrate Job exactly.

Flea Flicker added 1 commit 2026-08-06 09:12:17 +00:00

fix(GRO-2672): correct reset.ts content (raw TS, not base64)

CI / Lint & Typecheck (pull_request) Successful in 19s

Details

CI / Test (pull_request) Successful in 21s

Details

CI / Build & Push Docker Images (pull_request) Successful in 1m10s

Details

a91361296a

Flea Flicker merged commit f7f90a71fc into dev

2026-08-06 09:14:55 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: groombook/api#228