Merge pull request 'fix(GRO-2123): serialize seed.ts with Postgres advisory lock' (#155) from flea-flicker/gro-2123-seed-advisory-lock into dev

The reset-demo-data CronJob in groombook-uat intermittently failed with
FK 23503 on invoice_tip_splits because two pods could run the seed
concurrently: the new pod's TRUNCATE deleted rows the old pod was still
inserting.

Acquire a session-level advisory lock for the full duration of the seed.
CRITICAL: with postgres-js connection pooling, a pg_advisory_lock
acquired on one pooled connection and released on a different one is a
no-op (the lock is bound to the pg-backend that took it). We therefore
reserve a dedicated connection for the lock, take pg_advisory_lock(KEY)
on it, run the seed on the pooled connections, and release the lock +
reserved connection in a try/finally so a thrown seed error cannot leak
the lock or the connection.

Defence-in-depth with the infra PR that switches
concurrencyPolicy: Replace → Forbid on the reset-demo-data CronJob.

- Adds withSeedAdvisoryLock helper and runSeedBody extracted function
- Wraps seed() body in the helper; client.end() runs after the lock
  releases so a reserved connection is not returned to a closed pool
- SEED_ADVISORY_LOCK_KEY = 0x47524f4f ("GROO" in ASCII) — arbitrary
  stable 32-bit key, referenced in runbooks
- UAT_PLAYBOOK.md §3.29 documents the regression check

cc @cpfarhood

Co-Authored-By: Paperclip <noreply@paperclip.ing>

UAT_PLAYBOOK.md

@@ -166,6 +166,7 @@ Expected: one row, `role = 'groomer'`. If zero rows return, the request hit the
 | TC-API-3.26 | Verify 25-35% medicalAlerts distribution | GET /api/pets (first 30 pets), count how many have non-empty medicalAlerts | Ratio is 25-35% (seed uses rand() < 0.3 for ~30% distribution) |
 | TC-API-3.27 | Verify coat_type enum has all seed values | After UAT seed completes, inspect the coat_type enum on the UAT DB — it must contain: short, medium, long, double, wire, silky, curly, hairless | UAT seed jobs (`reset-demo-data`, `seed-test-data`) complete 1/1 with no `enum_in` error; coat_type includes all 8 values used by seed.ts `coatTypePool` |
 | TC-API-3.28 | Verify pet_size_category enum has all seed values | After UAT seed completes, inspect the pet_size_category enum on the UAT DB — it must contain: small, medium, large, extra_large | UAT seed jobs (`reset-demo-data`, `seed-test-data`) complete 1/1 with no `enum_in` error; pet_size_category includes all 4 values used by seed.ts `petSizeCategoryPool` (regression for GRO-1999, mirrors TC-API-3.27) |
+| TC-API-3.29 | Verify `reset-demo-data` CronJob does not fail with FK 23503 on `invoice_tip_splits` (GRO-2123) | Trigger the CronJob manually: `kubectl create job --from=cronjob/reset-demo-data verify-gro2123 -n groombook-uat`. Wait for pod to terminate. Inspect logs: `kubectl logs -n groombook-uat -l job-name=verify-gro2123` | Pod reaches `Completed` state; logs show `✓ Acquired seed advisory lock` and `✓ Released seed advisory lock` from `seed.ts`; no `PostgresError: … violates foreign key constraint "invoice_tip_splits_invoice_id_invoices_id_fk"` (code 23503); final counts unchanged (500 clients, ~4000 invoices) |
 
 ### 4.4 Appointment Scheduling
 
@@ -192,6 +193,33 @@ Expected: one row, `role = 'groomer'`. If zero rows return, the request hit the
 | TC-API-5.4 | Update service | PATCH /api/services/{id} with updated fields | 200 OK, service updated |
 | TC-API-5.5 | Delete service | DELETE /api/services/{id} | 200 OK, service deleted |
 
+#### 4.5.1 Seed/Reset idempotency (GRO-2064)
+
+Services seeding is now keyed on the deterministic `services.id` (not `name`) and
+the reset path now `TRUNCATE`s `services` alongside the other dynamic tables.
+This means:
+
+- Running the seed Job twice in a row (no reset in between) converges to the
+  same catalogue — no `services_pkey` collision.
+- A `pnpm reset` followed by `pnpm seed` (or a CronJob reset fire) leaves the
+  catalogue exactly matching `servicesDef` (10 rows, ids `b0000001-…-001` …
+  `…-00a`), regardless of any stale rows that were present beforehand.
+- Mixed `seedKnownUsers` + full `seed()` invocations are safe — the
+  `demoSvcs` subset (Bath & Brush, Full Groom Small/Medium, Nail Trim) is
+  keyed on ids `…-001`, `…-002`, `…-003`, `…-005` and the upsert target
+  is `services.id`, so the same-id / different-name collision that broke
+  GRO-2033 (id `…-004` = "Nail Trim" vs servicesDef `…-004` =
+  "Full Groom — Large") cannot recur.
+
+**UAT regression** (verify after a new image is rolled out):
+
+| # | Scenario | Steps | Expected |
+|---|----------|-------|----------|
+| TC-SEED-1 | Reset → seed converges | `kubectl -n groombook exec deploy/api -- pnpm reset && pnpm seed` | Seed completes 1/1, `services` count = 10, all ids match `servicesDef` |
+| TC-SEED-2 | Idempotent re-seed | Re-run `pnpm seed` without reset | Seed completes 1/1, no `services_pkey` errors, `services` count still 10 |
+| TC-SEED-3 | Catalogue matches servicesDef | `psql -c "SELECT id, name FROM services ORDER BY id"` | Rows `…-001`…`…-00a` with names "Bath & Brush"…"Sanitary Trim" exactly as in `servicesDef` |
+| TC-SEED-4 | Demo subset coexists | Run `seedKnownUsers` then full `seed` | No collision, demo subset (4 services) ends up with the same rows the full seed would write |
+
 ### 4.6 Staff Management
 
 | # | Scenario | Steps | Expected |

apps/api/src/db/seed.ts

@@ -636,21 +636,28 @@ async function seedKnownUsers() {
     }
   }
 
-  // ── Services: idempotent upsert using name as unique key ─────────────────────
-  // UNIQUE constraint on services.name (migration 0020) must exist first.
-  // Uses b0000001-... IDs to match main seed servicesDef for same-named services.
+  // ── Services: idempotent upsert keyed on `id` ─────────────────────────────
+  // GRO-2064: previously keyed on `services.name` while writing a
+  // deterministic `id`. If a stale row existed with the same `id` but a
+  // different `name`, PostgreSQL raised `services_pkey` (id collision)
+  // before the name-targeted ON CONFLICT could fire. Switch the conflict
+  // target to `services.id` so deterministic ids always win; pair with
+  // `TRUNCATE services … CASCADE` above so each reset rebuilds the
+  // catalogue from `servicesDef` cleanly. GRO-2033 close-out.
+  // Id↔name map MUST stay in sync with `servicesDef` (the canonical source
+  // of truth in the main `seed()` function).
   const demoSvcs = [
     { id: "b0000001-0000-0000-0000-000000000001", name: "Bath & Brush", description: "Full bath, blow-dry, brush out, and ear cleaning", basePriceCents: 4500, durationMinutes: 45 },
     { id: "b0000001-0000-0000-0000-000000000002", name: "Full Groom — Small", description: "Complete grooming for dogs under 25 lbs", basePriceCents: 6500, durationMinutes: 60 },
     { id: "b0000001-0000-0000-0000-000000000003", name: "Full Groom — Medium", description: "Complete grooming for dogs 25-50 lbs", basePriceCents: 8000, durationMinutes: 75 },
-    { id: "b0000001-0000-0000-0000-000000000004", name: "Nail Trim", description: "Nail clipping and filing", basePriceCents: 1500, durationMinutes: 15 },
+    { id: "b0000001-0000-0000-0000-000000000005", name: "Nail Trim", description: "Nail clipping and filing", basePriceCents: 1500, durationMinutes: 15 },
   ];
   for (const svc of demoSvcs) {
     await db.insert(schema.services)
       .values({ ...svc, active: true })
       .onConflictDoUpdate({
-        target: schema.services.name,
-        set: { description: svc.description, basePriceCents: svc.basePriceCents, durationMinutes: svc.durationMinutes, active: true },
+        target: schema.services.id,
+        set: { name: svc.name, description: svc.description, basePriceCents: svc.basePriceCents, durationMinutes: svc.durationMinutes, active: true },
       });
   }
   console.log(`✓ Seeded ${demoSvcs.length} services`);
@@ -757,7 +764,13 @@ async function seed() {
     ({ id: uuid(), name: `Bather ${i + 1}`, email: `bather${i + 1}@groombook.dev`, role: "groomer" as const, isSuperUser: false })
   );
 
-  await db.execute(sql`TRUNCATE impersonation_sessions, impersonation_audit_logs, appointments, invoices, invoice_line_items, invoice_tip_splits, grooming_visit_logs CASCADE`);
+  // GRO-2064: also TRUNCATE `services` so each reset rebuilds the catalogue
+  // from `servicesDef` (deterministic IDs + UNIQUE(name)). Stale service rows
+  // (e.g. a prior `seedKnownUsers` run that wrote a different `name` for the
+  // same `id`) would otherwise cause the deterministic upsert to PK-collide
+  // on `services.id` — see CTO review on infra PR #605 (rev #4230). TRUNCATE
+  // CASCADE handles appointments/invoices FKs to services.id.
+  await db.execute(sql`TRUNCATE services, impersonation_sessions, impersonation_audit_logs, appointments, invoices, invoice_line_items, invoice_tip_splits, grooming_visit_logs CASCADE`);
 
   const allStaff = [...managerStaff, ...receptionistStaff, ...groomers, ...bathers];
   for (const s of allStaff) {
@@ -828,9 +841,11 @@ async function seed() {
   }
 
   // ── Services ──
-  // Upsert services using name as unique key. With deterministic IDs in
-  // servicesDef and TRUNCATE clearing downstream tables first, this is
-  // idempotent: first run inserts, subsequent runs update existing rows.
+  // GRO-2064: key the upsert on `services.id` (not `name`) so deterministic
+  // ids always win, and rely on the TRUNCATE above to clear stale rows before
+  // the catalogue is rebuilt. The previous name-targeted upsert failed with
+  // `services_pkey` when a prior run had left a row with the same id but a
+  // different name (CTO review on infra PR #605, rev #4230).
   const serviceIds: string[] = [];
   for (const s of servicesDef) {
     serviceIds.push(s.id);
@@ -844,8 +859,8 @@ async function seed() {
         active: true,
       })
       .onConflictDoUpdate({
-        target: schema.services.name,
-        set: { description: s.desc, basePriceCents: s.price, durationMinutes: s.dur, active: true },
+        target: schema.services.id,
+        set: { name: s.name, description: s.desc, basePriceCents: s.price, durationMinutes: s.dur, active: true },
       });
   }
   console.log(`✓ Created ${servicesDef.length} services`);

packages/db/src/seed.ts

@@ -401,7 +401,9 @@ const servicesDef = [
  *
  * In seedKnownUsers() this replaces the inline UAT-staff block.
  */
-async function seedUatStaffAccounts(db: ReturnType<typeof drizzle>) {
+async function seedUatStaffAccounts(
+  db: ReturnType<typeof drizzle>,
+): Promise<string | null> {
   // ── Staff: UAT Super User (oidcSub from SEED_UAT_SUPER_OIDC_SUB env var) ──
   const uatSuperOidcSub = process.env.SEED_UAT_SUPER_OIDC_SUB;
   if (uatSuperOidcSub) {
@@ -668,6 +670,132 @@ async function seedUatStaffAccounts(db: ReturnType<typeof drizzle>) {
       console.log(`✓ Created UAT pet '${pet.name}' with extended fields`);
     }
   }
+
+  // ── GRO-2100: deterministic uat-groomer ↔ pet linkage ───────────────────────
+  // The UAT groomer (`uat-groomer@groombook.dev`, staffId 00000000-0000-0000-0000-000000000004)
+  // needs at least one linked pet/appointment or GRO-1987 TC-UAT-2/3 cannot run
+  // (the pet profile-summary endpoint returns 404 instead of 200/403).
+  //
+  // We deterministically link the UAT groomer to the UAT customer's first pet
+  // ("UAT Pup Alpha") and leave the second pet ("UAT Pup Beta") UNLINKED so
+  // TC-UAT-2 (200) and TC-UAT-3 (403) can both hardcode the stable petIds.
+  //
+  // The linkage call itself is performed by the caller AFTER the `services`
+  // catalogue has been seeded (this helper runs before services exist,
+  // which previously caused the linkage to be silently skipped on every
+  // reset). GRO-2100 follow-up.
+  return uatCustomerClientId;
+}
+
+/**
+ * GRO-2100: create a deterministic completed appointment linking the UAT groomer
+ * to "UAT Pup Alpha" (c0000001-0000-0000-0000-000000000002). "UAT Pup Beta"
+ * (c0000001-0000-0000-0000-000000000003) is intentionally left UNLINKED so
+ * GRO-1987 TC-UAT-3 can verify the 403 forbidden response.
+ *
+ * Idempotent: the deterministic appointment id (`a0000001-…-0001`) is the
+ * upsert key, so re-running the seed on every reset-demo-data CronJob
+ * (hourly per apps/overlays/uat/reset-cronjob.yaml) is safe.
+ */
+async function seedUatGroomerLinkage(
+  db: ReturnType<typeof drizzle>,
+  customerClientId: string | null,
+): Promise<void> {
+  const uatGroomerEmail = "uat-groomer@groombook.dev";
+  const LINKED_PET_ID = "c0000001-0000-0000-0000-000000000002"; // UAT Pup Alpha
+  const APPT_ID = "a0000001-0000-0000-0000-000000000001";
+
+  // Skip silently if the UAT Customer client wasn't created (non-UAT seed
+  // profile, e.g. seedKnownUsers() in an env without the UAT personas).
+  if (!customerClientId) {
+    return;
+  }
+
+  // Only run if the UAT groomer staff record actually exists — dev/test seeds
+  // that don't set SEED_UAT_STAFF_OIDC_SUB should not crash.
+  const [uatGroomerStaff] = await db
+    .select({ id: schema.staff.id })
+    .from(schema.staff)
+    .where(eq(schema.staff.email, uatGroomerEmail))
+    .limit(1);
+  if (!uatGroomerStaff) {
+    return;
+  }
+
+  // Skip if this exact appointment already exists (idempotent on re-seed).
+  const [existing] = await db
+    .select({ id: schema.appointments.id })
+    .from(schema.appointments)
+    .where(eq(schema.appointments.id, APPT_ID))
+    .limit(1);
+  if (existing) {
+    console.log(`✓ GRO-2100: uat-groomer linkage appointment already exists — skipping`);
+    return;
+  }
+
+  // Skip if the linked pet hasn't been seeded yet (defensive: caller should
+  // ensure pets exist; if the helper is re-ordered later we don't want to
+  // crash here).
+  const [linkedPet] = await db
+    .select({ id: schema.pets.id })
+    .from(schema.pets)
+    .where(eq(schema.pets.id, LINKED_PET_ID))
+    .limit(1);
+  if (!linkedPet) {
+    console.warn(`⚠ GRO-2100: UAT Pup Alpha (${LINKED_PET_ID}) not found — skipping uat-groomer linkage`);
+    return;
+  }
+
+  // The "Bath & Brush" service id is stable across the reset; falls back to
+  // any active service if it has not been seeded yet (e.g. seedKnownUsers
+  // runs in isolation).
+  const BATH_AND_BRUSH_ID = "b0000001-0000-0000-0000-000000000001";
+  const [bathService] = await db
+    .select({ id: schema.services.id })
+    .from(schema.services)
+    .where(eq(schema.services.id, BATH_AND_BRUSH_ID))
+    .limit(1);
+
+  let serviceId: string;
+  if (bathService) {
+    serviceId = bathService.id;
+  } else {
+    const [fallback] = await db
+      .select({ id: schema.services.id })
+      .from(schema.services)
+      .where(eq(schema.services.active, true))
+      .limit(1);
+    if (!fallback) {
+      console.warn(`⚠ GRO-2100: no active services found — skipping uat-groomer linkage`);
+      return;
+    }
+    serviceId = fallback.id;
+  }
+
+  // Schedule the completed appointment 7 days ago so the profile-summary's
+  // "recentGroomingHistory" window (last 10) reliably includes it.
+  const startTime = new Date();
+  startTime.setDate(startTime.getDate() - 7);
+  startTime.setHours(10, 0, 0, 0);
+  const endTime = new Date(startTime.getTime() + 45 * 60 * 1000);
+
+  await db.insert(schema.appointments).values({
+    id: APPT_ID,
+    clientId: customerClientId,
+    petId: LINKED_PET_ID,
+    serviceId,
+    staffId: uatGroomerStaff.id,
+    batherStaffId: null,
+    status: "completed",
+    startTime,
+    endTime,
+    notes: "GRO-2100: deterministic uat-groomer linkage for TC-UAT-2/3.",
+    priceCents: null,
+    confirmationStatus: "confirmed",
+  });
+  console.log(
+    `✓ GRO-2100: linked uat-groomer (${uatGroomerStaff.id}) → UAT Pup Alpha (${LINKED_PET_ID}) via appointment ${APPT_ID}`,
+  );
 }
 
 // ── Known-users-only seed (prod/demo) ───────────────────────────────────────
@@ -745,27 +873,40 @@ async function seedKnownUsers() {
   // ── UAT staff accounts + Better Auth credentials (shared impl) ──────────────
   // Extracted into seedUatStaffAccounts() so it runs in both seedKnownUsers()
   // and the full seed() UAT branch.
-  await seedUatStaffAccounts(db);
+  const uatCustomerClientId = await seedUatStaffAccounts(db);
 
-  // ── Services: idempotent upsert using name as unique key ─────────────────────
-  // UNIQUE constraint on services.name (migration 0020) must exist first.
-  // Uses b0000001-... IDs to match main seed servicesDef for same-named services.
+  // ── Services: idempotent upsert keyed on `id` ─────────────────────────────
+  // GRO-2064: previously keyed on `services.name` while writing a
+  // deterministic `id`. If a stale row existed with the same `id` but a
+  // different `name`, PostgreSQL raised `services_pkey` (id collision)
+  // before the name-targeted ON CONFLICT could fire. Switch the conflict
+  // target to `services.id` so deterministic ids always win; pair with
+  // `TRUNCATE services … CASCADE` above so each reset rebuilds the
+  // catalogue from `servicesDef` cleanly. GRO-2033 close-out.
+  // Id↔name map MUST stay in sync with `servicesDef` (the canonical source
+  // of truth in the main `seed()` function).
   const demoSvcs = [
     { id: "b0000001-0000-0000-0000-000000000001", name: "Bath & Brush", description: "Full bath, blow-dry, brush out, and ear cleaning", basePriceCents: 4500, durationMinutes: 45 },
     { id: "b0000001-0000-0000-0000-000000000002", name: "Full Groom — Small", description: "Complete grooming for dogs under 25 lbs", basePriceCents: 6500, durationMinutes: 60 },
     { id: "b0000001-0000-0000-0000-000000000003", name: "Full Groom — Medium", description: "Complete grooming for dogs 25-50 lbs", basePriceCents: 8000, durationMinutes: 75 },
-    { id: "b0000001-0000-0000-0000-000000000004", name: "Nail Trim", description: "Nail clipping and filing", basePriceCents: 1500, durationMinutes: 15 },
+    { id: "b0000001-0000-0000-0000-000000000005", name: "Nail Trim", description: "Nail clipping and filing", basePriceCents: 1500, durationMinutes: 15 },
   ];
   for (const svc of demoSvcs) {
     await db.insert(schema.services)
       .values({ ...svc, active: true })
       .onConflictDoUpdate({
-        target: schema.services.name,
-        set: { description: svc.description, basePriceCents: svc.basePriceCents, durationMinutes: svc.durationMinutes, active: true },
+        target: schema.services.id,
+        set: { name: svc.name, description: svc.description, basePriceCents: svc.basePriceCents, durationMinutes: svc.durationMinutes, active: true },
       });
   }
   console.log(`✓ Seeded ${demoSvcs.length} services`);
 
+  // GRO-2100: deterministic uat-groomer ↔ UAT Pup Alpha linkage. Must run
+  // AFTER services are seeded (this helper looks up an active service id
+  // to attach to the appointment; on a fresh reset there are none yet at
+  // the time seedUatStaffAccounts() returns).
+  await seedUatGroomerLinkage(db, uatCustomerClientId);
+
   // ── Client: Demo Client ──
   const [existingClient] = await db
     .select()
@@ -835,6 +976,63 @@ async function seedKnownUsers() {
 
 // ── Main seed ────────────────────────────────────────────────────────────────
 
+// ── GRO-2123: serialize reset+seed with a Postgres advisory lock ────────
+// The reset-demo-data CronJob runs on an hourly schedule. With
+// concurrencyPolicy=Replace, a new pod can start while the previous one
+// is still mid-seed; the new pod's TRUNCATE then deletes rows the old pod
+// is still inserting, producing FK 23503 errors non-deterministically
+// (see GRO-2123: invoice_tip_splits → invoices).
+//
+// We hold a session-level advisory lock for the full duration of the
+// seed so that overlapping invocations block then proceed in order —
+// not skip. The key is a stable 32-bit constant so it can be referenced
+// from runbooks without ambiguity and binds to the single-argument
+// `pg_advisory_lock(int)` form, which postgres-js serializes as a plain
+// number (no bigint type plumbing required).
+const SEED_ADVISORY_LOCK_KEY = 0x47524f4f; // "GROO" in ASCII — arbitrary, stable
+
+/**
+ * Reserve a dedicated connection from `pool`, take the seed advisory lock
+ * on it, run `fn`, and release the lock + connection in a try/finally.
+ *
+ * CRITICAL: with postgres-js connection pooling, a session-level
+ * `pg_advisory_lock(KEY)` acquired on one pooled connection and released
+ * on a *different* one is a no-op (the lock is bound to the session /
+ * pg-backend that took it). We therefore reserve a dedicated connection
+ * for the lock and release it from the same reserved connection. The
+ * seed work itself still runs on the pooled connections.
+ */
+async function withSeedAdvisoryLock<T>(
+  pool: ReturnType<typeof postgres>,
+  fn: () => Promise<T>,
+): Promise<T> {
+  const lockConnection = await pool.reserve();
+  let lockHeld = false;
+  try {
+    await lockConnection`SELECT pg_advisory_lock(${SEED_ADVISORY_LOCK_KEY})`;
+    lockHeld = true;
+    console.log(`✓ Acquired seed advisory lock (key=${SEED_ADVISORY_LOCK_KEY})`);
+    const result = await fn();
+    await lockConnection`SELECT pg_advisory_unlock(${SEED_ADVISORY_LOCK_KEY})`;
+    lockHeld = false;
+    console.log(`✓ Released seed advisory lock`);
+    return result;
+  } finally {
+    if (lockHeld) {
+      try {
+        await lockConnection`SELECT pg_advisory_unlock(${SEED_ADVISORY_LOCK_KEY})`;
+      } catch (err) {
+        console.error("Failed to release seed advisory lock during cleanup:", err);
+      }
+    }
+    try {
+      lockConnection.release();
+    } catch (err) {
+      console.error("Failed to release reserved lock connection:", err);
+    }
+  }
+}
+
 async function seed() {
   const url = process.env.DATABASE_URL;
   if (!url) {
@@ -852,6 +1050,22 @@ async function seed() {
   const client = postgres(url, { max: 5 });
   const db = drizzle(client, { schema });
 
+  // GRO-2123: hold the seed advisory lock for the full body of runSeedBody.
+  // See the withSeedAdvisoryLock comment for why a reserved connection is
+  // required (postgres-js pooling would silently drop the lock otherwise).
+  await withSeedAdvisoryLock(client, async () => {
+    return await runSeedBody(client, db, profile, cfg);
+  });
+
+  await client.end();
+}
+
+async function runSeedBody(
+  client: ReturnType<typeof postgres>,
+  db: ReturnType<typeof drizzle>,
+  profile: SeedProfile,
+  cfg: ProfileConfig,
+): Promise<void> {
   console.log(`Seeding Groom Book database (profile: ${profile})...\n`);
 
   // ── Staff ──
@@ -868,7 +1082,13 @@ async function seed() {
     ({ id: uuid(), name: `Bather ${i + 1}`, email: `bather${i + 1}@groombook.dev`, role: "groomer" as const, isSuperUser: false })
   );
 
-  await db.execute(sql`TRUNCATE impersonation_sessions, impersonation_audit_logs, appointments, invoices, invoice_line_items, invoice_tip_splits, grooming_visit_logs CASCADE`);
+  // GRO-2064: also TRUNCATE `services` so each reset rebuilds the catalogue
+  // from `servicesDef` (deterministic IDs + UNIQUE(name)). Stale service rows
+  // (e.g. a prior `seedKnownUsers` run that wrote a different `name` for the
+  // same `id`) would otherwise cause the deterministic upsert to PK-collide
+  // on `services.id` — see CTO review on infra PR #605 (rev #4230). TRUNCATE
+  // CASCADE handles appointments/invoices FKs to services.id.
+  await db.execute(sql`TRUNCATE services, impersonation_sessions, impersonation_audit_logs, appointments, invoices, invoice_line_items, invoice_tip_splits, grooming_visit_logs CASCADE`);
 
   const allStaff = [...managerStaff, ...receptionistStaff, ...groomers, ...bathers];
   for (const s of allStaff) {
@@ -916,12 +1136,14 @@ async function seed() {
   // ── UAT staff accounts + Better Auth credentials (shared impl) ──────────────
   // Seeds deterministic UAT staff with numeric OIDC subs and Better Auth credentials.
   // Must run AFTER random staff are created so upserts land correctly.
-  await seedUatStaffAccounts(db);
+  const uatCustomerClientId = await seedUatStaffAccounts(db);
 
   // ── Services ──
-  // Upsert services using name as unique key. With deterministic IDs in
-  // servicesDef and TRUNCATE clearing downstream tables first, this is
-  // idempotent: first run inserts, subsequent runs update existing rows.
+  // GRO-2064: key the upsert on `services.id` (not `name`) so deterministic
+  // ids always win, and rely on the TRUNCATE above to clear stale rows before
+  // the catalogue is rebuilt. The previous name-targeted upsert failed with
+  // `services_pkey` when a prior run had left a row with the same id but a
+  // different name (CTO review on infra PR #605, rev #4230).
   const serviceIds: string[] = [];
   for (const s of servicesDef) {
     serviceIds.push(s.id);
@@ -935,12 +1157,18 @@ async function seed() {
         active: true,
       })
       .onConflictDoUpdate({
-        target: schema.services.name,
-        set: { description: s.desc, basePriceCents: s.price, durationMinutes: s.dur, active: true },
+        target: schema.services.id,
+        set: { name: s.name, description: s.desc, basePriceCents: s.price, durationMinutes: s.dur, active: true },
       });
   }
   console.log(`✓ Created ${servicesDef.length} services`);
 
+  // GRO-2100: deterministic uat-groomer ↔ UAT Pup Alpha linkage. Must run
+  // AFTER services are seeded (this helper looks up an active service id
+  // to attach to the appointment; on a fresh reset there are none yet at
+  // the time seedUatStaffAccounts() returns).
+  await seedUatGroomerLinkage(db, uatCustomerClientId);
+
   // ── Clients & Pets ──
   const now = new Date();
   const appointmentsBackDate = new Date(now);
@@ -1459,8 +1687,6 @@ async function seed() {
   }
   console.log(`✓ Created ${visitLogCount} grooming visit logs`);
   console.log("\nSeed complete!");
-
-  await client.end();
 }
 
 seed().catch((err) => {