fix(GRO-2652): promote boot ECONNRESET resilience to UAT (dev→uat)
CI / Lint & Typecheck (push) Successful in 19s
CI / Test (push) Successful in 20s
CI / Build & Push Docker Images (push) Successful in 28s
CI / Lint & Typecheck (pull_request) Successful in 23s
CI / Test (pull_request) Successful in 23s
CI / Build & Push Docker Images (pull_request) Successful in 22s
CI / Lint & Typecheck (push) Successful in 19s
CI / Test (push) Successful in 20s
CI / Build & Push Docker Images (push) Successful in 28s
CI / Lint & Typecheck (pull_request) Successful in 23s
CI / Test (pull_request) Successful in 23s
CI / Build & Push Docker Images (pull_request) Successful in 22s
Merges boot ECONNRESET resilience fix + CORS enforcement + OOBE endpoint to uat. QA approved (Lint Roller) at #223 (comment) Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit was merged in pull request #223.
This commit is contained in:
@@ -102,7 +102,7 @@ jobs:
|
||||
git.farh.net/groombook/api:${{ steps.version.outputs.tag }}
|
||||
${{ github.ref == 'refs/heads/main' && 'git.farh.net/groombook/api:latest' || '' }}
|
||||
cache-from: type=registry,ref=git.farh.net/groombook/cache:api
|
||||
cache-to: type=registry,ref=git.farh.net/groombook/cache:api,mode=max
|
||||
cache-to: type=registry,ref=git.farh.net/groombook/cache:api,mode=max,ignore-error=true
|
||||
|
||||
- name: Build and push Migrate image
|
||||
uses: docker/build-push-action@v6
|
||||
@@ -116,7 +116,7 @@ jobs:
|
||||
git.farh.net/groombook/migrate:${{ steps.version.outputs.tag }}
|
||||
${{ github.ref == 'refs/heads/main' && 'git.farh.net/groombook/migrate:latest' || '' }}
|
||||
cache-from: type=registry,ref=git.farh.net/groombook/cache:migrate
|
||||
cache-to: type=registry,ref=git.farh.net/groombook/cache:migrate,mode=max
|
||||
cache-to: type=registry,ref=git.farh.net/groombook/cache:migrate,mode=max,ignore-error=true
|
||||
|
||||
- name: Smoke test migrate image (blackhole npmjs.org)
|
||||
run: |
|
||||
@@ -141,7 +141,7 @@ jobs:
|
||||
git.farh.net/groombook/seed:${{ steps.version.outputs.tag }}
|
||||
${{ github.ref == 'refs/heads/main' && 'git.farh.net/groombook/seed:latest' || '' }}
|
||||
cache-from: type=registry,ref=git.farh.net/groombook/cache:seed
|
||||
cache-to: type=registry,ref=git.farh.net/groombook/cache:seed,mode=max
|
||||
cache-to: type=registry,ref=git.farh.net/groombook/cache:seed,mode=max,ignore-error=true
|
||||
|
||||
- name: Build and push Reset image
|
||||
uses: docker/build-push-action@v6
|
||||
@@ -155,7 +155,7 @@ jobs:
|
||||
git.farh.net/groombook/reset:${{ steps.version.outputs.tag }}
|
||||
${{ github.ref == 'refs/heads/main' && 'git.farh.net/groombook/reset:latest' || '' }}
|
||||
cache-from: type=registry,ref=git.farh.net/groombook/cache:reset
|
||||
cache-to: type=registry,ref=git.farh.net/groombook/cache:reset,mode=max
|
||||
cache-to: type=registry,ref=git.farh.net/groombook/cache:reset,mode=max,ignore-error=true
|
||||
|
||||
- name: Smoke test seed image (blackhole npmjs.org)
|
||||
run: |
|
||||
@@ -185,3 +185,4 @@ jobs:
|
||||
"$IMAGE" \
|
||||
sh -c 'set -e; test "$(which pnpm)" = "/usr/local/bin/pnpm"; echo "HOME=$HOME"; pnpm --version'
|
||||
echo "reset image: pnpm resolves to /usr/local/bin/pnpm, HOME=/tmp, runs offline ✓"
|
||||
|
||||
|
||||
@@ -439,6 +439,38 @@ Both use the stops' stored `latitude`/`longitude` in `stopOrder`: **origin = fir
|
||||
| TC-API-18.10 | Groomer cannot export another's route | As groomer, export a route owned by a different groomer | 403 Forbidden (`groomers may only access their own route`) |
|
||||
| TC-API-18.11 | Receptionist denied | As **receptionist**, export any route | 403 Forbidden (role not permitted) |
|
||||
|
||||
|
||||
### 4.19 Boot Resilience — ECONNRESET Recovery (GRO-2652)
|
||||
|
||||
Verifies the API process does not crash on transient boot-time DB connection resets and that auth routes degrade gracefully until initialization succeeds.
|
||||
|
||||
| TC | Test Case | Steps | Expected Result |
|
||||
|----|-----------|-------|-----------------|
|
||||
| TC-API-19.1 | Health endpoint available before auth init | 1. Deploy the image (or restart the api pod)<br>2. `GET /health` immediately (within first 2 s of pod start) | 200 `{"status":"ok"}` — server accepts requests before `initAuth()` completes |
|
||||
| TC-API-19.2 | Auth routes return 503 when auth not yet initialized | 1. Temporarily set `OIDC_ISSUER` to an unreachable host so `initAuth()` keeps retrying<br>2. `POST /api/auth/sign-in/email` during the retry window | 503 `{"error":"Authentication not configured"}` — process stays alive, does not exit |
|
||||
| TC-API-19.3 | Pod does not crash on first-attempt DB reset | 1. Review pod restart count after normal deployment<br>2. Confirm `kubectl get pod -n groombook` shows `RESTARTS: 0` (or same as before deploy) for the new pod | No new restarts — ECONNRESET causes retry, not process exit |
|
||||
| TC-API-19.4 | DB query retry log lines visible | After deploy, `kubectl logs -n groombook <api-pod>` | If any DB retry occurred, log lines matching `[auth] DB query attempt N failed` are present; on clean boot no retry lines appear |
|
||||
| TC-API-19.5 | Auth init retry log lines visible | When auth init fails and retries, check pod logs | Log lines matching `[auth] initAuth attempt N failed` present; process continues; no `process.exit` |
|
||||
| TC-API-19.6 | Auth succeeds after transient DB hiccup | 1. Allow pod to retry until DB is available<br>2. `POST /api/auth/sign-in/email` with valid credentials after init succeeds | 200 with session cookie — auth recovers without pod restart |
|
||||
| TC-API-19.7 | Normal sign-in still works end-to-end | Follow TC-WEB-SSO-3 (SSO sign-in) on UAT | Successful sign-in, staff list visible — no regression from resilience changes |
|
||||
| TC-API-19.8 | Public routes unaffected during auth retry | While auth is retrying (TC-API-19.2 setup), `GET /api/branding` | 200 with branding data — public routes bypass auth and serve normally |
|
||||
|
||||
### 4.20 Portal OOBE — Create Client from Auth (GRO-2359)
|
||||
|
||||
Verifies the `POST /api/portal/clients-from-auth` endpoint that creates a new `clients` row for a first-time SSO user (out-of-box-experience registration). This endpoint requires a valid Better Auth session but does NOT require a portal session; it is the pre-portal step in the new-user OOBE flow.
|
||||
|
||||
| TC | Test Case | Steps | Expected Result |
|
||||
|----|-----------|-------|-----------------|
|
||||
| TC-API-20.1 | Successful client creation | 1. Sign in via SSO to obtain a Better Auth session<br>2. `POST /api/portal/clients-from-auth` with `{ "name": "Test User" }` | 201 `{ "id": "<uuid>", "name": "Test User", "email": "<sso-email>" }` — new `clients` row created |
|
||||
| TC-API-20.2 | All optional fields accepted | `POST /api/portal/clients-from-auth` with `{ "name": "Test User", "phone": "555-1234", "address": "1 Main St", "notes": "VIP" }` (authenticated) | 201 with `id`, `name`, `email`; row in DB has all four fields |
|
||||
| TC-API-20.3 | Invalid body — missing name | `POST /api/portal/clients-from-auth` with `{}` (authenticated) | 400 (Zod validation failure); no row created |
|
||||
| TC-API-20.4 | Invalid body — empty name | `POST /api/portal/clients-from-auth` with `{ "name": "" }` (authenticated) | 400 — name must be at least 1 character |
|
||||
| TC-API-20.5 | No session — 401 | `POST /api/portal/clients-from-auth` with a valid body but **no** Better Auth session cookie | 401 `{ "error": "Unauthorized" }` |
|
||||
| TC-API-20.6 | Existing email — 409 | 1. Create a client row whose email matches the signed-in SSO user's email<br>2. `POST /api/portal/clients-from-auth` as that user | 409 `{ "error": "A customer record with this email already exists" }` — no duplicate row |
|
||||
| TC-API-20.7 | Auth not configured — 503 | Temporarily disable auth (e.g., point `OIDC_ISSUER` to an invalid host) so `getAuth()` throws<br>2. `POST /api/portal/clients-from-auth` | 503 `{ "error": "Authentication not configured" }` — graceful degradation |
|
||||
| TC-API-20.8 | Concurrent insert race — 409 | Simulate two near-simultaneous requests from the same SSO user (before any row exists) | At most one request returns 201; the other returns 409 — no duplicate row, no 500 |
|
||||
|
||||
|
||||
## Pass/Fail Criteria
|
||||
|
||||
**Pass:**
|
||||
@@ -460,3 +492,4 @@ Both use the stops' stored `latitude`/`longitude` in `stopOrder`: **origin = fir
|
||||
## Update Policy
|
||||
|
||||
Any PR that changes user-facing behaviour MUST update this file. Test cases must be added, modified, or removed to reflect the new behaviour. The PR description must reference which playbook section was updated (e.g., "Updated UAT_PLAYBOOK.md §4.4 — new appointment rescheduling flow").
|
||||
|
||||
|
||||
@@ -69,9 +69,14 @@ describe("auth init", () => {
|
||||
beforeEach(() => {
|
||||
dbSelectResult = [];
|
||||
vi.clearAllMocks();
|
||||
// Stub fetch so OIDC discovery requests resolve instantly during tests.
|
||||
// Without this, AbortSignal.timeout(5000) in auth.ts races with vitest's
|
||||
// 5000ms default test timeout and causes flaky failures.
|
||||
vi.stubGlobal("fetch", vi.fn().mockResolvedValue({ ok: false, status: 503 }));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals();
|
||||
process.env = { ...originalEnv };
|
||||
});
|
||||
|
||||
|
||||
@@ -69,9 +69,14 @@ describe("auth init", () => {
|
||||
beforeEach(() => {
|
||||
dbSelectResult = [];
|
||||
vi.clearAllMocks();
|
||||
// Stub fetch so OIDC discovery requests resolve instantly during tests.
|
||||
// Without this, AbortSignal.timeout(5000) in auth.ts races with vitest's
|
||||
// 5000ms default test timeout and causes flaky failures.
|
||||
vi.stubGlobal("fetch", vi.fn().mockResolvedValue({ ok: false, status: 503 }));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals();
|
||||
process.env = { ...originalEnv };
|
||||
});
|
||||
|
||||
|
||||
+22
-2
@@ -292,14 +292,34 @@ api.route("/search", searchRouter);
|
||||
api.route("/buffer-rules", bufferRulesRouter);
|
||||
api.route("/routes", routesRouter);
|
||||
|
||||
// Start the HTTP server first so /health and public routes are available immediately.
|
||||
// Auth initialization runs afterward with retry — a transient DB ECONNRESET at boot
|
||||
// must not crash the process (GRO-2652). Auth routes return 503 until initAuth succeeds.
|
||||
const port = Number(process.env.PORT ?? 3000);
|
||||
await initAuth();
|
||||
console.log(`API server listening on port ${port}`);
|
||||
const server = serve({ fetch: app.fetch, port });
|
||||
console.log(`API server listening on port ${port}`);
|
||||
|
||||
// Start background reminder scheduler (runs every minute to check for upcoming appointments)
|
||||
startReminderScheduler();
|
||||
|
||||
let initAttempt = 0;
|
||||
while (true) {
|
||||
try {
|
||||
await initAuth();
|
||||
break;
|
||||
} catch (err) {
|
||||
initAttempt++;
|
||||
const delay = Math.min(2 ** initAttempt * 500, 30_000);
|
||||
console.error(`[auth] initAuth attempt ${initAttempt} failed: ${err}`);
|
||||
if (initAttempt >= 10) {
|
||||
console.error("[auth] auth init permanently failed — auth endpoints will serve 503");
|
||||
break;
|
||||
}
|
||||
console.error(`[auth] retrying in ${delay}ms`);
|
||||
await new Promise((r) => setTimeout(r, delay));
|
||||
}
|
||||
}
|
||||
|
||||
function shutdown() {
|
||||
console.log("Shutting down gracefully...");
|
||||
// SIGTERM/SIGINT → server.close() → callback → process.exit(0)
|
||||
|
||||
+22
-2
@@ -124,13 +124,28 @@ export async function initAuth(): Promise<void> {
|
||||
return;
|
||||
}
|
||||
|
||||
// Step 1: Try to load config from DB
|
||||
// Step 1: Try to load config from DB, with retry-with-backoff for transient ECONNRESET (GRO-2652).
|
||||
// A single connection reset during boot must not abort initialization.
|
||||
const db = getDb();
|
||||
const [dbConfig] = await db
|
||||
let dbQueryRows: (typeof authProviderConfig.$inferSelect)[] = [];
|
||||
let dbAttempt = 0;
|
||||
while (true) {
|
||||
try {
|
||||
dbQueryRows = await db
|
||||
.select()
|
||||
.from(authProviderConfig)
|
||||
.where(eq(authProviderConfig.enabled, true))
|
||||
.limit(1);
|
||||
break;
|
||||
} catch (err) {
|
||||
dbAttempt++;
|
||||
if (dbAttempt >= 5) throw err;
|
||||
const delay = Math.min(1000 * 2 ** (dbAttempt - 1), 8_000);
|
||||
console.warn(`[auth] DB query attempt ${dbAttempt} failed (${err}), retrying in ${delay}ms`);
|
||||
await new Promise((r) => setTimeout(r, delay));
|
||||
}
|
||||
}
|
||||
const [dbConfig] = dbQueryRows;
|
||||
|
||||
let providerConfig: {
|
||||
providerId: string;
|
||||
@@ -314,5 +329,10 @@ export async function initAuth(): Promise<void> {
|
||||
});
|
||||
})();
|
||||
|
||||
try {
|
||||
await authInitPromise;
|
||||
} catch (err) {
|
||||
authInitPromise = null; // allow retry on next call
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user