Merge pull request 'docs: add MCP-driven execution method to UAT playbook (GRO-1502)' (#432) from docs/GRO-1502-uat-mcp-migration into dev
CI / Web E2E (Dev) (push) Has been cancelled
CI / Deploy PR to groombook-dev (push) Has been cancelled
CI / Lint & Typecheck (push) Successful in 22s
CI / Build (push) Successful in 21s
CI / Test (push) Successful in 24s
CI / Build & Push Docker Images (push) Successful in 33s
CI / Update Infra Image Tags (push) Failing after 1s

docs: add MCP-driven UAT execution method (GRO-1502)
This commit was merged in pull request #432.
This commit is contained in:
2026-05-22 11:48:03 +00:00
2 changed files with 98 additions and 6 deletions
+50
View File
@@ -0,0 +1,50 @@
# Shedward Scissorhands — UAT Agent Instructions
You are the GroomBook User Acceptance Tester. Your sole job is to execute UAT playbooks against deployed environments and report results.
## Mandatory Tooling
You MUST use the **groombook-playwright MCP server** (`mcp__playwright-groombook__*` tools) for ALL browser interaction. Do not:
- Run scripted Playwright suites (`npx playwright test`, `pnpm test:e2e`, etc.)
- Use manual browser commands or shell-based browser automation
- Open browsers outside the MCP server
Every page navigation, click, form fill, and verification MUST go through MCP tools.
## Available MCP Tools
| Tool | When to use |
|------|-------------|
| `browser_navigate` | Open a URL |
| `browser_snapshot` | Read page state (preferred over screenshot for assertions) |
| `browser_take_screenshot` | Capture visual evidence |
| `browser_click` | Click an element (use ref from snapshot) |
| `browser_fill_form` | Fill form fields |
| `browser_type` | Type text into focused element |
| `browser_press_key` | Press keyboard keys |
| `browser_select_option` | Select dropdown options |
| `browser_hover` | Hover over elements |
| `browser_wait_for` | Wait for elements or navigation |
| `browser_console_messages` | Check for JS errors |
| `browser_network_requests` | Inspect API calls |
| `browser_evaluate` | Run JS in page context |
| `browser_resize` | Test responsive layouts |
| `browser_close` | Close browser session |
## Execution Workflow
1. Read the `UAT_PLAYBOOK.md` in the repo being tested.
2. For each test case, translate the human-readable steps into MCP tool calls.
3. Capture evidence: use `browser_snapshot` for assertions, `browser_take_screenshot` for visual proof.
4. Report pass/fail per test case with evidence.
5. If a test fails, document: severity, steps to reproduce, actual vs expected, and attach screenshots.
## Environments
| Environment | URL | Auth |
|-------------|-----|------|
| Dev | `https://dev.groombook.dev` | Dev login selector (no OIDC) |
| UAT | `https://uat.groombook.dev` | Authentik OIDC at `https://auth.farh.net` |
| Production | `https://demo.groombook.dev` | Authentik OIDC |
| Site | `https://groombook.farh.net` | No auth required |
+48 -6
View File
@@ -4,7 +4,49 @@
GroomBook is an open-source, self-hostable pet grooming business management & CRM platform. The monorepo contains the Hono API (`apps/api`), React PWA web app (`apps/web`), E2E tests (`apps/e2e`), and shared packages (`packages/db`, `packages/types`). Tech stack: Hono + React 19 + Vite + PostgreSQL + Drizzle ORM + Authentik OIDC.
## 2. Environments
## 2. Execution Method
All UAT is executed by **Shedward Scissorhands** via the **groombook-playwright MCP server**. No manual browser checks or scripted Playwright suites are used for UAT.
### MCP Tools
Shedward uses the `mcp__playwright-groombook__*` tool family:
| Tool | Purpose |
|------|---------|
| `browser_navigate` | Navigate to a URL |
| `browser_snapshot` | Capture accessibility snapshot (preferred over screenshot) |
| `browser_take_screenshot` | Capture visual screenshot when needed |
| `browser_click` | Click an element by ref or selector |
| `browser_fill_form` | Fill form fields |
| `browser_type` | Type text into focused element |
| `browser_press_key` | Press keyboard keys (Enter, Tab, etc.) |
| `browser_select_option` | Select dropdown options |
| `browser_hover` | Hover over elements |
| `browser_wait_for` | Wait for elements or conditions |
| `browser_console_messages` | Check console for errors |
| `browser_network_requests` | Inspect network traffic |
| `browser_evaluate` | Run JavaScript in page context |
| `browser_tabs` | Manage browser tabs |
| `browser_close` | Close browser |
### How Test Cases Map to MCP Calls
Each test case in Section 4 describes steps like "Navigate to X" or "Click Y". Shedward translates these to MCP tool calls:
- **"Navigate to [URL]"** → `browser_navigate` with the environment URL
- **"Click [element]"** → `browser_snapshot` to find the element ref, then `browser_click`
- **"Fill in [field]"** → `browser_fill_form` or `browser_click` + `browser_type`
- **"Verify [state]"** → `browser_snapshot` and inspect the accessibility tree
- **"Check for errors"** → `browser_console_messages` + `browser_snapshot`
Shedward reads this playbook, executes each test case via MCP tools, captures evidence (snapshots/screenshots), and reports pass/fail per test case.
### Legacy CI Tests
The scripted Playwright suites in `apps/e2e/` and `apps/web/e2e/` are retained for CI regression testing only. They are **not** the primary UAT mechanism. UAT is exclusively MCP-driven by Shedward.
## 3. Environments
| Environment | URL | Notes |
|-------------|-----|-------|
@@ -14,7 +56,7 @@ GroomBook is an open-source, self-hostable pet grooming business management & CR
**Local Development:** Run `docker compose up --build` at repository root. Web app available at `localhost:8080`, API at `localhost:3000`.
## 3. Pre-conditions
## 4. Pre-conditions
- UAT environment is accessible at `https://uat.groombook.dev`
- Test accounts are seeded with the following personas:
@@ -29,7 +71,7 @@ GroomBook is an open-source, self-hostable pet grooming business management & CR
- Stripe test keys are configured for payment flow testing
- Email/SMS providers (Telnyx, etc.) are configured for notification testing
## 4. Test Cases
## 5. Test Cases
### 4.1 Authentication
@@ -252,7 +294,7 @@ GroomBook is an open-source, self-hostable pet grooming business management & CR
| TC-APP-4.21.10 | Whitespace trimming | 1. Send ` START ` or `\tSTOP\n` | Keywords are trimmed before matching |
| TC-APP-4.21.11 | Non-keyword messages ignored | 1. Send `STOP IT`, `help me`, `hello` | Returns null from `detectKeyword`, no consent event inserted, no reply sent |
| TC-APP-4.21.12 | Consent event audit log | 1. After any keyword, query `messageConsentEvents` table | Record exists with correct `clientId`, `businessId`, `kind`, and `source: "sms_keyword"` |
## 5. Pass/Fail Criteria
## 6. Pass/Fail Criteria
**Pass:** All test cases execute without errors. Expected results match actual results. No regressions are observed. All functionality works as documented.
@@ -265,7 +307,7 @@ GroomBook is an open-source, self-hostable pet grooming business management & CR
**Regressions:** If a previously working feature fails during this UAT run, it is considered a regression and must be addressed before the release can proceed.
## 6. Update Policy
## 7. Update Policy
**Any PR that changes user-facing behaviour MUST update this file.**
@@ -275,4 +317,4 @@ When modifying features that affect:
- Configuration (settings, integrations)
- Data visibility (reports, search, filtering)
The corresponding test case(s) in Section 4 must be updated to reflect the new behaviour. The PR description must reference which playbook section was updated (e.g., "Updated UAT_PLAYBOOK.md §4.5 — new appointment group scheduling feature").
The corresponding test case(s) in Section 5 must be updated to reflect the new behaviour. The PR description must reference which playbook section was updated (e.g., "Updated UAT_PLAYBOOK.md §4.5 — new appointment group scheduling feature").