Compare commits

..

2 Commits

Author SHA1 Message Date
Chris Farhood 629261625c Remove antipattern content from UAT skill
Strip "What UAT Is NOT" section per board feedback on PRI-1487.
Focus skill on positive guidance only.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 05:09:28 +00:00
Chris Farhood 3abfe53a1e Refactor UAT skill: delegate to UAT_PLAYBOOK.md per plugin
- Rewrite skills/uat/SKILL.md to be a general UAT overview that
  delegates per-plugin test procedures to UAT_PLAYBOOK.md in each
  plugin repository (authoritative source of truth)
- Add "What UAT Is NOT" section to explicitly prevent GitHub-browsing
  anti-pattern
- Update SDLC skill to require UAT_PLAYBOOK.md as a required deliverable
  alongside code changes affecting plugin testing
- Create UAT_PLAYBOOK.md in sealed-secrets and rook plugin repos

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 05:07:28 +00:00
8 changed files with 62 additions and 140 deletions
-56
View File
@@ -1,56 +0,0 @@
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install linters
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends shellcheck yamllint
- name: Lint Markdown
uses: DavidAnson/markdownlint-cli2-action@v19
with:
globs: "**/*.md"
- name: Lint YAML
run: yamllint .
- name: Shellcheck
run: shellcheck scripts/*.sh
- name: Validate skill frontmatter
run: |
set -e
fail=0
for f in skills/*/SKILL.md; do
fm=$(awk 'BEGIN{c=0} /^---$/{c++; next} c==1{print} c>=2{exit}' "$f")
for key in name description; do
if ! printf '%s\n' "$fm" | grep -qE "^${key}:[[:space:]]"; then
echo "::error file=${f}::missing '${key}' in YAML frontmatter"
fail=1
fi
done
done
exit $fail
ci:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Validate JSON files
run: |
find . -name "*.json" -not -path "./.git/*" | while read -r f; do
python3 -m json.tool "$f" > /dev/null || { echo "::error file=$f::Invalid JSON"; exit 1; }
done
echo "All JSON files valid"
-24
View File
@@ -1,24 +0,0 @@
name: Promotion Gate
on:
pull_request:
branches: [main]
jobs:
promotion_gate:
name: Promotion Gate
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Validate skills directory structure
run: |
set -e
fail=0
for dir in skills/*/; do
if [ ! -f "${dir}SKILL.md" ]; then
echo "::error::Missing SKILL.md in ${dir}"
fail=1
fi
done
exit $fail
+17
View File
@@ -0,0 +1,17 @@
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Lint Markdown
uses: DavidAnson/markdownlint-cli2-action@v19
with:
globs: "**/*.md"
-7
View File
@@ -1,7 +0,0 @@
extends: default
rules:
line-length: disable
document-start: disable
truthy:
check-keys: false
-6
View File
@@ -1,6 +0,0 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": [
"local>privilegedescalation/.github:renovate-config"
]
}
+7 -5
View File
@@ -10,7 +10,9 @@ description: >
## GitHub Authentication
Access to GitHub is done via token in your env **Never** run `gh auth login` directly — it hangs headless agents.
**Invoke the `github-app-token` skill** before any GitHub operation. It generates a short-lived installation token and sets `GH_TOKEN`. **Never** run `gh auth login` directly — it hangs headless agents.
Token expires after ~1 hour. Re-invoke the skill to regenerate if needed.
## GitHub Issues — Board Approval Required
@@ -67,13 +69,15 @@ Each promotion is a PR reviewed and merged by its gate owner:
2. **dev → uat** — QA (Regression Regina) reviews code quality: test coverage, regressions, edge cases. QA merges to `uat` after approval.
3. **uat → main** — UAT (Pixel Patty) validates the deployed application via Playwright browser testing. UAT merges to `main` after validation passes. For detailed UAT testing procedures, see the `uat` company skill.
**Playbook maintenance:** When plugin testing procedures change (new features, changed UI, different access paths), the engineer must update the plugin's `UAT_PLAYBOOK.md` in the same PR. This is a required deliverable alongside the code change — not optional cleanup.
**Each gate owner has merge authority.** No separate merge step by another role. No agent merges their own code to `uat` or `main` — only the gate owner merges promotions they review.
## Pipeline
### Pipeline A: Plugin/Feature Changes
```text
```
Engineer → PR to dev → self-merge → deploys to dev
→ Engineer validates on dev
→ PR from dev → uat → QA reviews → QA merges
@@ -84,11 +88,9 @@ Engineer → PR to dev → self-merge → deploys to dev
Applies to changes in `headlamp-*-plugin/` repos (plugin code, features, bug fixes).
**UAT_PLAYBOOK.md maintenance:** When modifying a plugin in any way that changes how it must be tested — including new features, changed behavior, updated UI flows, or different data sources — the engineer must update the `UAT_PLAYBOOK.md` file in the plugin repository root with the current testing steps before requesting UAT. This ensures the playbook stays current as plugins evolve and UAT agents have accurate test guidance.
### Pipeline B: Infrastructure Changes (No UI Impact)
```text
```
Engineer → PR to main → CI passes → QA reviews → QA merges
→ Production
```
+1
View File
@@ -132,3 +132,4 @@ flowchart LR
class UATB uat
class MAIN prod
```
+37 -42
View File
@@ -1,20 +1,40 @@
---
name: uat
description: >
Functional UAT procedures for Privileged Escalation Headlamp plugins. General
behavior, acceptance criteria, artifact requirements, and reference to
plugin-specific test steps in UAT_PLAYBOOK.md.
Functional UAT procedures for Privileged Escalation Headlamp plugins. Delegates per-plugin testing procedures to UAT_PLAYBOOK.md in each plugin repository.
---
# UAT Procedures
## Purpose
This skill defines **functional User Acceptance Testing** for all Privileged Escalation Headlamp plugins. UAT validates that plugins work correctly in the deployed environment — by exercising plugin features in a running Headlamp instance, not by reviewing code or CI results.
This skill defines **functional User Acceptance Testing** for all Privileged Escalation Headlamp plugins. UAT validates that plugins work correctly in the deployed environment — by loading plugins in a running Headlamp instance and exercising their features.
## UAT Environment
## Pipeline Stage
The UAT Headlamp instance runs in the `headlamp-uat` Kubernetes namespace. Navigate to the Headlamp UAT URL using your Playwright browser. The plugin under test must be deployed to UAT before testing begins.
UAT is the final promotion gate before production: `uat → main`. Pixel Patty (UAT agent) validates the deployed application via Playwright browser testing before merging.
## Per-Plugin Testing Procedures
Detailed, up-to-date testing procedures for each plugin are maintained in the plugin's own repository at `UAT_PLAYBOOK.md`. This is the authoritative source — do not copy procedure details into this skill.
| Plugin | Repository |
|--------|------------|
| headlamp-polaris-plugin | `polaris` |
| headlamp-sealed-secrets-plugin | `sealed-secrets` |
| headlamp-intel-gpu-plugin | `intel-gpu` |
| headlamp-kube-vip-plugin | `kube-vip` |
| headlamp-tns-csi-plugin | `tns-csi` |
| headlamp-rook-plugin | `rook` |
| headlamp-argocd-plugin | `argocd` |
Each plugin's `UAT_PLAYBOOK.md` contains:
- Access path (sidebar entry or app bar location)
- Step-by-step test actions
- Expected results for each step
- Pass/fail criteria
- Artifact requirements (screenshots, console errors, environment info)
## General Process
@@ -22,48 +42,23 @@ For every `uat→main` promotion:
1. Open the Headlamp UAT instance in the browser
2. Confirm the plugin appears in the sidebar or app bar
3. Read the plugin's `UAT_PLAYBOOK.md` for the specific test steps to run
4. Execute the test steps from the playbook, capturing screenshots at each verification
5. Check the browser console for errors throughout
6. Post a structured test report (see Artifacts section)
3. Retrieve the plugin's `UAT_PLAYBOOK.md` from its repository
4. Execute the test steps in the playbook
5. Capture screenshots of the **running plugin** at each verification step
6. Check the browser console for errors
7. Post a structured test report (see Artifacts section)
## Acceptance Criteria
A plugin passes UAT when:
- **Plugin loads** — sidebar entry or app bar action is visible and accessible
- **Features work** — all core features in the playbook execute without errors
- **No console errors** — browser console shows no errors during normal operation
- **Data matches cluster state** — plugin data is consistent with `kubectl` queries against the cluster
A plugin fails UAT when:
- Plugin does not load or renders only an error state
- Any core feature is inaccessible or produces errors
- Console errors are present and not explainable as unrelated noise
- Displayed data contradicts known cluster state
## Artifact Requirements
## UAT Artifacts
For each plugin tested, the UAT report must include:
1. **Screenshots** of the plugin running in Headlamp — sidebar entry visible, main view loaded, at least one detail view
2. **Test checklist** — each step from `UAT_PLAYBOOK.md` marked pass/fail
2. **Test checklist** — each step from the playbook marked pass/fail
3. **Console errors** — any browser console errors observed (attach screenshot if present)
4. **Environment info** — Headlamp version, plugin version, browser used, namespace context
## Reading UAT_PLAYBOOK.md
Each plugin repository contains a `UAT_PLAYBOOK.md` in its root directory. That file contains the canonical test steps for that specific plugin. Before running UAT, read the relevant playbook to know:
- Which features to exercise
- What the expected results are
- What screenshots to capture at each step
If `UAT_PLAYBOOK.md` does not exist for a plugin, treat that as a gap — report it in the UAT findings and flag it as a documentation issue.
4. **Environment** — Headlamp version, plugin version, browser used
## Decision Criteria
- **Approve** the `uat→main` promotion when all applicable test steps from the playbook pass and no console errors are present
- **Request changes** when any test step fails — include specific failing steps, observed results vs. expected results, and failure screenshots
- **Block** if the plugin fails to load entirely — escalate to CTO as a deployment issue requiring immediate resolution
- **Approve** the `uat→main` PR when all applicable test steps pass
- **Request changes** with specific failing steps and failure screenshots
- **Block** if the plugin fails to load entirely — escalate to CTO as a deployment issue