Compare commits
17 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| f5d570ea60 | |||
| 1b4913c0fd | |||
| 983498765e | |||
| f901d622d1 | |||
| ae024551bb | |||
| 1f18a1d982 | |||
| d62d5da70d | |||
| 4c71fab41b | |||
| 7183381140 | |||
| 611334167b | |||
| a3bab704df | |||
| c48eccd70c | |||
| ea1f585722 | |||
| bedef6ab6a | |||
| 1fe4f900b0 | |||
| 44e528c373 | |||
| f9b3ea1882 |
@@ -0,0 +1,56 @@
|
|||||||
|
name: CI
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [main]
|
||||||
|
pull_request:
|
||||||
|
branches: [main]
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
lint:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Install linters
|
||||||
|
run: |
|
||||||
|
sudo apt-get update
|
||||||
|
sudo apt-get install -y --no-install-recommends shellcheck yamllint
|
||||||
|
|
||||||
|
- name: Lint Markdown
|
||||||
|
uses: DavidAnson/markdownlint-cli2-action@v19
|
||||||
|
with:
|
||||||
|
globs: "**/*.md"
|
||||||
|
|
||||||
|
- name: Lint YAML
|
||||||
|
run: yamllint .
|
||||||
|
|
||||||
|
- name: Shellcheck
|
||||||
|
run: shellcheck scripts/*.sh
|
||||||
|
|
||||||
|
- name: Validate skill frontmatter
|
||||||
|
run: |
|
||||||
|
set -e
|
||||||
|
fail=0
|
||||||
|
for f in skills/*/SKILL.md; do
|
||||||
|
fm=$(awk 'BEGIN{c=0} /^---$/{c++; next} c==1{print} c>=2{exit}' "$f")
|
||||||
|
for key in name description; do
|
||||||
|
if ! printf '%s\n' "$fm" | grep -qE "^${key}:[[:space:]]"; then
|
||||||
|
echo "::error file=${f}::missing '${key}' in YAML frontmatter"
|
||||||
|
fail=1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
done
|
||||||
|
exit $fail
|
||||||
|
|
||||||
|
ci:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Validate JSON files
|
||||||
|
run: |
|
||||||
|
find . -name "*.json" -not -path "./.git/*" | while read -r f; do
|
||||||
|
python3 -m json.tool "$f" > /dev/null || { echo "::error file=$f::Invalid JSON"; exit 1; }
|
||||||
|
done
|
||||||
|
echo "All JSON files valid"
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
name: Promotion Gate
|
||||||
|
|
||||||
|
on:
|
||||||
|
pull_request:
|
||||||
|
branches: [main]
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
promotion_gate:
|
||||||
|
name: Promotion Gate
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Validate skills directory structure
|
||||||
|
run: |
|
||||||
|
set -e
|
||||||
|
fail=0
|
||||||
|
for dir in skills/*/; do
|
||||||
|
if [ ! -f "${dir}SKILL.md" ]; then
|
||||||
|
echo "::error::Missing SKILL.md in ${dir}"
|
||||||
|
fail=1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
exit $fail
|
||||||
@@ -1,17 +0,0 @@
|
|||||||
name: CI
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [main]
|
|
||||||
pull_request:
|
|
||||||
branches: [main]
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
lint:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v4
|
|
||||||
- name: Lint Markdown
|
|
||||||
uses: DavidAnson/markdownlint-cli2-action@v19
|
|
||||||
with:
|
|
||||||
globs: "**/*.md"
|
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
extends: default
|
||||||
|
|
||||||
|
rules:
|
||||||
|
line-length: disable
|
||||||
|
document-start: disable
|
||||||
|
truthy:
|
||||||
|
check-keys: false
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
|
||||||
|
"extends": [
|
||||||
|
"local>privilegedescalation/.github:renovate-config"
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -10,9 +10,7 @@ description: >
|
|||||||
|
|
||||||
## GitHub Authentication
|
## GitHub Authentication
|
||||||
|
|
||||||
**Invoke the `github-app-token` skill** before any GitHub operation. It generates a short-lived installation token and sets `GH_TOKEN`. **Never** run `gh auth login` directly — it hangs headless agents.
|
Access to GitHub is done via token in your env **Never** run `gh auth login` directly — it hangs headless agents.
|
||||||
|
|
||||||
Token expires after ~1 hour. Re-invoke the skill to regenerate if needed.
|
|
||||||
|
|
||||||
## GitHub Issues — Board Approval Required
|
## GitHub Issues — Board Approval Required
|
||||||
|
|
||||||
@@ -75,7 +73,7 @@ Each promotion is a PR reviewed and merged by its gate owner:
|
|||||||
|
|
||||||
### Pipeline A: Plugin/Feature Changes
|
### Pipeline A: Plugin/Feature Changes
|
||||||
|
|
||||||
```
|
```text
|
||||||
Engineer → PR to dev → self-merge → deploys to dev
|
Engineer → PR to dev → self-merge → deploys to dev
|
||||||
→ Engineer validates on dev
|
→ Engineer validates on dev
|
||||||
→ PR from dev → uat → QA reviews → QA merges
|
→ PR from dev → uat → QA reviews → QA merges
|
||||||
@@ -86,9 +84,11 @@ Engineer → PR to dev → self-merge → deploys to dev
|
|||||||
|
|
||||||
Applies to changes in `headlamp-*-plugin/` repos (plugin code, features, bug fixes).
|
Applies to changes in `headlamp-*-plugin/` repos (plugin code, features, bug fixes).
|
||||||
|
|
||||||
|
**UAT_PLAYBOOK.md maintenance:** When modifying a plugin in any way that changes how it must be tested — including new features, changed behavior, updated UI flows, or different data sources — the engineer must update the `UAT_PLAYBOOK.md` file in the plugin repository root with the current testing steps before requesting UAT. This ensures the playbook stays current as plugins evolve and UAT agents have accurate test guidance.
|
||||||
|
|
||||||
### Pipeline B: Infrastructure Changes (No UI Impact)
|
### Pipeline B: Infrastructure Changes (No UI Impact)
|
||||||
|
|
||||||
```
|
```text
|
||||||
Engineer → PR to main → CI passes → QA reviews → QA merges
|
Engineer → PR to main → CI passes → QA reviews → QA merges
|
||||||
→ Production
|
→ Production
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -132,4 +132,3 @@ flowchart LR
|
|||||||
class UATB uat
|
class UATB uat
|
||||||
class MAIN prod
|
class MAIN prod
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
+35
-116
@@ -1,27 +1,20 @@
|
|||||||
---
|
---
|
||||||
name: uat
|
name: uat
|
||||||
description: >
|
description: >
|
||||||
Functional UAT procedures for Privileged Escalation Headlamp plugins. Concrete testing steps, pass/fail criteria, and artifact requirements for each plugin.
|
Functional UAT procedures for Privileged Escalation Headlamp plugins. General
|
||||||
|
behavior, acceptance criteria, artifact requirements, and reference to
|
||||||
|
plugin-specific test steps in UAT_PLAYBOOK.md.
|
||||||
---
|
---
|
||||||
|
|
||||||
# UAT Procedures
|
# UAT Procedures
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
This skill defines **functional User Acceptance Testing** for all Privileged Escalation Headlamp plugins. UAT validates that plugins work correctly in the deployed environment — by **loading plugins in a running Headlamp instance and exercising their features**, not by browsing GitHub or inspecting PR diffs.
|
This skill defines **functional User Acceptance Testing** for all Privileged Escalation Headlamp plugins. UAT validates that plugins work correctly in the deployed environment — by exercising plugin features in a running Headlamp instance, not by reviewing code or CI results.
|
||||||
|
|
||||||
## What UAT Is NOT
|
|
||||||
|
|
||||||
- Browsing GitHub PRs and taking screenshots of code diffs
|
|
||||||
- Checking CI status on GitHub
|
|
||||||
- Reading commit messages or PR descriptions
|
|
||||||
- Approving based on QA's review alone
|
|
||||||
|
|
||||||
If your test evidence is screenshots of GitHub pages, you are not performing UAT.
|
|
||||||
|
|
||||||
## UAT Environment
|
## UAT Environment
|
||||||
|
|
||||||
The UAT Headlamp instance runs in the `headlamp-uat` Kubernetes namespace. Navigate to the Headlamp UAT URL using your Playwright browser. The plugin under test must be deployed to UAT before testing.
|
The UAT Headlamp instance runs in the `headlamp-uat` Kubernetes namespace. Navigate to the Headlamp UAT URL using your Playwright browser. The plugin under test must be deployed to UAT before testing begins.
|
||||||
|
|
||||||
## General Process
|
## General Process
|
||||||
|
|
||||||
@@ -29,122 +22,48 @@ For every `uat→main` promotion:
|
|||||||
|
|
||||||
1. Open the Headlamp UAT instance in the browser
|
1. Open the Headlamp UAT instance in the browser
|
||||||
2. Confirm the plugin appears in the sidebar or app bar
|
2. Confirm the plugin appears in the sidebar or app bar
|
||||||
3. Run the plugin-specific test steps below
|
3. Read the plugin's `UAT_PLAYBOOK.md` for the specific test steps to run
|
||||||
4. Capture screenshots of the **running plugin** at each verification step
|
4. Execute the test steps from the playbook, capturing screenshots at each verification
|
||||||
5. Check the browser console for errors
|
5. Check the browser console for errors throughout
|
||||||
6. Post a structured test report (see Artifacts section)
|
6. Post a structured test report (see Artifacts section)
|
||||||
|
|
||||||
## Plugin Test Procedures
|
## Acceptance Criteria
|
||||||
|
|
||||||
### headlamp-polaris-plugin — Kubernetes Best Practices
|
A plugin passes UAT when:
|
||||||
|
|
||||||
**Access:** Sidebar → Polaris section
|
- **Plugin loads** — sidebar entry or app bar action is visible and accessible
|
||||||
|
- **Features work** — all core features in the playbook execute without errors
|
||||||
|
- **No console errors** — browser console shows no errors during normal operation
|
||||||
|
- **Data matches cluster state** — plugin data is consistent with `kubectl` queries against the cluster
|
||||||
|
|
||||||
| Step | Action | Expected Result |
|
A plugin fails UAT when:
|
||||||
|------|--------|-----------------|
|
|
||||||
| 1 | Navigate to the Polaris dashboard | Cluster score loads with a numeric value |
|
|
||||||
| 2 | Verify workload list populates | Shows deployments/pods with individual scores |
|
|
||||||
| 3 | Click into any workload | Detail view shows pass/fail/warning checks with descriptions |
|
|
||||||
| 4 | Cross-check one score against `kubectl` | Polaris CLI output matches plugin display |
|
|
||||||
|
|
||||||
**Pass:** Dashboard loads, shows real workload data, scores are non-zero, detail views navigate correctly.
|
- Plugin does not load or renders only an error state
|
||||||
**Fail:** Page errors, empty data when workloads exist, scores show 0/NaN, detail navigation broken.
|
- Any core feature is inaccessible or produces errors
|
||||||
|
- Console errors are present and not explainable as unrelated noise
|
||||||
|
- Displayed data contradicts known cluster state
|
||||||
|
|
||||||
### headlamp-sealed-secrets-plugin — Sealed Secrets Management
|
## Artifact Requirements
|
||||||
|
|
||||||
**Access:** Sidebar → Sealed Secrets section
|
|
||||||
|
|
||||||
| Step | Action | Expected Result |
|
|
||||||
|------|--------|-----------------|
|
|
||||||
| 1 | Navigate to the Sealed Secrets list | List view loads (may be empty if no secrets exist) |
|
|
||||||
| 2 | Check the create form | Create sealed secret form is accessible with expected fields |
|
|
||||||
| 3 | View an existing sealed secret | Detail view shows metadata, status, and namespace |
|
|
||||||
| 4 | Verify sealed secret status | Status reflects actual K8s state |
|
|
||||||
|
|
||||||
**Pass:** List loads, create form accessible, detail views work, status accurate.
|
|
||||||
**Fail:** Page errors, CRUD forms don't render, missing UI elements, status mismatch.
|
|
||||||
|
|
||||||
### headlamp-intel-gpu-plugin — Intel GPU Monitoring
|
|
||||||
|
|
||||||
**Access:** Sidebar → Intel GPU section
|
|
||||||
|
|
||||||
| Step | Action | Expected Result |
|
|
||||||
|------|--------|-----------------|
|
|
||||||
| 1 | Navigate to the GPU section | GPU device list or status page loads |
|
|
||||||
| 2 | Check node-level GPU info | Per-node GPU allocation is displayed |
|
|
||||||
| 3 | Verify device status | Device plugin status matches `kubectl describe node` GPU capacity |
|
|
||||||
|
|
||||||
**Pass:** Section loads, shows GPU device information or empty state if no Intel GPUs present, no console errors.
|
|
||||||
**Fail:** Page errors, data loading failures, broken rendering.
|
|
||||||
|
|
||||||
### headlamp-kube-vip-plugin — Load Balancer Management
|
|
||||||
|
|
||||||
**Access:** Sidebar → Kube-VIP section
|
|
||||||
|
|
||||||
| Step | Action | Expected Result |
|
|
||||||
|------|--------|-----------------|
|
|
||||||
| 1 | Navigate to the Kube-VIP section | Load balancer status view loads |
|
|
||||||
| 2 | Check VIP list | VIP addresses and their status are displayed |
|
|
||||||
| 3 | View configuration details | Configuration is accessible and readable |
|
|
||||||
| 4 | Verify against cluster state | VIP data matches `kubectl get svc` LoadBalancer entries |
|
|
||||||
|
|
||||||
**Pass:** Section loads, shows VIP data or empty state, navigation works.
|
|
||||||
**Fail:** Page errors, data not rendering, broken navigation.
|
|
||||||
|
|
||||||
### headlamp-tns-csi-plugin — TrueNAS CSI Monitoring
|
|
||||||
|
|
||||||
**Access:** Sidebar → TrueNAS CSI section
|
|
||||||
|
|
||||||
| Step | Action | Expected Result |
|
|
||||||
|------|--------|-----------------|
|
|
||||||
| 1 | Navigate to the TrueNAS CSI section | Volume list loads |
|
|
||||||
| 2 | Check PV/PVC list | Shows provisioned volumes with capacity and status |
|
|
||||||
| 3 | View volume details | Detail view shows storage class, capacity, access modes |
|
|
||||||
| 4 | Verify against cluster state | Volume data matches `kubectl get pv,pvc` output |
|
|
||||||
|
|
||||||
**Pass:** Section loads, shows CSI volumes/status, detail views work.
|
|
||||||
**Fail:** Page errors, empty state when volumes exist, broken detail views.
|
|
||||||
|
|
||||||
### headlamp-rook-plugin — Rook/Ceph Storage
|
|
||||||
|
|
||||||
**Access:** Sidebar → Rook/Ceph section
|
|
||||||
|
|
||||||
| Step | Action | Expected Result |
|
|
||||||
|------|--------|-----------------|
|
|
||||||
| 1 | Navigate to the Rook/Ceph dashboard | Cluster health overview loads |
|
|
||||||
| 2 | Check storage pools | Pool list shows capacity and utilization |
|
|
||||||
| 3 | Verify OSD status | OSD count and status displayed |
|
|
||||||
| 4 | Check placement groups | PG status summary is available |
|
|
||||||
|
|
||||||
**Pass:** Dashboard loads, shows cluster health, pool data renders correctly.
|
|
||||||
**Fail:** Page errors, cluster data missing, health indicators broken.
|
|
||||||
|
|
||||||
### headlamp-argocd-plugin — Argo CD Application Delivery
|
|
||||||
|
|
||||||
**Access:** Sidebar → Argo CD section
|
|
||||||
|
|
||||||
| Step | Action | Expected Result |
|
|
||||||
|------|--------|-----------------|
|
|
||||||
| 1 | Navigate to the Argo CD application list | Application list loads with sync status |
|
|
||||||
| 2 | Check sync indicators | Each app shows Synced/OutOfSync/Unknown status |
|
|
||||||
| 3 | Click into an application | Detail view shows resources, sync history, health |
|
|
||||||
| 4 | Verify health status | Health icons (Healthy/Degraded/Progressing) are correct |
|
|
||||||
| 5 | Cross-check against `argocd app list` | Status matches CLI output |
|
|
||||||
|
|
||||||
**Pass:** App list loads, sync/health status visible, detail views work, data matches CLI.
|
|
||||||
**Fail:** Page errors, empty list when apps exist, status missing, broken navigation.
|
|
||||||
|
|
||||||
## UAT Artifacts
|
|
||||||
|
|
||||||
For each plugin tested, the UAT report must include:
|
For each plugin tested, the UAT report must include:
|
||||||
|
|
||||||
1. **Screenshots** of the plugin running in Headlamp — sidebar entry visible, main view loaded, at least one detail view
|
1. **Screenshots** of the plugin running in Headlamp — sidebar entry visible, main view loaded, at least one detail view
|
||||||
2. **Test checklist** — each step from the plugin table above marked pass/fail
|
2. **Test checklist** — each step from `UAT_PLAYBOOK.md` marked pass/fail
|
||||||
3. **Console errors** — any browser console errors observed (attach screenshot if present)
|
3. **Console errors** — any browser console errors observed (attach screenshot if present)
|
||||||
4. **Environment** — Headlamp version, plugin version, browser used
|
4. **Environment info** — Headlamp version, plugin version, browser used, namespace context
|
||||||
|
|
||||||
|
## Reading UAT_PLAYBOOK.md
|
||||||
|
|
||||||
|
Each plugin repository contains a `UAT_PLAYBOOK.md` in its root directory. That file contains the canonical test steps for that specific plugin. Before running UAT, read the relevant playbook to know:
|
||||||
|
|
||||||
|
- Which features to exercise
|
||||||
|
- What the expected results are
|
||||||
|
- What screenshots to capture at each step
|
||||||
|
|
||||||
|
If `UAT_PLAYBOOK.md` does not exist for a plugin, treat that as a gap — report it in the UAT findings and flag it as a documentation issue.
|
||||||
|
|
||||||
## Decision Criteria
|
## Decision Criteria
|
||||||
|
|
||||||
- **Approve** the `uat→main` PR when all applicable test steps pass
|
- **Approve** the `uat→main` promotion when all applicable test steps from the playbook pass and no console errors are present
|
||||||
- **Request changes** with specific failing steps and failure screenshots
|
- **Request changes** when any test step fails — include specific failing steps, observed results vs. expected results, and failure screenshots
|
||||||
- **Block** if the plugin fails to load entirely — escalate to CTO as a deployment issue
|
- **Block** if the plugin fails to load entirely — escalate to CTO as a deployment issue requiring immediate resolution
|
||||||
|
|||||||
Reference in New Issue
Block a user