Compare commits

..

1 Commits

Author SHA1 Message Date
privilegedescalation-engineer[bot] 2a156c4e63 fix(e2e): serialize concurrent E2E runs to prevent environment conflicts
Concurrent E2E runs share the same headlamp-e2e release name in
privilegedescalation-dev. When two runs overlap, one run's teardown
(if: always()) deletes the shared Deployment/Service/ConfigMap while
the other is still using it, causing Playwright auth setup to time out
waiting for the Headlamp UI.

Adds a repo-wide concurrency group so only one E2E run executes at a
time. cancel-in-progress: false queues incoming runs rather than
cancelling in-flight ones to avoid leaving dangling cluster resources
when teardown is interrupted.

Fixes: PRI-815

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-24 16:31:14 +00:00
12 changed files with 187 additions and 580 deletions
-2
View File
@@ -16,5 +16,3 @@ jobs:
dual-approval:
uses: privilegedescalation/.github/.github/workflows/dual-approval-check.yaml@main
secrets: inherit
with:
pr_number: ${{ github.event.pull_request.number }}
+7 -20
View File
@@ -10,22 +10,19 @@ on:
permissions:
contents: read
# Only one E2E run at a time: the shared E2E_RELEASE (headlamp-e2e) in
# headlamp-dev cannot be shared across concurrent runs.
# cancel-in-progress: false (queue, don't cancel) — cancelling in-flight
# runs may skip the if:always() teardown, leaving dangling cluster resources.
# Serialize E2E runs repo-wide. All concurrent runs share the same
# E2E_RELEASE name (headlamp-e2e) in a single namespace. Without
# serialization, one run's teardown (if: always()) deletes the
# deployment while a concurrent run is still using it, causing auth
# setup timeouts. cancel-in-progress: false queues rather than kills
# to avoid leaving dangling cluster resources.
concurrency:
group: e2e-${{ github.repository }}
cancel-in-progress: false
env:
E2E_NAMESPACE: headlamp-dev
E2E_NAMESPACE: privilegedescalation-dev
E2E_RELEASE: headlamp-e2e
# Pin to a known-good Headlamp version. Using :latest is risky because
# the tag can change between CI runs, causing flaky failures when a newer
# image is pulled on some nodes but not others (IfNotPresent pull policy).
# Update this when Headlamp is upgraded in production (kube-system).
HEADLAMP_VERSION: v0.40.1
jobs:
e2e:
@@ -72,16 +69,6 @@ jobs:
HEADLAMP_URL: ${{ env.HEADLAMP_URL }}
HEADLAMP_TOKEN: ${{ env.HEADLAMP_TOKEN }}
- name: Collect deployment diagnostics on failure
if: failure()
run: |
echo "=== Pod state ==="
kubectl get pods -n "$E2E_NAMESPACE" -l "app.kubernetes.io/instance=$E2E_RELEASE" 2>&1 || true
echo "=== Pod describe ==="
kubectl describe pods -n "$E2E_NAMESPACE" -l "app.kubernetes.io/instance=$E2E_RELEASE" 2>&1 || true
echo "=== Recent namespace events ==="
kubectl get events -n "$E2E_NAMESPACE" --sort-by='.lastTimestamp' 2>&1 | tail -20 || true
- name: Teardown E2E instance
if: always()
run: scripts/teardown-e2e-headlamp.sh
+1 -1
View File
@@ -229,7 +229,7 @@ Headlamp v0.39.0 with default `watchPlugins: true` treats catalog-managed plugin
**Action Items:**
- [ ] Parallelize test execution
- [ ] Add npm cache to GitHub Actions
- [x] Renovate is configured org-wide via `github>privilegedescalation/.github:renovate-config`
- [ ] Integrate Dependabot
- [ ] Add semantic-release
---
+1 -1
View File
@@ -212,7 +212,7 @@ If you discover a security vulnerability in this plugin, please report it via:
The project uses:
- **npm audit**: Runs automatically during `npm install`
- **Renovate**: Automated dependency updates via Mend Renovate (org-wide configured)
- **Dependabot**: GitHub Dependabot monitors dependencies and creates PRs for updates
- **GitHub Actions**: CI workflow runs `npm audit` on every commit
### Updating Dependencies
+1 -1
View File
@@ -57,7 +57,7 @@ changes:
- kind: added
description: ExemptionManager test suite — full coverage of annotation-based exemption flows
- kind: fixed
description: E2E infrastructure overhauled — ConfigMap volume mount replaces Dockerfile-based approach, tests run in headlamp-dev namespace
description: E2E infrastructure overhauled — ConfigMap volume mount replaces Dockerfile-based approach, tests run in privilegedescalation-dev namespace
- kind: fixed
description: E2E workflow uses token auth and waits for HTTP reachability before running tests
- kind: fixed
+7 -7
View File
@@ -2,26 +2,26 @@
# RBAC for the GitHub Actions CI runner to manage the E2E Headlamp instance.
# CI-only test fixture — NOT for production use.
#
# Grants the ARC runner service account permissions in the headlamp-dev
# Grants the ARC runner service account permissions in the privilegedescalation-dev
# namespace to deploy and tear down a dedicated Headlamp instance via Helm.
# E2E resources run in `headlamp-dev` — nothing persists beyond a test run.
# E2E resources run in `privilegedescalation-dev` — nothing persists beyond a test run.
#
# Plugin is loaded via ConfigMap volume mount — no custom Docker images.
#
# Note: This RBAC is mirrored in privilegedescalation/infra (base/rbac/)
# and managed by Flux GitOps. The infra repo is the source of truth.
# Prerequisites:
# kubectl apply -f deployment/e2e-ci-runner-rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: e2e-ci-runner
namespace: headlamp-dev
namespace: privilegedescalation-dev
rules:
# Helm needs to manage these resources for the Headlamp chart
- apiGroups: ["apps"]
resources: ["deployments"]
verbs: ["get", "list", "create", "update", "patch", "delete", "watch"]
- apiGroups: [""]
resources: ["services", "serviceaccounts", "configmaps", "secrets", "events"]
resources: ["services", "serviceaccounts", "configmaps", "secrets"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
resources: ["pods"]
@@ -35,7 +35,7 @@ apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: e2e-ci-runner-binding
namespace: headlamp-dev
namespace: privilegedescalation-dev
subjects:
- kind: ServiceAccount
name: runners-privilegedescalation-gha-rs-no-permission
+2 -6
View File
@@ -45,12 +45,8 @@ async function authenticateWithToken(page: Page, token: string): Promise<void> {
await page.waitForURL(/\/(login|token)$/);
if (page.url().includes('/login')) {
// OIDC login page — click "use a token" to reach token auth.
// Wait explicitly before clicking so failures surface at 15 s
// with a clear message rather than silently timing out at 60 s.
const useTokenBtn = page.getByRole('button', { name: /use a token/i });
await useTokenBtn.waitFor({ state: 'visible', timeout: 15_000 });
await useTokenBtn.click();
// OIDC login page — click "use a token" to reach token auth
await page.getByRole('button', { name: /use a token/i }).click();
await page.waitForURL('**/token');
}
+1 -4
View File
@@ -35,10 +35,7 @@
"overrides": {
"tar": "^7.5.11",
"undici": "^7.24.3",
"flatted": "^3.4.2",
"lodash": ">=4.18.0",
"picomatch": ">=4.0.4",
"vite": ">=6.4.2"
"flatted": "^3.4.2"
}
},
"devDependencies": {
+141 -518
View File
File diff suppressed because it is too large Load Diff
+17 -1
View File
@@ -1,5 +1,21 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": ["github>privilegedescalation/.github:renovate-config"]
"extends": ["config:recommended"],
"baseBranches": ["main"],
"schedule": ["every weekend"],
"prConcurrentLimit": 10,
"pinDigests": true,
"packageRules": [
{
"matchManagers": ["npm"],
"matchUpdateTypes": ["minor", "patch"],
"groupName": "npm minor and patch"
},
{
"matchManagers": ["github-actions"],
"matchUpdateTypes": ["minor", "patch"],
"groupName": "github-actions minor and patch"
}
]
}
+7 -17
View File
@@ -5,26 +5,26 @@
# a ConfigMap volume mount. No custom Docker images — the plugin is built
# in CI and injected as a ConfigMap.
#
# E2E resources are deployed to the `headlamp-dev` namespace. Nothing
# persists beyond a test run — teardown cleans up all created resources.
# E2E resources are deployed to the `privilegedescalation-dev` namespace. Nothing
# persists beyond the test run — teardown cleans up all created resources.
#
# Prerequisites:
# - Plugin built (dist/ exists with plugin-main.js + package.json)
# - kubectl configured with cluster access
# - RBAC applied (managed by Flux GitOps in privilegedescalation/infra)
# - RBAC applied: kubectl apply -f deployment/e2e-ci-runner-rbac.yaml
#
# Environment:
# E2E_NAMESPACE — namespace for E2E Headlamp (default: headlamp-dev)
# E2E_NAMESPACE — namespace for E2E Headlamp (default: privilegedescalation-dev)
# E2E_RELEASE — release/resource name prefix (default: headlamp-e2e)
# HEADLAMP_VERSION — Headlamp image tag (default: v0.40.1, pinned to match production)
# HEADLAMP_VERSION — Headlamp image tag (default: latest)
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
DIST_DIR="$REPO_ROOT/dist"
E2E_NAMESPACE="${E2E_NAMESPACE:-headlamp-dev}"
E2E_NAMESPACE="${E2E_NAMESPACE:-privilegedescalation-dev}"
E2E_RELEASE="${E2E_RELEASE:-headlamp-e2e}"
HEADLAMP_VERSION="${HEADLAMP_VERSION:-v0.40.1}"
HEADLAMP_VERSION="${HEADLAMP_VERSION:-latest}"
if [ ! -d "$DIST_DIR" ]; then
echo "ERROR: dist/ not found. Run 'npm run build' first." >&2
@@ -58,16 +58,6 @@ kubectl create configmap headlamp-polaris-plugin \
--from-file="$DIST_DIR" \
--from-file=package.json="$REPO_ROOT/package.json"
# --- Tear down any existing E2E deployment for a clean start ---
# kubectl apply without prior deletion only patches in-place: if the pod spec is
# unchanged between runs, no new rollout is triggered and a degraded pod keeps
# serving. Delete first to guarantee a fresh pod regardless of prior state.
echo ""
echo "Removing any existing E2E deployment (clean-start)..."
kubectl delete deployment "${E2E_RELEASE}" -n "$E2E_NAMESPACE" --ignore-not-found --wait
kubectl delete service "${E2E_RELEASE}" -n "$E2E_NAMESPACE" --ignore-not-found --wait
kubectl delete serviceaccount "${E2E_RELEASE}" -n "$E2E_NAMESPACE" --ignore-not-found --wait
# --- Deploy Headlamp via kubectl apply ---
echo ""
echo "Deploying Headlamp E2E instance..."
+2 -2
View File
@@ -4,13 +4,13 @@
# Tears down the dedicated E2E Headlamp instance deployed by deploy-e2e-headlamp.sh.
#
# Environment:
# E2E_NAMESPACE — namespace to clean up (default: headlamp-dev)
# E2E_NAMESPACE — namespace to clean up (default: privilegedescalation-dev)
# E2E_RELEASE — release/resource name prefix (default: headlamp-e2e)
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
E2E_NAMESPACE="${E2E_NAMESPACE:-headlamp-dev}"
E2E_NAMESPACE="${E2E_NAMESPACE:-privilegedescalation-dev}"
E2E_RELEASE="${E2E_RELEASE:-headlamp-e2e}"
echo "=== E2E Headlamp Teardown ==="