privilegedescalation/headlamp-polaris-plugin
12
0
Fork 0
Code Issues 4 Pull Requests 3 Actions Packages Projects Releases 50 Wiki Activity

fix(e2e): serialize concurrent E2E runs to prevent environment conflicts #111

Closed
privilegedescalation-engineer[bot] wants to merge 1 commits from fix/e2e-concurrency-serialization into main
pull from: fix/e2e-concurrency-serialization
merge into: privilegedescalation:main
Conversation 1 Commits 1 Files Changed 1
+10
privilegedescalation-engineer[bot] commented 2026-03-24 16:31:30 +00:00 (Migrated from github.com)
Copy Link
Copy Source

Summary

Concurrent E2E runs share the same headlamp-e2e release name in privilegedescalation-dev. When two PRs trigger E2E tests simultaneously, one run's teardown step (if: always()) deletes the shared Deployment/Service/ConfigMap while the other run is still using it — causing Playwright auth setup to time out trying to reach the Headlamp UI.

Root cause evidence: Pod events in privilegedescalation-dev showed headlamp-e2e cycling through scale 0→1 multiple times in minutes, with each cycle matching a concurrent CI run lifecycle. Run 23499990163 (feat/renovate-extend-org-config) failed while a concurrent run on fix/node24-action-versions was tearing down the shared environment.

Fix: Add a repo-wide concurrency group to the E2E workflow. Only one E2E run executes at a time; others queue (cancel-in-progress: false) rather than cancel, to avoid interrupting teardown and leaving dangling cluster resources.

Changes

  • .github/workflows/e2e.yaml: Add concurrency block with group: e2e-${{ github.repository }} and cancel-in-progress: false

Test Plan

  • Open a second PR while an E2E run is in progress — second run should show "Waiting for pending jobs" in GitHub Actions
  • Confirm the first run completes and tears down cleanly before the second begins
  • Confirm E2E auth setup no longer times out due to environment interference

cc @cpfarhood

## Summary Concurrent E2E runs share the same `headlamp-e2e` release name in `privilegedescalation-dev`. When two PRs trigger E2E tests simultaneously, one run's teardown step (`if: always()`) deletes the shared Deployment/Service/ConfigMap while the other run is still using it — causing Playwright auth setup to time out trying to reach the Headlamp UI. **Root cause evidence:** Pod events in `privilegedescalation-dev` showed `headlamp-e2e` cycling through scale 0→1 multiple times in minutes, with each cycle matching a concurrent CI run lifecycle. Run 23499990163 (`feat/renovate-extend-org-config`) failed while a concurrent run on `fix/node24-action-versions` was tearing down the shared environment. **Fix:** Add a repo-wide `concurrency` group to the E2E workflow. Only one E2E run executes at a time; others queue (`cancel-in-progress: false`) rather than cancel, to avoid interrupting teardown and leaving dangling cluster resources. ## Changes - `.github/workflows/e2e.yaml`: Add `concurrency` block with `group: e2e-${{ github.repository }}` and `cancel-in-progress: false` ## Test Plan - [ ] Open a second PR while an E2E run is in progress — second run should show "Waiting for pending jobs" in GitHub Actions - [ ] Confirm the first run completes and tears down cleanly before the second begins - [ ] Confirm E2E auth setup no longer times out due to environment interference cc @cpfarhood
privilegedescalation-engineer[bot] commented 2026-03-24 16:33:48 +00:00 (Migrated from github.com)
Copy Link
Copy Source

Closing as duplicate of #110. Both PRs add the same group to the E2E workflow. One difference worth noting on #110: I've used cancel-in-progress: false here vs true in #110. Recommend changing #110 to false — when GitHub cancels an in-progress run, the if: always() teardown step may not execute, leaving dangling Deployment/Service/ConfigMap resources in privilegedescalation-dev. Queuing (false) is safer for shared cluster environments.

Closing as duplicate of #110. Both PRs add the same group to the E2E workflow. One difference worth noting on #110: I've used `cancel-in-progress: false` here vs `true` in #110. Recommend changing #110 to `false` — when GitHub cancels an in-progress run, the `if: always()` teardown step may not execute, leaving dangling Deployment/Service/ConfigMap resources in `privilegedescalation-dev`. Queuing (`false`) is safer for shared cluster environments.

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
P0
Must fix - blocking
 P0
Must fix - blocking
 bug
Something isn't working
 bug
Something isn't working
 cla:approved
cla:approved
confirmed
confirmed
documentation
Improvements or additions to documentation
 documentation
Improvements or additions to documentation
 duplicate
This issue or pull request already exists
 duplicate
This issue or pull request already exists
 e2e
e2e
enhancement
New feature or request
 enhancement
New feature or request
 good first issue
Good for newcomers
 good first issue
Good for newcomers
 help wanted
Extra attention is needed
 help wanted
Extra attention is needed
 infra
Infrastructure/ops work
 infra
Infrastructure/ops work
 invalid
This doesn't seem right
 invalid
This doesn't seem right
 pri-917
pri-917
question
Further information is requested
 question
Further information is requested
 typecheck
typecheck
typescript
typescript
wontfix
This will not be worked on
 wontfix
This will not be worked on
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
cpfarhood (Chris Farhood) ci (Continuous Integration [bot]) pe_countess (Countess von Containerheim) flux (Flux CD) pe_gandalf (Gandalf the Greybeard) admin (Gitea Admin) pe_hugh (Hugh Hackman) pe_karen (Kubectl Karen) renovate (Mend Renovate) pe_nancy (Null Pointer Nancy) pe_patty (Pixel Patty) pe_regina (Regression Regina)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: privilegedescalation/headlamp-polaris-plugin#111
Delete Branch "fix/e2e-concurrency-serialization"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?