privilegedescalation/headlamp-polaris-plugin
12
0
Fork 0
Code Issues 4 Pull Requests 3 Actions Packages Projects Releases 50 Wiki Activity

E2E Test Failure: Headlamp deployment rollout timeout #115

New Issue
Closed
opened 2026-03-24 21:19:55 +00:00 by privilegedescalation-qa[bot] · 2 comments
privilegedescalation-qa[bot] commented 2026-03-24 21:19:55 +00:00 (Migrated from github.com)
Copy Link
Copy Source

E2E Test Failure

Run: https://github.com/privilegedescalation/headlamp-polaris-plugin/actions/runs/23506620087

Failure: The E2E test deploys a temporary Headlamp instance to the privilegedescalation-dev namespace, but the deployment times out waiting for rollout.

Log excerpt:

Waiting for deployment "headlamp-e2e" rollout to finish: 0 out of 1 new replicas have been updated...
Waiting for deployment "headlamp-e2e" rollout to finish: 0 of 1 updated replicas are available...
error: timed out waiting for the condition

Impact: E2E tests cannot run, preventing validation of the v1.0.0 release.

Reproduced: 2026-03-24 on main branch

Need to investigate: Why is the headlamp-e2e deployment not starting in the CI environment?

## E2E Test Failure **Run:** https://github.com/privilegedescalation/headlamp-polaris-plugin/actions/runs/23506620087 **Failure:** The E2E test deploys a temporary Headlamp instance to the `privilegedescalation-dev` namespace, but the deployment times out waiting for rollout. **Log excerpt:** ``` Waiting for deployment "headlamp-e2e" rollout to finish: 0 out of 1 new replicas have been updated... Waiting for deployment "headlamp-e2e" rollout to finish: 0 of 1 updated replicas are available... error: timed out waiting for the condition ``` **Impact:** E2E tests cannot run, preventing validation of the v1.0.0 release. **Reproduced:** 2026-03-24 on main branch **Need to investigate:** Why is the headlamp-e2e deployment not starting in the CI environment?
privilegedescalation-qa[bot] commented 2026-03-24 21:35:45 +00:00 (Migrated from github.com)
Copy Link
Copy Source

Root cause identified and fix in review.

Root cause: The :latest Headlamp image tag drifted on the buttons cluster node. That node has a newer image (sha256:89c6c65) that fails readiness probes within 120s, while the other nodes have v0.40.1 (sha256:281756c) which is known-good. Additionally, 3 simultaneous merges triggered concurrent E2E runs that deleted each other's deployments (one-time bootstrap issue).

Fix: PR #116 pins HEADLAMP_VERSION=v0.40.1 (matching production). QA approved ✅, CI and E2E pass on the feature branch. Waiting on CTO review before merge.

Once PR #116 merges, this bug should be resolved.

Root cause identified and fix in review. **Root cause:** The `:latest` Headlamp image tag drifted on the `buttons` cluster node. That node has a newer image (`sha256:89c6c65`) that fails readiness probes within 120s, while the other nodes have `v0.40.1` (`sha256:281756c`) which is known-good. Additionally, 3 simultaneous merges triggered concurrent E2E runs that deleted each other's deployments (one-time bootstrap issue). **Fix:** PR #116 pins `HEADLAMP_VERSION=v0.40.1` (matching production). QA approved ✅, CI and E2E pass on the feature branch. Waiting on CTO review before merge. Once PR #116 merges, this bug should be resolved.
privilegedescalation-qa[bot] commented 2026-03-24 21:45:29 +00:00 (Migrated from github.com)
Copy Link
Copy Source

Closing: E2E tests are passing on main following the merge of PR #116 (fix(e2e): pin Headlamp image to v0.40.1 instead of :latest). The root cause was :latest image drift with IfNotPresent pull policy — now pinned to v0.40.1. CI run #23513632751 completed with conclusion: success.

Closing: E2E tests are passing on main following the merge of PR #116 (fix(e2e): pin Headlamp image to v0.40.1 instead of :latest). The root cause was `:latest` image drift with `IfNotPresent` pull policy — now pinned to `v0.40.1`. CI run [#23513632751](https://github.com/privilegedescalation/headlamp-polaris-plugin/actions/runs/23513632751) completed with conclusion: **success**.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
P0
Must fix - blocking
 P0
Must fix - blocking
 bug
Something isn't working
 bug
Something isn't working
 cla:approved
cla:approved
confirmed
confirmed
documentation
Improvements or additions to documentation
 documentation
Improvements or additions to documentation
 duplicate
This issue or pull request already exists
 duplicate
This issue or pull request already exists
 e2e
e2e
enhancement
New feature or request
 enhancement
New feature or request
 good first issue
Good for newcomers
 good first issue
Good for newcomers
 help wanted
Extra attention is needed
 help wanted
Extra attention is needed
 infra
Infrastructure/ops work
 infra
Infrastructure/ops work
 invalid
This doesn't seem right
 invalid
This doesn't seem right
 pri-917
pri-917
question
Further information is requested
 question
Further information is requested
 typecheck
typecheck
typescript
typescript
wontfix
This will not be worked on
 wontfix
This will not be worked on
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
cpfarhood (Chris Farhood) ci (Continuous Integration [bot]) pe_countess (Countess von Containerheim) flux (Flux CD) pe_gandalf (Gandalf the Greybeard) admin (Gitea Admin) pe_hugh (Hugh Hackman) pe_karen (Kubectl Karen) renovate (Mend Renovate) pe_nancy (Null Pointer Nancy) pe_patty (Pixel Patty) pe_regina (Regression Regina)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: privilegedescalation/headlamp-polaris-plugin#115
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?