Hugh Hackman
08a3009ba8
Board directive: E2E tests must run in the `default` namespace. Nothing should persist beyond a test run; no dedicated namespace needed. Changes: - e2e-ci-runner-rbac.yaml: retarget Role/RoleBinding to `default`, remove ClusterRole/ClusterRoleBinding (no longer needed since we don't need cluster-scoped namespace read permission) - e2e.yaml: set E2E_NAMESPACE=default - deploy-e2e-headlamp.sh: default namespace to `default`, remove namespace existence check (default always exists) - teardown-e2e-headlamp.sh: default namespace to `default`, remove namespace existence check guard - headlamp-e2e-values.yaml: update usage comment - e2e/README.md: remove namespace creation prerequisite Closes #78 #79 Co-Authored-By: Paperclip <noreply@paperclip.ing>
304 lines
10 KiB
Markdown
304 lines
10 KiB
Markdown
# E2E Smoke Tests
|
|
|
|
Playwright-based smoke tests that validate the Polaris plugin against a live Headlamp deployment.
|
|
|
|
## CI
|
|
|
|
E2E tests run automatically in GitHub Actions on pushes to `main` and pull requests. The workflow (`.github/workflows/e2e.yaml`):
|
|
|
|
1. Builds the plugin (`npm run build`)
|
|
2. Creates a ConfigMap from the built `dist/` output
|
|
3. Deploys a stock Headlamp instance via Helm with the plugin mounted as a ConfigMap volume
|
|
4. Generates a ServiceAccount token for test auth
|
|
5. Runs Playwright tests against the E2E instance
|
|
6. Tears down the E2E instance
|
|
|
|
This approach uses the stock `ghcr.io/headlamp-k8s/headlamp` image with no custom Docker builds. The plugin is loaded via `HEADLAMP_PLUGINS_DIR` volume mount.
|
|
|
|
### Required GitHub Secrets
|
|
|
|
Configure these in GitHub repository settings (Settings → Secrets and variables → Actions):
|
|
|
|
| Secret | Required | Description |
|
|
| -------------------- | -------- | -------------------------------------------------------------- |
|
|
| `AUTHENTIK_USERNAME` | OIDC | Authentik email or username for a CI user with Headlamp access |
|
|
| `AUTHENTIK_PASSWORD` | OIDC | Password for that user |
|
|
|
|
Token-based auth is auto-generated by the deploy script. OIDC secrets are only needed if testing against the shared Headlamp instance.
|
|
|
|
No `GHCR_TOKEN` or Docker registry secrets are needed — the stock Headlamp image is public.
|
|
|
|
## Running Locally
|
|
|
|
### Option 1: OIDC via Authentik (same as CI)
|
|
|
|
```bash
|
|
AUTHENTIK_USERNAME=you@example.com AUTHENTIK_PASSWORD=... npm run e2e
|
|
```
|
|
|
|
The default base URL is `https://headlamp.animaniacs.farh.net`. Override with `HEADLAMP_URL` if needed.
|
|
|
|
### Option 2: K8s bearer token (port-forward)
|
|
|
|
```bash
|
|
kubectl port-forward -n kube-system svc/headlamp 4466:80
|
|
export HEADLAMP_TOKEN=$(kubectl create token headlamp -n kube-system)
|
|
HEADLAMP_URL=http://localhost:4466 npm run e2e
|
|
```
|
|
|
|
Or in headed mode (opens a browser window):
|
|
|
|
```bash
|
|
HEADLAMP_URL=http://localhost:4466 npm run e2e:headed
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
| Variable | Required | Default | Description |
|
|
| -------------------- | -------- | -------------------------------------- | --------------------------------------- |
|
|
| `HEADLAMP_URL` | No | `https://headlamp.animaniacs.farh.net` | Base URL of the Headlamp instance |
|
|
| `AUTHENTIK_USERNAME` | OIDC | — | Authentik email/username |
|
|
| `AUTHENTIK_PASSWORD` | OIDC | — | Authentik password |
|
|
| `HEADLAMP_TOKEN` | Token | — | Kubernetes bearer token (auto-generated in CI) |
|
|
|
|
In CI, `HEADLAMP_URL` and `HEADLAMP_TOKEN` are set automatically by the deploy script. For local runs, set either OIDC credentials or a token manually.
|
|
|
|
## What the Tests Validate
|
|
|
|
- **Sidebar entry** — The Polaris sidebar item appears after login
|
|
- **Overview page** — Cluster score and check distribution render correctly
|
|
- **Namespaces page** — Table of namespaces loads with clickable links
|
|
- **Namespace detail** — Clicking a namespace shows its score and resource table
|
|
|
|
These are smoke tests against real cluster data. They verify the plugin loads and renders without errors, not specific data values.
|
|
|
|
## Test Coverage
|
|
|
|
### Current Tests (`polaris.spec.ts`)
|
|
|
|
1. **`sidebar contains Polaris entry`**
|
|
- Verifies Polaris appears in the navigation sidebar
|
|
- Ensures plugin successfully registered sidebar entry
|
|
|
|
2. **`overview page renders cluster score`**
|
|
- Navigates to `/c/main/polaris`
|
|
- Checks for "Polaris — Overview" heading
|
|
- Verifies cluster score percentage is displayed
|
|
- Validates data fetching and rendering
|
|
|
|
3. **`namespaces page renders table with namespace buttons`**
|
|
- Navigates to `/c/main/polaris/namespaces`
|
|
- Checks for "Polaris — Namespaces" heading
|
|
- Verifies table is visible with at least one row
|
|
- Ensures namespace buttons are clickable
|
|
|
|
4. **`namespace detail drawer opens from table button`**
|
|
- Clicks first namespace button in table
|
|
- Verifies drawer opens with namespace name in heading
|
|
- Checks "Namespace Score" section is visible
|
|
- Confirms "Resources" table is displayed
|
|
- Validates URL hash is updated with namespace name
|
|
|
|
5. **`namespace detail drawer closes with Escape key`**
|
|
- Opens namespace drawer
|
|
- Presses Escape key
|
|
- Verifies drawer closes
|
|
- Checks URL hash is cleared
|
|
|
|
6. **`namespace detail drawer opens from URL hash`**
|
|
- Navigates directly to `/c/main/polaris/namespaces#<namespace>`
|
|
- Verifies drawer automatically opens
|
|
- Checks namespace details are displayed
|
|
|
|
## Prerequisites
|
|
|
|
### Cluster Requirements
|
|
|
|
1. **Polaris Deployment**
|
|
```bash
|
|
# Verify Polaris is running
|
|
kubectl -n polaris get pods
|
|
kubectl -n polaris get svc polaris-dashboard
|
|
```
|
|
|
|
2. **Polaris Audit Data**
|
|
```bash
|
|
# Check if Polaris has generated audit results
|
|
kubectl get --raw /api/v1/namespaces/polaris/services/polaris-dashboard:80/proxy/results.json | jq '.AuditTime'
|
|
```
|
|
|
|
3. **RBAC Permissions**
|
|
- Headlamp service account (or test user) needs `get` on `services/proxy` for `polaris-dashboard`
|
|
- See main README for RBAC setup
|
|
|
|
### Local Setup
|
|
|
|
```bash
|
|
# 1. Install dependencies
|
|
npm install
|
|
npx playwright install chromium
|
|
|
|
# 2. Create .env file (optional, for persistent config)
|
|
cp .env.example .env
|
|
|
|
# 3. Set environment variables
|
|
export HEADLAMP_URL=https://your-headlamp-instance.com
|
|
export HEADLAMP_TOKEN=$(kubectl create token headlamp -n kube-system)
|
|
|
|
# 4. Run tests
|
|
npm run e2e
|
|
```
|
|
|
|
## Debugging
|
|
|
|
### Run in Headed Mode
|
|
|
|
See the browser UI while tests run:
|
|
|
|
```bash
|
|
npm run e2e:headed
|
|
```
|
|
|
|
### Enable Debug Mode
|
|
|
|
Step through tests with Playwright Inspector:
|
|
|
|
```bash
|
|
npx playwright test --debug
|
|
```
|
|
|
|
### Generate Trace
|
|
|
|
Record full trace for failed tests:
|
|
|
|
```bash
|
|
npx playwright test --trace on
|
|
npx playwright show-trace test-results/<test-name>/trace.zip
|
|
```
|
|
|
|
### Screenshot on Failure
|
|
|
|
Tests automatically capture screenshots on failure in `test-results/`
|
|
|
|
### Common Issues
|
|
|
|
**Auth fails with "Sign In button not found":**
|
|
- Check HEADLAMP_URL is correct
|
|
- Verify Headlamp is accessible
|
|
- Ensure OIDC is configured if using Authentik
|
|
|
|
**Polaris sidebar entry not found:**
|
|
- Plugin may not be installed: Check Settings → Plugins in Headlamp
|
|
- Plugin may have failed to load: Check browser console
|
|
- Clear browser cache and hard refresh
|
|
|
|
**Cluster score not displayed:**
|
|
- Polaris may not have audit data yet
|
|
- Check Polaris is running: `kubectl -n polaris get pods`
|
|
- Verify service proxy: `kubectl get --raw /api/v1/namespaces/polaris/services/polaris-dashboard:80/proxy/results.json`
|
|
|
|
**Namespace table empty:**
|
|
- Polaris hasn't run audit yet (wait a few minutes)
|
|
- Check Polaris logs: `kubectl -n polaris logs -l app.kubernetes.io/name=polaris`
|
|
|
|
## Writing New Tests
|
|
|
|
### Example: Testing Plugin Settings
|
|
|
|
```typescript
|
|
test('plugin settings page shows Polaris configuration', async ({ page }) => {
|
|
await page.goto('/c/main/settings/plugins');
|
|
|
|
// Find and click Polaris plugin
|
|
await page.getByText('headlamp-polaris-plugin').click();
|
|
|
|
// Check settings are visible
|
|
await expect(page.getByText('Polaris Settings')).toBeVisible();
|
|
await expect(page.getByText('Refresh Interval')).toBeVisible();
|
|
await expect(page.getByText('Dashboard URL')).toBeVisible();
|
|
});
|
|
```
|
|
|
|
### Example: Testing App Bar Badge
|
|
|
|
```typescript
|
|
test('app bar displays Polaris score badge', async ({ page }) => {
|
|
await page.goto('/c/main');
|
|
|
|
// Badge should be visible in app bar
|
|
const badge = page.getByRole('button', { name: /Polaris: \d+%/ });
|
|
await expect(badge).toBeVisible();
|
|
|
|
// Clicking should navigate to overview
|
|
await badge.click();
|
|
await expect(page).toHaveURL(/\/c\/main\/polaris$/);
|
|
});
|
|
```
|
|
|
|
### Example: Testing Dark Mode
|
|
|
|
```typescript
|
|
test('plugin UI adapts to dark mode', async ({ page }) => {
|
|
await page.goto('/c/main/polaris');
|
|
|
|
// Toggle dark mode
|
|
await page.getByRole('button', { name: /theme/i }).click();
|
|
|
|
// Check background color changes
|
|
const body = page.locator('body');
|
|
await expect(body).toHaveCSS('background-color', 'rgb(18, 18, 18)');
|
|
|
|
// Plugin components should adapt
|
|
const sectionBox = page.locator('[class*="MuiPaper"]').first();
|
|
await expect(sectionBox).not.toHaveCSS('background-color', 'rgb(255, 255, 255)');
|
|
});
|
|
```
|
|
|
|
## CI/CD Integration
|
|
|
|
Tests run automatically in GitHub Actions on pushes to `main` and pull requests. See `.github/workflows/e2e.yaml` for workflow configuration.
|
|
|
|
### Architecture
|
|
|
|
The E2E workflow deploys a **dedicated Headlamp instance** for each test run:
|
|
|
|
1. Build plugin (`npm run build`)
|
|
2. Create ConfigMap from `dist/` output (`scripts/deploy-e2e-headlamp.sh`)
|
|
3. Deploy stock Headlamp via Helm with ConfigMap volume mount
|
|
4. Run Playwright tests against the E2E instance
|
|
5. Tear down (`scripts/teardown-e2e-headlamp.sh`)
|
|
|
|
No custom Docker images, no PVCs, no kubectl exec/cp, no patching of existing deployments. The plugin is mounted from a ConfigMap into the stock Headlamp image.
|
|
|
|
### Cluster Prerequisites
|
|
|
|
One-time setup by a cluster admin:
|
|
|
|
```bash
|
|
kubectl apply -f deployment/e2e-ci-runner-rbac.yaml
|
|
```
|
|
|
|
### Manual Trigger
|
|
|
|
You can manually trigger E2E tests from GitHub Actions:
|
|
1. Go to Actions → E2E Tests
|
|
2. Click "Run workflow"
|
|
3. Select branch and run
|
|
|
|
## Best Practices
|
|
|
|
1. **Use semantic selectors**: `getByRole`, `getByText` over CSS selectors
|
|
2. **Wait for visibility**: Use `await expect(...).toBeVisible()` instead of `waitForTimeout`
|
|
3. **Keep tests independent**: Each test should work in isolation
|
|
4. **Test user flows**: Complete journeys, not just page loads
|
|
5. **Clean up state**: Close drawers/modals after tests
|
|
6. **Use storage state**: Reuse auth across tests (already configured)
|
|
7. **Parallelize carefully**: Currently disabled due to shared state
|
|
|
|
## Resources
|
|
|
|
- [Playwright Documentation](https://playwright.dev/)
|
|
- [Playwright Best Practices](https://playwright.dev/docs/best-practices)
|
|
- [Headlamp Plugin Development](https://headlamp.dev/docs/latest/development/plugins/)
|
|
- [Project Main README](../README.md)
|