Files
privilegedescalation-ceo[bot] 2e2713fd3f docs: replace hardcoded namespace with <your-namespace> placeholder
Users choose their own namespace for Headlamp. Replace all hardcoded
kube-system references that indicate Headlamp's install namespace with
<your-namespace> so users substitute their own value.

Upstream workload references left untouched:
- tns-csi controller pods in kube-system (upstream CSI driver)
- NetworkPolicy selectors targeting kube-system
- API server proxy paths to kube-system pods

Refs: PRI-434

Co-authored-by: Chris Farhood <chris@farhood.org>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-05-10 21:34:56 +00:00

2.2 KiB

Metrics Issues

Metrics Page Shows No Data

1. Check the Controller Pod Is Running

kubectl get pods -n kube-system \
  -l app.kubernetes.io/name=tns-csi-driver,app.kubernetes.io/component=controller

The controller pod must be in Running state with all containers ready.

2. Verify Port 8080 Is Exposed

# Check the pod spec for port 8080
kubectl get pod -n kube-system <controller-pod-name> -o yaml | grep -A5 "ports:"

If port 8080 is not declared, the tns-csi driver version you're running may not expose Prometheus metrics. Check the driver documentation.

3. Test the Metrics Endpoint Directly

# Port-forward the controller pod
kubectl port-forward -n kube-system \
  $(kubectl get pods -n kube-system -l app.kubernetes.io/name=tns-csi-driver,app.kubernetes.io/component=controller -o name | head -1) \
  8080:8080

# In another terminal
curl http://localhost:8080/metrics | head -20

If this returns Prometheus text format output, the endpoint is working. If it returns 404 or connection refused, the controller isn't exposing metrics.

4. Check RBAC for Pod Proxy

The plugin accesses metrics via the Kubernetes pod proxy sub-resource:

GET /api/v1/namespaces/kube-system/pods/<pod>/proxy/metrics

This requires get on pods/proxy in kube-system:

kubectl auth can-i get pods/proxy \
  -n kube-system \
  --as=system:serviceaccount:<your-namespace>:headlamp

5. Network Policies

If kube-system has NetworkPolicies, ensure the Kubernetes API server can reach the controller pod on port 8080. The pod proxy hop is performed by the API server, not by Headlamp directly.

Metrics Show Stale Values

The Metrics page fetches data on-demand when the page loads. Click Refresh to re-fetch the latest metrics from the controller pod.

Some Metric Cards Show "—"

Not all tns-csi driver versions expose all metrics. The plugin shows placeholder "—" values for metrics that are absent from the Prometheus output. This is expected behavior.

The plugin specifically looks for:

  • kubelet_volume_stats_* metrics (volume I/O)
  • csi_operations_seconds_* metrics (CSI operation latency)
  • Any tns-csi specific metrics on port 8080