Collect Hubble traffic continuously; graph from informer caches
The ring buffer on each agent holds 4095 flows, ~10 s at the observed ~380 flows/s, so the 5/15/60 min windows were really showing the last few seconds. A background follow stream now fills per-minute buckets for the last hour, and counts are distinct connections (5-tuples) instead of Hubble events, with opened (SYN) and dropped broken out. /api/flows reports coveredSec so the UI can say how much of the window the collector has seen since start. /api/graph reads informer caches instead of listing every kind per request (~0.9 s before). Pod-to-workload resolution for flows uses the same cache. YAML view drops managedFields. Frontend: traffic refreshes in place every 30 s without relayout; dash animation rationed (12 fps, only highlighted edges in dense views, paused after a minute idle) since each step redraws the whole element layer, ~15 ms on a 200-element view. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
cf84e9976f
commit
6ead0a0d6d
@@ -10,7 +10,8 @@ Topology map of the Project Chimera Deckhouse cluster.
|
||||
→ workload by label selector); dashed animated arrows are real traffic
|
||||
from Hubble Relay over the chosen window (5 min / 15 min / 1 h), red when
|
||||
Cilium dropped some of it. Peers from other namespaces are drawn as
|
||||
dashed "ghost" cards.
|
||||
dashed "ghost" cards. Traffic refreshes in place every 30 s without
|
||||
relayout; peers that would need new cards wait for "Обновить".
|
||||
- **Right**: details for the selected object — overview, links (config and
|
||||
traffic, clickable), live YAML, pod logs.
|
||||
|
||||
@@ -20,18 +21,27 @@ way Hubble names them, so traffic attaches to the same cards.
|
||||
|
||||
## Backend
|
||||
|
||||
Go, `client-go` in-cluster config. Every `/api/graph` call lists the API
|
||||
fresh — no cache, no watch. `/api/flows` reads recent flows from
|
||||
`hubble-relay.d8-cni-cilium.svc:443` (TLS, verified with the public part of
|
||||
Deckhouse's hubble CA mounted at `/etc/hubble-ca/ca.crt`; relay does not
|
||||
ask for a client certificate) and aggregates them per source → destination
|
||||
→ port. Replies are excluded, so each connection counts once.
|
||||
Go, `client-go` in-cluster config. The graph is built from informer caches
|
||||
(watch-backed, managedFields stripped), so `/api/graph` costs milliseconds
|
||||
instead of ~0.9 s of List calls.
|
||||
|
||||
Traffic: each Cilium agent keeps only 4095 flows in its ring buffer — at a
|
||||
few hundred flows/s that is ~10 s of history, so reading the buffer on
|
||||
request can't answer "last 15 minutes". A background follow stream from
|
||||
`hubble-relay.d8-cni-cilium.svc:443` (TLS verified with the public part of
|
||||
Deckhouse's hubble CA at `/etc/hubble-ca/ca.crt`; relay does not ask for a
|
||||
client certificate) feeds per-minute buckets for the last hour. Counts are
|
||||
**distinct connections** (5-tuples), not Hubble events — one connection
|
||||
produces several events. `new` = opened by TCP SYN inside the window,
|
||||
`dropped` = tuples with a DROPPED verdict. After a restart the collector
|
||||
only knows what it has seen since; `coveredSec` says how much of the
|
||||
window that is.
|
||||
|
||||
| Endpoint | What |
|
||||
|---|---|
|
||||
| `/api/graph` | nodes + config edges + pod placement |
|
||||
| `/api/flows?namespaces=a,b&window=300` | aggregated Hubble traffic |
|
||||
| `/api/resource?kind=&namespace=&name=` | live YAML |
|
||||
| `/api/flows?namespaces=a,b&window=300` | aggregated traffic touching those namespaces |
|
||||
| `/api/resource?kind=&namespace=&name=` | live YAML (managedFields stripped) |
|
||||
| `/api/pods?id=wl/<ns>/<kind>/<name>&namespace=` | pods of a workload |
|
||||
| `/api/logs?namespace=&pod=&container=` | last 500 log lines |
|
||||
| `/healthz` | liveness/readiness |
|
||||
@@ -46,16 +56,12 @@ CDN. Theme follows `prefers-color-scheme`.
|
||||
|
||||
## Build and deploy
|
||||
|
||||
The in-cluster registry can't be fed from inside the cluster yet (Kaniko's
|
||||
image lives on gcr.io, blocked here), so for now:
|
||||
|
||||
```
|
||||
docker build -t cluster-map:vX .
|
||||
docker save cluster-map:vX -o cluster-map.tar
|
||||
scp cluster-map.tar xeon:/tmp/
|
||||
ssh xeon 'sudo /opt/deckhouse/bin/ctr -n k8s.io images import /tmp/cluster-map.tar'
|
||||
R=git.project-chimera.ru/fanchji/cluster-map/cluster-map-app
|
||||
docker build -t $R:vX . && docker push $R:vX
|
||||
docker inspect --format '{{index .RepoDigests 0}}' $R:vX
|
||||
```
|
||||
|
||||
The Deployment pins `nodeSelector: xeon` + `imagePullPolicy: Never`.
|
||||
Manifests: `03-kubernetes/apps/cluster-map/` in `project-chimera`, mirrored
|
||||
to `k8s-manifests` for Argo.
|
||||
The Deployment pins the image by digest. Anonymous pull works (public Gitea
|
||||
user), no imagePullSecret. Manifests: `03-kubernetes/apps/cluster-map/` in
|
||||
`project-chimera`, mirrored to `k8s-manifests` for Argo.
|
||||
|
||||
Reference in New Issue
Block a user