Package cluster-map as a Deckhouse module

Layout, build and templates follow deckhouse/modules-template; dmt lint
passes with no findings (Go files are excluded from the Flant-only
license-header rule, see .dmtlint.yaml).

- module.yaml, openapi schemas with Russian docs, lib-helm templates:
  https modes, ingress class, publicDomainTemplate, node placement,
  priority class, PSS-restricted security context, VPA, PDB, Ingress
  plus Gateway API ListenerSet/HTTPRoute, DexAuthenticator.
- module-sdk hooks: hubble CA from d8-cni-cilium (public certificate
  only), external_auth, copy_custom_certificate.
- The app moves to images/cluster-map/src and is split into packages;
  traffic mode Continuous/OnDemand/Disabled and retention are module
  settings; graceful shutdown; Prometheus metrics and two alerts;
  frontend embedded, English and Russian, strict CSP.
- Built with werf on Deckhouse's published base images; Makefile
  publishes the registry layout modules-actions uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Vitaliy Chefanov
2026-09-23 21:17:52 +03:00
co-authored by Claude Opus 5.5
parent 41f9b47556
commit 4518393ce1
82 changed files with 5712 additions and 1415 deletions
+93 -62
View File
@@ -1,74 +1,105 @@
# cluster-map
Topology map of the Project Chimera Deckhouse cluster.
A Deckhouse Kubernetes Platform module: an interactive topology map of the cluster with live traffic from Cilium Hubble. User documentation is in [docs/README.md](docs/README.md) ([на русском](docs/README_RU.md)).
- **Left**: inventory tree in two views. *Namespaces* groups objects by
namespace and kind; *Ноды* shows what is scheduled on each node. Search
matches names, images and ingress hosts.
- **Centre**: the selected namespace(s) or node as compact cards inside
namespace boxes. Solid arrows come from configuration (Ingress → Service
→ workload by label selector); dashed animated arrows are real traffic
from Hubble Relay over the chosen window (5 min / 15 min / 1 h), red when
Cilium dropped some of it. Peers from other namespaces are drawn as
dashed "ghost" cards. Kubelet probes, DNS, cilium health and node-to-node
traffic are not collected (see Backend). Traffic refreshes in place every 30 s without
relayout; peers that would need new cards wait for "Обновить".
- **Right**: details for the selected object — overview, links (config and
traffic, clickable), live YAML, pod logs.
The module follows [deckhouse/modules-template](https://github.com/deckhouse/modules-template): werf build, lib-helm templates, module-sdk hooks, and it passes `dmt lint` with no findings.
Workloads under non-native controllers (kruise Advanced DaemonSet behind
ingress-nginx, database operators, bare Jobs) are included and named the
way Hubble names them, so traffic attaches to the same cards.
## Layout
## Backend
Go, `client-go` in-cluster config. The graph is built from informer caches
(watch-backed, managedFields stripped), so `/api/graph` costs milliseconds
instead of ~0.9 s of List calls.
Traffic: each Cilium agent keeps only 4095 flows in its ring buffer — at a
few hundred flows/s that is ~10 s of history, so reading the buffer on
request can't answer "last 15 minutes". A background follow stream from
`hubble-relay.d8-cni-cilium.svc:443` (TLS verified with the public part of
Deckhouse's hubble CA at `/etc/hubble-ca/ca.crt`; relay does not ask for a
client certificate) feeds per-minute buckets for the last hour. Counts are
**distinct connections** (5-tuples), not Hubble events — one connection
produces several events. `new` = opened by TCP SYN inside the window,
`dropped` = tuples with a DROPPED verdict. After a restart the collector
only knows what it has seen since; `coveredSec` says how much of the
window that is.
Infra traffic is filtered at the agents (`infraFilters` in `flows.go`):
host-originated flows (kubelet probes, hostNetwork pods), remote-node,
cilium health and DNS :53. On a 12k-event sample they were ~74% of
non-reply events, none of them attach to a workload card, and streaming
them cost hubble-relay ~400m CPU.
| Endpoint | What |
| Path | What |
|---|---|
| `/api/graph` | nodes + config edges + pod placement |
| `/api/flows?namespaces=a,b&window=300` | aggregated traffic touching those namespaces |
| `/api/resource?kind=&namespace=&name=` | live YAML (managedFields stripped) |
| `/api/pods?id=wl/<ns>/<kind>/<name>&namespace=` | pods of a workload |
| `/api/logs?namespace=&pod=&container=` | last 500 log lines |
| `/healthz` | liveness/readiness |
| `module.yaml`, `Chart.yaml`, `.helmignore`, `oss.yaml`, `changelog.yaml` | module metadata |
| `openapi/` | `config-values.yaml` (ModuleConfig settings), `doc-ru-config-values.yaml`, `values.yaml` (internal values) |
| `templates/` | Helm templates on [lib-helm](https://github.com/deckhouse/lib-helm) (`charts/deckhouse_lib_helm-*.tgz`) |
| `hooks/batch/` | Go hooks on [module-sdk](https://github.com/deckhouse/module-sdk): hubble CA, external auth, custom certificate |
| `images/cluster-map/` | the application image: `werf.inc.yaml` and the Go source in `src/` |
| `monitoring/prometheus-rules/` | alert rules |
| `docs/` | user documentation, English and Russian |
| `.werf/`, `werf.yaml`, `werf-giterminism.yaml` | build, from modules-template |
| `tests/values/` | values for `make render` |
## Frontend
### The application (`images/cluster-map/src`)
Plain JS, no build step: `static/index.html`, `app.css`, `app.js`.
Cytoscape.js draws, elkjs lays out in two passes (each box on its own with
its unconnected groups packed in rows, then the boxes against each other).
Both libraries are vendored in `static/vendor/` so the page works without a
CDN. Theme follows `prefers-color-scheme`.
| Package | What |
|---|---|
| `cmd/cluster-map` | wiring, signals, graceful shutdown |
| `internal/config` | settings from the environment (the Deployment template is the only writer) |
| `internal/kube` | informer caches, managedFields stripped |
| `internal/topology` | the graph: workloads, Services, Ingresses, nodes, and the pod-owner rules shared with traffic |
| `internal/traffic` | Hubble follow stream, per-minute buckets of distinct connections |
| `internal/server` | HTTP API, CSP headers, request metrics |
| `internal/metrics` | Prometheus metrics |
| `web/` | the frontend, embedded into the binary: plain JS, Cytoscape.js, elkjs, English and Russian |
## Build and deploy
API: `/api/graph`, `/api/flows?namespaces=a,b&window=<seconds>`, `/api/resource?kind=&namespace=&name=`, `/api/pods?id=&namespace=`, `/api/logs?namespace=&pod=&container=`, `/healthz`, `/readyz`; metrics on `:9090/metrics`.
```
R=git.project-chimera.ru/fanchji/cluster-map/cluster-map-app
docker build -t $R:vX . && docker push $R:vX
docker inspect --format '{{index .RepoDigests 0}}' $R:vX
## Development
```shell
make test # go test -race for the app, hook tests on the module-sdk framework
make lint # golangci-lint and dmt
make render # helm template with tests/values/*.yaml
```
The Deployment pins the image by digest. Anonymous pull works (public Gitea
user), no imagePullSecret. Manifests: `03-kubernetes/apps/cluster-map/` in
`project-chimera`, mirrored to `k8s-manifests` for Argo.
`dmt` is built from [deckhouse/dmt](https://github.com/deckhouse/dmt); `crane` from go-containerregistry.
## Build and publish
Base images come from the `base_images.yml` Deckhouse publishes for its own modules (`make base-images`; the file is not committed, as in the template).
```shell
docker login git.project-chimera.ru
make publish VERSION=v0.7.0 # build, push bundle and release, register, channel alpha
make release VERSION=v0.7.0 CHANNEL=stable
```
The registry layout matches [deckhouse/modules-actions](https://github.com/deckhouse/modules-actions):
```text
<SOURCE>:cluster-map module registration (empty image)
<SOURCE>/cluster-map:v0.7.0 bundle
<SOURCE>/cluster-map:<content-tag> application image
<SOURCE>/cluster-map/release:v0.7.0 release
<SOURCE>/cluster-map/release:<channel> alpha | beta | early-access | stable | rock-solid
```
## Install
```yaml
apiVersion: deckhouse.io/v1alpha1
kind: ModuleSource
metadata:
name: cluster-map
spec:
registry:
repo: git.project-chimera.ru/fanchji/cluster-map
scheme: HTTPS
---
apiVersion: deckhouse.io/v1alpha2
kind: ModuleUpdatePolicy
metadata:
name: cluster-map
spec:
releaseChannel: Alpha
update:
mode: Auto
---
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
name: cluster-map
spec:
enabled: true
version: 1
updatePolicy: cluster-map
source: cluster-map
settings:
auth:
allowedUserGroups: [admins]
```
The module is `Experimental`: enable `allowExperimentalModules` in the `deckhouse` ModuleConfig first.
## License
Apache 2.0, see [LICENSE](LICENSE). Go files carry an Apache-2.0 header with a neutral copyright line; DMT's license rule expects `Copyright <year> Flant JSC`, so `.dmtlint.yaml` excludes the two Go directories from that rule until the headers are changed on transfer.