Monitor

What the platform exposes today

Signal	Where	Notes
Health and readiness	/status/healthz and /status/readyz on each service	Three services serve only healthz. See Known limitations
Relay mode and upstream	/status/healthz on the relay	Returns the mode, the upstream and the active relay classes
Prometheus metrics	Worker port 8080	Exposed, but no Service is created for workers, so scraping needs a direct pod target
Kubernetes state	Pods, Deployments, custom resources	The primary signal

No ServiceMonitor, alert rules or dashboards ship with the charts at the moment. If you run Prometheus, you build the scrape configuration.

The four checks worth alerting on

If you set up nothing else, set up these.

1. Pods not ready

kubectl -n gen0sec get pods --no-headers \
  | awk '{split($2,r,"/"); if (r[1] != r[2]) print}'

Any output is a problem. Read the ready column, never a field selector on phase: a pod in CrashLoopBackOff still reports status.phase: Running.

2. The database cluster is not Running

kubectl -n gen0sec-system get postgresql core \
  -o jsonpath='{.status.PostgresClusterStatus}'

Expect Running. Anything else and nothing downstream works.

3. The relay lost its upstream

kubectl -n gen0sec exec deploy/g0s-download-api -- \
  curl -fsS localhost:9999/status/healthz

Expect "status": "ready" and "mode": "proxy".

An unreachable upstream answers 502 on relayed routes rather than 404 or 500. That is deliberate: a polling agent reads 502 as "ask again later" and keeps the artifact it already holds, instead of treating it as withdrawn. So a broken upstream degrades your fleet slowly rather than immediately, which is safer and also easier to miss. Alert on it.

4. The platform stopped serving agents

$KIT/scripts/verify-deployment.sh

Read-only, safe against production, exits non-zero and names the failing check. Running it on a schedule is the single most useful synthetic check available, because it is the only one that proves data actually moves.

Signals that are noisy on purpose

Do not page on these.

Signal	Why it is not an incident
falling back to L1-only cache	The shared cache is never fatal. Services run on their in-process cache. Reads get slower, nothing breaks
A 502 from a relayed route, once	Upstream hiccup. Agents retry and keep what they have
The migration job absent	That is success. Helm deletes it on completion

Logs

kubectl -n gen0sec logs deploy/<service> --tail=200
kubectl -n gen0sec-system logs deploy/g0s-infra-postgres-operator --tail=200

All services log to standard output as structured JSON, so any collector that reads container stdout works without configuration.

Log level is set per service through the values file. See Values reference.

What a healthy install looks like

Five statements, the same five as Verify the install. If you build a dashboard, build it to answer these:

1. Every pod in both namespaces is Running and fully ready.
2. postgresql/core reports Running.
3. kafka/core reports Ready.
4. The relay reports "mode": "proxy" with your upstream.
5. The smoke test exits zero.