Skip to content

Right-sizing and usage evidence

A right-sizing answers one question per container: how much of what it reserves does it actually use? On FinOps → Cost & savings → Savings, each right-sizing proposal carries that evidence — requests vs limits vs actual use — and a Send for approval button. See Savings (open the Rightsizing lever).

From the cluster’s own metrics store first. When the seylo-observability chart is installed, a right-sizing reads p50, p95, maximum and mean CPU and working-set memory per container from it, over the window you chose, scoped to the namespaces you can see. The card says Usage evidence: and names the store.

Otherwise from what the agent collects. On a cluster without the store, the agent collects usage itself and pushes aggregates every 60 seconds; the card says so, says why the store could not be used, and shows the command that installs it. Each cluster has one metrics source for that collection, which an admin sets on the Clusters page:

Source CPU and memory Network I/O CPU throttling Needs in the cluster
kubelet (default) Yes Yes Yes Nothing. Read from each node’s kubelet through the Kubernetes API
prometheus Yes Yes Yes A Prometheus the agent can reach inside the cluster
metrics-server Yes No No metrics-server
none No No No Collection is switched off for this cluster

On an agent-managed cluster the agent collects inside your cluster and pushes aggregates out; Seylo never connects to your Prometheus. Raw samples are never stored — only per-window aggregates.

A signal the source cannot report is absent, not zero. metrics-server cannot report network or throttling for anyone. A hardened cluster may refuse the kubelet’s cAdvisor endpoint, in which case network and throttling show as unavailable with the reason.

A namespace-scoped agent shows “unavailable”, never zero

Section titled “A namespace-scoped agent shows “unavailable”, never zero”

A Node is a cluster-scoped object, and a namespaced Role cannot grant access to one. So on an agent installed namespace-scoped, the kubelet source cannot read anything, and Seylo says so:

Cannot list nodes, so the kubelet metrics source cannot be scraped: ... Nodes are CLUSTER-SCOPED: an agent installed with the NAMESPACED Role variant (agent-namespaced.yaml) cannot read them at all. Install the cluster-wide variant, or set this cluster's metrics source to metrics-server or prometheus.

Seylo reports this as unavailable rather than as zero usage, because zero is the one number that would make every workload look over-provisioned and every reduction look safe.

What to do, best first:

  1. Install the seylo-observability chart. Usage is then read from its store, with no cluster-scoped permission needed.
  2. Point the cluster at an in-cluster Prometheus (prometheus source).
  3. Use metrics-server: CPU and memory only.
  4. Install the agent cluster-wide.

Pricing needs Nodes too, so on a namespace-scoped agent the Cost tab shows the agent’s refusal instead of a cost. See troubleshooting.

The defaults:

Value Default rule
CPU request p95 × 1.2
Memory request p95 × 1.2
Memory limit maximum × 1.5, and always strictly above the largest working set observed
CPU limit Not proposed: CPU is left burstable
Bounds CPU 10m to 8000m; memory 32 MiB to 32 GiB
Minimum evidence 30 samples over at least 6 hours

A container is recommended for a change when a request or the memory limit is unset, when a value would move by more than 15%, or when it was OOM-killed or throttled. Its confidence is high at 300 samples over 24 hours or more, medium at 90 samples over 12 hours or more, and low otherwise.

On the Savings tab, each container’s CPU and memory then get one direction. A container pressing on its limit (CPU’s p95 at 90% of the CPU limit or more, or throttled; memory’s maximum at 90% of the memory limit or more, or OOM-killed) gets a Raise of that limit and no lower; otherwise a request above use gets a Lower. See Rightsizing.

The flags behind a recommendation:

Flag Rule
over-provisioned p95 is below 0.5× the request
under-provisioned p95 is above 0.9× the request
throttled more than 1 second of CPU throttling per hour
OOM-killed, restarting seen in the window

Right-sizing is proposed for Deployments. A recommendation for another kind is listed under Not judged with the reason: change its requests in its manifest.

The node reduction plan uses its own, stricter guardrails for reductions.

When you choose Send for approval, the server checks your cluster for an HPA or VPA on the workload at that moment, not from stored data, and checks every rule again over the same usage and window.

Found on the workload Result
A VPA in updateMode other than Off Blocked. The VPA will overwrite whatever is applied. Set it to Off or remove it first
A VPA in updateMode: Off Warning. It will not overwrite the change; its recommendation is a second opinion
An HPA scaling on CPU utilisation Warning. Utilisation is a percentage of the CPU request, so changing the request moves the HPA’s set-point and can cause an immediate scale-out or scale-in
An HPA on other metrics Warning. The replica count is managed automatically
HPA and VPA objects could not be read Warning: treat the patch as possibly conflicting with an autoscaler

Other refusals at submit time:

  • the proposed memory limit is at or below the largest working set observed — it would guarantee an OOM kill;
  • a value is outside the bounds;
  • there is no usage data, or not enough of it.

A change to a workload that Helm, Argo CD or Flux owns is refused until you acknowledge the owner, because the next sync would revert it.

The history the node reduction plan reasons from

Section titled “The history the node reduction plan reasons from”

The node reduction plan sizes from a longer view than a single window: a baseline, an hour-of-week profile and a trend, built from the aggregates the agent pushes.

How long history is kept. By default Seylo keeps per-window rows for 2 days, hourly rows to 14 days, and daily rows to 30 days. An hour-of-week profile needs hourly rows, so it rests on at most 14 days — two observations of each hour of the week. These are settings of the Seylo server, not of your cluster.

Baseline. p50, p95, p99 and maximum CPU and memory over the window, with restarts and OOM kills. Because only aggregates are stored, the p95 is a p95 of per-window p95s: approximate, and biased slightly high — the safe direction for sizing. With fewer than 7 days of history there is no baseline, and the plan says The observation window is not complete.

Hour-of-week profile. p95 CPU for each of the 168 hours of the week, in UTC, starting Monday 00:00. It is what the plan’s busiest-hour replay uses. A pattern is worth scheduling against only with at least 14 days of hourly history, no more than a quarter of the hours empty, and the busiest hour at least 1.3× the quietest.

Trend. Growth in percent per week, fitted on daily p95 values after removing the day-of-week pattern, from at least 7 daily points. A fit with r² below 0.3 is labelled weak.