Skip to content

The node reduction plan

The node reduction plan, behind the Fewer, better nodes lever on the Savings tab, is one reviewable plan for a cluster: smaller CPU requests, fewer replicas or a lower HPA floor, the blockers stopping nodes from being removed, and the nodes that could then go. Each sizing item carries the argument for why it is safe. You pick which items to send, and an approver decides. With managed savings on, Seylo carries out its replica and HPA-floor items on its own, inside the managed grant’s limits. Messages still call it the optimisation proposal, and some still name the old Cost & proposals tab; both mean this plan.

  1. Open FinOps → Cost & savings, tab Savings, and pick the cluster.
  2. Under Node reduction plan, choose Generate proposal on the Optimisation proposal card (viewer). This reads the cluster and changes nothing. Once a proposal exists, the same card offers Regenerate.
  3. Read the items, the caveats above them, and the rejections below them.

A proposal needs usage history: a workload needs at least 7 days of it before it has a baseline. Until then the card says The observation window is not complete and how many days have been observed. If the cluster collects no metrics at all, it says Metrics are not yet available for this cluster and why. See the history the plan reasons from.

The items come in a fixed order, because the order is the argument: right-sizing frees capacity, clearing blockers lets the autoscaler use it, and only then can a node go.

Item What it changes On approval
CPU request of a container resources.requests.cpu Applied
Replicas of a Deployment with no HPA spec.replicas Applied
HPA floor of a Deployment with an HPA the HPA’s minReplicas. maxReplicas is never proposed Applied to the HPA the proposal reasoned about
Blocker fix Whatever the blocker needs — a PDB, an annotation, an owner Not executed. A task for a person, shown for review and not selectable
Node reduction Nothing Seylo can do Not executed. A recommendation, reported as outstanding

The proposal does not include memory requests, memory limits or CPU limits. For those, use the Rightsizing lever on Savings. It never proposes a different instance type or node pool.

Node reduction is a recommendation, always

Section titled “Node reduction is a recommendation, always”

Seylo holds no cloud credentials for agent-managed clusters and does not remove nodes or change node pool bounds. A node-reduction item tells you which node your autoscaler could remove, or what you would change at the source.

On some clusters — Karpenter, or a cluster onboarded with cloud credentials — the scale-down card shows node count adjustable and the node-reduction item is marked automated and can be selected. Approving it still changes nothing. It is returned as outstanding with this reason:

Node reduction has no executor in this tool yet; it is reported as outstanding, not done.

The safety argument behind each sizing item

Section titled “The safety argument behind each sizing item”

A sizing item is proposed only if it survives every check below. An item that fails one is not proposed with a warning — it is rejected, and listed with the check that rejected it.

Check Rule
Starting value p95 + 30%, but never below the observed maximum
Never below the observed maximum The workload needed that much at least once. Going below it plans for an event that already happened
Headroom over p95 At least 30%
Busiest-hour replay Replayed against the busiest hour of the week: at least 15% margin must remain
Growth A reduction that the observed trend would outgrow within 8 weeks is refused
Single step No item reduces a value by more than 40% at once. A bigger cut is clamped to 40% and marked as a first step
OOM kills A reduction for a container OOM-killed in the window is refused: its high-water mark is the limit it hit, not what it wanted

The Safety argument panel on each item lists the sentences behind it: the headroom, the busiest hour and its margin, the growth, and whether it was clamped. When no hour-of-week profile exists yet, it says the weekly peak could not be replayed and that the p95 and maximum checks are the only evidence.

The same checks apply to a replica count, in totals: the workload’s current replicas × per-replica usage, against the proposed replicas × request. The evidence says so, and that the total is a snapshot if the count varied during the window. It never proposes fewer than 1 replica.

  • No HPA: spec.replicas is proposed.
  • An HPA: only a lower minReplicas is proposed — a floor the HPA can still scale above. If the HPA already allows going that low, nothing is proposed: the autoscaler can take the saving itself.

A replica reduction must leave the workload drainable. Seylo finds the PodDisruptionBudgets that select the Deployment’s pod-template labels and raises the proposed count until at least one voluntary disruption is still permitted. Percentages round the way Kubernetes rounds them: minAvailable up, maxUnavailable down. If no count below the current one works, the item is rejected and names the budget. If more than one PDB selects the same pods, Kubernetes permits no voluntary disruption at all, and the item is rejected for that reason.

“PDB unknown” is a refusal, not a pass. If the PDB list could not be read — the agent lacks the permission, or the list was truncated — the replica item is rejected with PDB unknown, because a failed read is not evidence that there are no budgets. HPA unknown works the same way: a replica count an unseen HPA would overwrite within a minute is a churn loop, not a saving.

Every candidate the analysis considered and refused is listed under the items, with the rule that refused it. “12 proposed, 30 rejected because of the busiest-hour replay” is what makes the 12 credible. The exact strings are in troubleshooting.

Items that are individually safe can be unsafe together. So the whole plan is checked against the cluster’s forecast peak CPU demand: after the proposed nodes are removed, the remaining allocatable CPU must be at least the forecast peak plus 20%. Freed requests do not count as new capacity — right-sizing reduces reservations, not the CPU a node has.

If the plan fails, it is refused as a set: no items are offered, and the card says Refused as a set - nothing can be submitted with the arithmetic. Seylo does not drop items until it passes, because which items to drop is a decision for you, with the whole picture.

If any workload could not be forecast, the check cannot run, and the proposal carries this caveat instead of passing silently:

No cluster-level forecast peak was available, so the aggregate headroom check could not run. Each item cleared its own guardrails; the SET has not been checked against forecast demand.

Right-sizing and blocker fixes carry no saving of their own. A smaller request frees capacity; the bill only moves when a node goes. So the saving is counted once, on the node-reduction item, at its on-demand rate — and only if that node is priced. See cost and the price book.

Proposal potential saving sums the priced items only. With no price on file it is a dash, and the reason says so; the capacity findings stand regardless. Lower CPU requests do lower your Seylo platform fee directly, though.

  1. Tick the automated items you want (releaser), or Select all automated. Manual items are shown for review only.
  2. Choose Submit for approval. All selected items become one approval request.
  3. An approver or admin reviews and approves it on Approvals. One approval is enough, and an approver may approve their own request.

When you submit, Seylo regenerates the proposal on the server rather than trusting the values in your browser, so nothing reaches an approver that no guardrail has seen. If any item you selected no longer exists — a workload was resized, a node filled up — the whole submission is refused:

The proposal has changed since it was generated, so nothing was submitted. Re-read it and re-select: approving a plan that no longer matches what was reviewed is the one thing an approval gate must not allow.

The tab regenerates the proposal so you can select again.

On approval, Seylo applies the items that were submitted, with the values the approver saw. It does not regenerate the proposal at that point. It does re-read each item’s live value first, and compares it with the current value the approver reviewed:

  • If it still matches, the item goes through the same engine as any other change — the ownership check, a dry run, server-side apply and a read-back.

  • If it has changed since review, for example someone resized the workload by hand while the request waited, that item is refused and nothing is written to it. Applying it would overwrite a change nobody reviewed. The refusal names the field and both values, for example:

    NOT APPLIED: the CPU request of shop/checkout-api is 300m on the cluster now, but this proposal was reviewed against 500m. It was changed after the proposal was submitted, so applying 250m would overwrite a change nobody reviewed. Nothing was written to it; the other items were handled on their own. Generate a new proposal from Cost & proposals (it starts from the live value) and submit that.

    The approval detail marks it changed since review - not applied.

  • If the live value can’t be read, the item is refused too, rather than applied unchecked. The message begins NOT APPLIED: the live ... could not be read.

Items are handled one at a time, and one refused or failed item doesn’t stop the others. Items that are tasks for a person are returned as outstanding, and nothing is done to them.

The request’s status follows its items: applied when every automated item applied and read back, partial when some did and some didn’t, and failed when none did. A refused item counts as not applied. To apply a refused item, generate a new proposal, which starts from the live value, and submit that.

The Nodes and scale-down card answers: which nodes could the autoscaler remove, and for each of the rest, exactly why not? It is read-only (viewer).

The header names the autoscaler Seylo found — Cluster Autoscaler, Karpenter, a managed autoscaler, none, or unknown — and what can be done about node count from here. On an agent-managed cluster with a managed autoscaler, the node pool bounds live in your cloud’s API, which Seylo cannot reach by design. With no autoscaler, freeing capacity does not remove a node at all; it leaves the same nodes emptier.

A node counts as removable when it has no blockers and both its CPU and its memory requests are at or below 50% of allocatable — Cluster Autoscaler’s own default threshold. A node held only by utilisation says Only utilisation holds it; right-sizing could free it.

Each node lists all its blockers, not the first one, so clearing one does not reveal another:

Badge Code What pins the node
PDB pdb-would-be-violated A PodDisruptionBudget currently allows 0 disruptions
safe-to-evict=false safe-to-evict-false A pod sets cluster-autoscaler.kubernetes.io/safe-to-evict=false
local storage local-storage A pod uses emptyDir or hostPath not listed in safe-to-evict-local-volumes
no controller no-controller A pod with no owner — nothing would recreate it
system pod system-pod A kube-system, kube-public or kube-node-lease pod with no PDB
do-not-disrupt do-not-disrupt Karpenter’s karpenter.sh/do-not-disrupt=true on the node or a pod
scale-down disabled node-scale-down-disabled The node carries cluster-autoscaler.kubernetes.io/scale-down-disabled=true
no room elsewhere insufficient-space-elsewhere The other schedulable nodes do not have the free requests to take its pods
capacity unknown capacity-unknown The node’s allocatable was not reported, so it is not assessed

Hover a badge for the object responsible and the suggested remedy. None of them is automated: adding a PDB is an availability decision, and clearing safe-to-evict on a pod that set it on purpose can lose data.

The “no room elsewhere” check is a conservative sum of requests, not a scheduler simulation. It reports a blocker only when arithmetic alone proves the pods cannot fit; passing it means “not obviously impossible”.

A scan that could not read every node or pod says This scan is incomplete, so every “removable” below is an upper bound.