Skip to content

Autoscaler or a bigger request

When a workload uses more CPU than it requests, there are two fixes, and picking the wrong one costs money or does nothing:

  • An autoscaler (HPA) adds pods when load rises. It helps only when the load splits across pods.
  • A bigger request gives each pod what it uses. It is right when the load is flat, or when one pod does all the work.

Nightshift does not guess which. It asks the question an SRE asks first - if I add a pod, does each pod’s CPU go down? - and answers it from measurements.

Measurement Read from
Each pod’s CPU over the last 24 hours, or since the workload was created if that is more recent Your metrics store, at a 2-minute step: its 95th percentile, its quietest level, and how much the total moves. A workload deleted and created again under the same name does not inherit the old one’s load
The workload’s traffic The first source that answers: span metrics from traces, http_requests_total (and its Spring and OpenTelemetry names), or the ingress-nginx controller in front of its Services
Whether CPU follows traffic How closely the two move together over the same timestamps
How the load spreads With two or more pods, whether one of them does the work
What already scales it An existing HPA, or a VerticalPodAutoscaler that sets its CPU
Room and limits Room on the nodes for another pod, and its PodDisruptionBudget. Said beside the answer, never deciding it

These come from the observability chart’s stores, so the chart needs to be installed. Without a traffic metric, Nightshift cannot tell whether CPU follows traffic, and says so.

Answer When What Nightshift proposes
Autoscaler CPU rises and falls with traffic, and the load splits across pods A HorizontalPodAutoscaler on CPU utilisation
Request CPU is flat, or does not follow traffic, or one pod does the work, or it is a DaemonSet A CPU request sized from the measured 95th percentile
Both An autoscaler is right, but today’s request is wrong for one The request first; the HPA once the request holds. An HPA reads utilisation against the request, so a wrong request makes the HPA wrong too
Unknown Something needed was not read, or the measurements sit between the rules Nothing on CPU. The answer names the read that would decide it

Each rule is a stated number over measured values:

Rule Value
Enough history to judge a shape At least 30 samples, an hour at the 2-minute step
CPU is flat Its variation is below 0.15
CPU follows traffic Correlation of 0.7 or more
CPU does not follow traffic Correlation of 0.3 or less. Between 0.3 and 0.7 is unknown
One pod does the work The busiest pod averages 3 times the others or more
The request is wrong for an HPA Even its quietest pods use more than the request (an HPA would never scale in), or its busiest 5% use under a quarter of it (an HPA would never scale out)
A VerticalPodAutoscaler sets CPU Its own bounds are the request to change. Never an HPA on CPU beside it
A one-replica StatefulSet, an uneven split, a loose correlation Unknown: a second pod sharing the load is measured, not assumed

The answer is a numbered piece of evidence in the investigation, with its measurements, and Nightshift’s proposal must cite it:

  • An HPA is created only when the answer is “autoscaler” and the proposal cites it. Otherwise the proposal is refused, with the reason and what to send instead.
  • A larger CPU request is refused when the answer is “autoscaler”: a bigger request would pay for the peak on every pod all day.
  • A new HPA waits for a person’s approval, unless a grant that ticks Add an HPA covers the workload. Then it may be created on its own, only when the answer is “autoscaler”, with minReplicas set to the pods running now and maxReplicas never above the grant’s Most replicas, which the grant must set.
  • A larger CPU request waits for a person’s approval, unless a grant that ticks Raise a CPU request covers the workload. Then it may be raised on its own, only when the answer is “request” or “both”, to at most the measured 95th percentile plus 20%, at most 4 times today’s request, and never above the grant’s CPU ceiling.
  • A new HPA is verified by the HPA reporting it can scale and holding utilisation at or under its target (within 10%) for 5 minutes, within 30 minutes. A daily traffic curve may not reach a scale-out inside that time, so a scale-out event is not what is checked.

Raising an existing HPA’s maxReplicas by one step can run on its own under a one-fix grant.

Savings also raises sizes, but a different field: when a container presses on its CPU or memory limit, Rightsizing raises the limit. This page is about the CPU request and the replica count, decided from how load behaves. When managed savings and Nightshift’s CPU throttling fix would both raise the same limit on their own, the grants’ rate limit on that workload keeps them from both acting in the same window: whichever comes first acts.