Grants
By default nothing in Seylo changes your cluster without an approval. A grant relaxes that, narrowly: it says Seylo may make this kind of change to these workloads, by at most this much, during these hours, at most this often, until this date. Inside those limits Seylo acts on its own. Outside them, the change waits for a person.
What an approver signs is not a change. It is a bounded permission, and every decision taken under it, including every refusal, is recorded.
What uses a grant
Section titled “What uses a grant”| What | What a grant lets it do |
|---|---|
| Nightshift | The actions ticked in the grant’s Nightshift actions section, such as a rollback after a release, raising a CPU request or adding an HPA, each inside its own bound. See what Nightshift may do on its own |
| Nightshift’s and Seylo’s remediation changes to scale and resources | Changes to the dimensions the grant names - CPU or memory request or limit, replicas - inside its limits. For example, raising a throttled container’s CPU limit by 50% |
| APM coverage | Instrument a workload for tracing, inside the grant’s namespace and languages |
Managed savings has its own grant, signed when
an admin switches it on for a cluster. It is shown separately on
Automation, never used by anything else, and a grant you sign here never
covers a managed saving.
Sizing for savings is not done under a grant any more. Until 0.4.9 a separate engine lowered requests and replicas on its own under grants that named those dimensions. It is retired: sizing for savings is Rightsizing on the Savings tab, sent for approval, or applied by managed savings. A grant naming CPU request, memory request or replicas now covers only the Nightshift and remediation changes that use them.
If Operate → Automation says
Unattended changes are off on this deployment: grants are stored, and nothing uses them yet.,
nothing acts on its own under any grant on this Seylo deployment; every
change goes through approval.
Request a grant
Section titled “Request a grant”- Open Operate → Automation and pick the cluster.
- Choose New grant (
admin). Or, from a resource or replica change waiting on Approvals, choose Allow this kind to open the form pre-filled for that workload. See allow this kind of change. - Fill in the sections below. The form shows every problem at once, using the
same rules the server applies. A problem that is the same for several
dimensions is listed once. While any problem is left, Request approval
stays off, and the line beside it says why, for example
Fix the 2 problems listed above to request approval. - Read What the approver will read, then choose Request approval. The grant always becomes an approval request, even on a deployment where approvals are switched off for other changes.
- An
approveroradminapproves it on Approvals. One approval is enough, and an admin may approve their own grant.
Approving a grant changes nothing on the cluster. It creates the grant; each later action is recorded against the grant’s id. The grant is checked again at approval time, so one that sat in the queue past its expiry cannot be approved into existence:
This autoscale policy is no longer valid at approval time, so no grant was created: ...There is no way to edit a grant’s limits in place. Widening one is a new grant with a new approval.
What you set
Section titled “What you set”A cluster, and optionally a Namespace and a Workload. Empty means every
namespace, or every workload in scope. Names match exactly — there are no
patterns, because prod-* matching prod-experimental is precisely the
accident a scope exists to prevent. A workload with no namespace is refused:
it would match that name in every namespace.
When several grants cover the same workload, the narrowest decides: workload, then namespace, then cluster. A broader grant is never searched for a more permissive answer, and a suspended narrow grant is not bypassed by a live broad one.
Dimensions and limits
Section titled “Dimensions and limits”A grant may name five dimensions: CPU request, Memory request, CPU limit, Memory limit and Replicas. Tick the dimensions the grant permits. Anything not ticked is queued for a person. The limits are asked once, and apply to every ticked dimension:
| Field | Rule |
|---|---|
| Max increase per change % | Required, 1 to 50. The form starts at 20 |
| Max decrease per change % | Required, 1 to 50. The form starts at 10 |
To treat one dimension differently, tick Set a different limit, a floor or a ceiling for one dimension. A row then opens for each ticked dimension:
| Field | Rule |
|---|---|
| Max increase %, Max decrease % | Optional, 1 to 50. Empty means the shared value above |
| Floor, Ceiling | Optional. Millicores for CPU, bytes for memory, a count for replicas |
50% is a hard cap on any single unattended change, whatever a grant asks for. It is fixed in Seylo, not a setting. A bigger move happens as several changes, spaced by the rate limit and cooldown. A dimension listed without limits authorises nothing; its card says no limit: never acts. Increases and decreases are limited separately because they are different decisions — scaling up costs money, scaling down risks availability.
A grant may name Nightshift actions and no dimension.
Nightshift actions
Section titled “Nightshift actions”Each action covers one kind of change Nightshift may make on its own when its investigation of an alert proposes it, inside this grant’s scope, hours and rate limit. Anything not ticked waits for a person. There is no wildcard, and a grant signed before an action existed does not cover it.
| Action | Notes |
|---|---|
| Restart a Deployment | A rolling restart |
| Delete one controlled pod | Only a pod a controller will recreate |
| Roll back to the previous revision (any reason) | Never implied by anything else: tick it to grant it |
| Roll back when the new image cannot start (pull fails or name invalid) | Only when none of the release’s pods ever ran. See roll back an image that never started |
| Roll back a crash or stuck rollout to the last release Seylo saw run healthy (within 30 min of the release) | Only to a release Seylo recorded as succeeded. See roll back a crash or stuck rollout |
| Raise a CPU request to measured use + 20% (at most 4x, never above the CPU ceiling you set) | Ticking it shows CPU request ceiling (millicores), which is required: without it the action raises nothing. See raise a CPU request |
| Add an HPA where load splits across pods (min = today’s pods, max within your replica ceiling) | Ticking it shows Most replicas, which is required: the highest maxReplicas an HPA Nightshift creates may have. See add an HPA |
| Cordon a node, Uncordon a node | Covered only by a cluster-wide grant (no namespace) |
| Instrument a workload for tracing (one annotation; restarts its pods) | Adds one OpenTelemetry Operator annotation to a workload that serves traffic and sends no spans, and removes it if spans do not arrive or the pods fail. The grant must name its namespace. Tick the languages it covers - Java, Node.js, Python, .NET; never Go - and Workloads instrumented per day, 1 to 20. Never a workload Argo CD, Flux or Helm owns. See APM coverage |
| Restart a workload whose pods mount an older version of a Key Vault secret than the vault’s current one | See restart after a Key Vault secret rotated |
| The one-fix actions, from Let pods pull their image on | Each covers one Nightshift fix and nothing else, inside the bound its label states. See one-fix grants |
The two ceilings are kept on the grant, and shown beside the action wherever the grant is: on its card on Automation, on the request an approver signs, and on Nightshift → Autonomy. A grant that names Add an HPA and was signed before Most replicas was required says so on its card: only the Highest maxReplicas on Automation bounds it, and with neither it creates nothing.
The grant still holds the limits per dimension: What the approver will read lists each ticked dimension with its own increase and decrease.
Permitted window
Section titled “Permitted window”Days of the week and hour ranges, in a timezone — required, never
defaulted to UTC, because “not during business hours” is about someone’s local
clock. Choose it from the Timezone list: your browser’s zone comes first,
then a short list of common zones, then every zone, each shown with its
current offset from UTC, such as Asia/Kolkata (GMT+05:30). Hours run
0 to 24 with the start before the end. A window that crosses
midnight is two ranges, such as 22–24 and 0–6. An empty window means never.
Rate limit, confidence and expiry
Section titled “Rate limit, confidence and expiry”| Field | Rule |
|---|---|
| Max actions per window | At least 1, per workload and dimension |
| Window (hours) | Positive |
| Cooldown (minutes) | Zero or more. Time between two unattended changes to the same target, so the effect of one is seen before the next |
| Minimum baseline confidence | medium or high. Never lower: acting on evidence Seylo itself calls weak is what this design exists to prevent |
| Duration (days) | At most 90. A grant always expires; renewing it is a deliberate decision |
| Notes | Carried into the record of every action the grant authorises |
How each decision is made
Section titled “How each decision is made”For a change to a dimension, the checks run in this order. The first one that fails decides.
- Pause all automation, for your organisation or for this cluster → refuse.
- No grant covers the workload → a person decides.
- The covering grant is suspended or expired → a person decides.
- Helm, Argo CD or Flux owns the workload, or ownership cannot be read →
a person decides. A reconciler would revert the change within minutes and
it would be proposed again — a churn loop. A Helm release would silently
discard it at the next
helm upgrade. - Baseline confidence below the grant’s minimum → a person decides.
- The dimension is not permitted, or has no limits → a person decides.
- The size of the change: more than the grant’s percentage or the 50% cap, below the floor, above the ceiling, or from a current value of 0 (no percentage exists) → a person decides.
- Outside the window → a person decides. A window that cannot be evaluated is treated as closed.
- 3 consecutive failed or reverted unattended changes on this target → refuse, and the grant suspends itself.
- Rate limit reached, or inside the cooldown → a person decides.
- A reduction whose usage pattern is not actionable — too flat, or too little hourly history → a person decides. Being wrong downward costs an incident, not a bill.
- Otherwise → act.
A Nightshift action is decided the same way for scope, window, rate limit, expiry and the pause, and then held to the action’s own bound.
An act is applied through the same engine as a human-approved change: the ownership check, a dry run, server-side apply and a read-back. A replica change applies to Deployments only.
A change no grant covers is not lost. Nightshift files it for approval, once
per workload and dimension, and the fix says so, for example
Sent for approval as ... Not applied unattended: .... The exact reasons are in
troubleshooting.
Suspend, resume and auto-suspension
Section titled “Suspend, resume and auto-suspension”- Suspend a grant (
releaser): it authorises nothing until resumed. Changes it would have covered wait for a person. - Resume it (
releaser): it is active again with its original limits and expiry. Resuming re-approves nothing. - Auto-suspension: after 3 consecutive unattended changes on one target
that failed or were reverted, the grant suspends itself so a person looks.
Something is fighting the change, and repeating it unattended will not win.
The grant’s card shows the count against 3, for example
2/3 failed.
A change counts as reverted when the write was accepted but reading it back showed something else, and as failed when the write failed. Neither counts as a success, and both use up a slot in the rate limit.
Pause all automation
Section titled “Pause all automation”Pause all automation at the top of Operate → Automation (releaser)
stops every grant on every cluster in your organisation at once, managed
savings included. See pause and limits.
Where decisions are recorded
Section titled “Where decisions are recorded”Every change a grant allowed, and every refusal, is a line in What Seylo did on its own on Automation: what changed, why, and which grant allowed it. See what Seylo did on its own. Each grant’s card also lists its own last decisions under Details, with how many changes it made.
Decisions that only waited for a person - not covered, outside the window, inside a cooldown - are not log lines. They are on the fix that was filed, or on Approvals.