Choose an RBAC posture
Two independent decisions control what the agent can do in your cluster:
- Scope — cluster-wide, or a
Rolein each of a named set of namespaces. - Posture — how much the agent may change within that scope.
Both are enforced by Kubernetes itself, not by Seylo. That is the point: the tier model in the portal can be argued with in a code review; an RBAC grant cannot be argued with at all.
Posture
Section titled “Posture”Pick a named posture rather than assembling individual toggles. The most common RBAC mistake here is a half-set combination nobody meant — browse enabled with writes left on, say.
| Posture | The agent can | Use when |
|---|---|---|
read-only |
See the cluster and change nothing in it | You want visibility first, or a cluster Seylo should never mutate |
deploy |
Read everything it needs, and create/update/patch the workload kinds a release touches. No delete anywhere | Most clusters. This is the product’s intended posture |
admin |
deploy, plus the destructive tier: delete and evict pods, cordon and drain nodes, delete workloads |
Someone is expected to remediate from the portal |
| (unset) | Whatever the individual toggles say | An existing install keeps exactly the Role it has |
read-only grants exactly one write, and it is not a cluster permission
Section titled “read-only grants exactly one write, and it is not a cluster permission”get/patch/update on the agent’s own credential Secret, narrowed by
name to seylo-agent-credential.
The agent has to persist its own rotated credential or it cannot stay connected. This is stated here rather than left to be discovered in a rendered Role.
Every posture can read one Secret of connection keys
Section titled “Every posture can read one Secret of connection keys”From agent 0.2.15, every posture may get one more Secret, narrowed by name to
seylo-connections in the agent’s own namespace (chart value
connections.secretName). It holds the keys you put there for
Grafana, Prometheus and Alertmanager connections
made through the agent. The agent reads a key by name for one request to
that tool’s Service and never sends it to Seylo. It cannot list Secrets or read
any other. If you never create seylo-connections, the rule matches nothing.
Every posture can read CustomResourceDefinitions
Section titled “Every posture can read CustomResourceDefinitions”From agent 0.2.14, the agent may get, list and watch
CustomResourceDefinitions in every posture, read-only included. A CRD is a
schema, never data: reading it tells Resources which kinds in your cluster
are custom. Reading the custom resources themselves still needs
rbac.genericRead or a write tier. Before 0.2.14, Resources showed
customresourcedefinitions.apiextensions.k8s.io is forbidden and guessed
which kinds were custom from their API group.
Every posture can read endpoints and StorageClasses
Section titled “Every posture can read endpoints and StorageClasses”From agent 0.2.16, every posture adds three read-only grants, get and list
only:
| Resource | API group | Where it is granted |
|---|---|---|
endpoints |
core ("") |
The ClusterRole, and the Role in each watched namespace |
endpointslices |
discovery.k8s.io |
The ClusterRole, and the Role in each watched namespace |
storageclasses |
storage.k8s.io |
The ClusterRole only |
They let Nightshift find a Service with no endpoints without the observability chart, and tell whether a nearly full volume’s StorageClass allows expansion before it proposes growing the volume. None of them carries a Secret value.
A StorageClass is cluster-scoped, and a Role cannot grant a cluster-scoped kind, so a namespace-scoped install cannot read it. There, the volume fix says the agent’s role may not read StorageClasses and that a cluster admin expands the claim by hand, rather than guessing whether the class allows it.
Every cluster-wide posture can read PriorityClasses
Section titled “Every cluster-wide posture can read PriorityClasses”From agent 0.2.18, the ClusterRole adds get and list on
scheduling.k8s.io priorityclasses. Nightshift reads them before it proposes
a capacity buffer: whether the
buffer’s own PriorityClass exists below every workload, and which class is the
cluster’s default. Seylo never creates or changes a PriorityClass. A
PriorityClass is cluster-scoped, so a namespace-scoped install cannot read it,
and the buffer is refused there.
Every cluster-wide posture can read your node autoscaler
Section titled “Every cluster-wide posture can read your node autoscaler”From agent 0.2.21, the ClusterRole adds get, list and watch on
Karpenter NodePools and NodeClaims (karpenter.sh), AKS node
auto-provisioning AKSNodeClasses (karpenter.azure.com), EKS EC2NodeClasses
(karpenter.k8s.aws), EKS Auto Mode node classes (eks.amazonaws.com) and GKE
ComputeClasses (cloud.google.com). The
Savings tab reads them to say
which autoscaler manages your nodes. None holds a Secret value. An EKS node
class carries a node bootstrap script; Seylo reads named fields only and never
returns it. They are cluster-scoped, so a namespace-scoped install cannot read
them.
Every posture can read budgets, Jobs, quotas and certificates
Section titled “Every posture can read budgets, Jobs, quotas and certificates”From agent 0.2.22, every posture adds read-only grants, get and list
only, including read-only:
| Resource | API group | Where it is granted |
|---|---|---|
poddisruptionbudgets |
policy |
The ClusterRole, and the Role in each watched namespace |
jobs, cronjobs |
batch |
The ClusterRole, and the Role in each watched namespace |
resourcequotas |
core ("") |
The ClusterRole, and the Role in each watched namespace |
certificates, certificaterequests, issuers |
cert-manager.io |
The ClusterRole, and the Role in each watched namespace |
clusterissuers |
cert-manager.io |
The ClusterRole only |
orders, challenges |
acme.cert-manager.io |
The ClusterRole, and the Role in each watched namespace |
Nightshift reads them to find a PodDisruptionBudget that blocks a drain, a CronJob that missed its schedule or keeps failing, a namespace at its ResourceQuota, and a certificate that is not ready. None of them carries a Secret value, and the agent writes none of them with these grants. cert-manager’s objects are read only where cert-manager is installed.
Before 0.2.22, Jobs, CronJobs and quotas were readable only where tier 1 is on (the default). On an older agent, the PodDisruptionBudget and certificate reads are refused: Nightshift says so, and names the upgrade. See upgrade the agent.
No posture ever grants RBAC write
Section titled “No posture ever grants RBAC write”An agent that can edit Roles can widen its own permissions. That is a decision
no preset should make on your behalf, so it is never included in a posture —
admin included. If you need it, you turn it on deliberately, and you should
read the warning below first.
What is never granted, at any setting
Section titled “What is never granted, at any setting”pods/attachappears in no grant this chart can produce, and there is no interactive shell.pods/execappears in no posture. The one way to add it is the separate chart valuesre.exec.enabled=true, off by default, which lets Nightshift, Seylo’s AI SRE, run a fixed list of read-only commands — and only once a Seyloadminalso opts the cluster in. See commands in containers.
Secret values are never returned by Seylo, at any setting. That guarantee is enforced in code, not by RBAC, because the default grant does include Secrets. See the next section.
Secrets in the default grant
Section titled “Secrets in the default grant”Tier 1 (rbac.write.tier1.enabled, on by default) grants get, list,
create, update, patch and delete on core secrets.
In the default cluster scope that is every Secret in every namespace,
kube-system included, not only the namespaces Seylo manages. The rule is
part of a ClusterRole bound cluster-wide. With rbac.scope=namespaced it is a
Role in each watched namespace and nowhere else.
Seylo uses it to:
- set and rotate Secret values;
- apply Secrets in releases, provisioning and a namespace copy;
- carry a Secret value over inside your cluster during a copy;
- show the key names of Secrets in the environment view.
Values are never shown. The agent strips them before anything is serialised onto the wire, and the server strips them again. See how secrets are handled. But the agent’s token itself can read a Secret, so anyone able to replace the agent’s image could read every Secret that token reaches.
For a regulated cluster, install namespace-scoped, so the token reaches Secrets only in the namespaces you list:
--set rbac.scope=namespaced --set 'watchNamespaces={prod,staging}'Namespace scoping has one real consequence for metrics. See cluster-wide vs namespace-scoped.
To give the token no Secret rule from tier 1 at all, set
rbac.write.tier1.enabled=false, or choose the read-only or deploy
posture, which leave tier 1 off. Setting, rotating, applying and carrying over
Secrets then stop working, and you manage Secrets outside Seylo.
The two grants to think hardest about
Section titled “The two grants to think hardest about”Both are off by default.
Destructive (tier 3). Permits deleting PersistentVolumeClaims — which
deletes the real disk when the bound volume’s reclaim policy is Delete —
PersistentVolumes, StorageClasses, Namespaces (cascading to everything inside),
CustomResourceDefinitions (deleting every custom resource of that type) and
VolumeSnapshots.
The portal additionally requires the object’s exact name to be typed and the consequences acknowledged, and shows the reclaim policy, the cascade contents and the custom-resource count first. Those are guardrails on someone who already holds the permission. Leaving it off is the control that does not depend on anybody reading a dialog.
Turn it on with rbac.write.destructive.enabled=true, or with the admin
posture, which includes it. One Nightshift fix needs it: recreating a
volume claim whose StorageClass does not exist, so that a Pending pod can
start. That fix touches only a claim that never bound and holds no data,
always waits for an approver who types the claim’s name, and never runs
unattended. Without the destructive tier, the delete is refused, the old claim
is left exactly as it was, and the fix names this value. The default role
creates and patches claims but never deletes them, and that default does not
change. See
a claim whose StorageClass does not exist.
RBAC (tier 4). With it the agent can write Roles, RoleBindings, ClusterRoles and ClusterRoleBindings — including ones that widen its own permissions.
Node autoscaler (rbac.write.autoscaler.enabled). From agent 0.2.21,
off by default, and in no posture, admin included: whether Seylo may change
which machines your cluster buys is a choice you make for
managed savings, lever by lever, not one a preset
makes for you. It grants create, update and patch on Karpenter
NodePools, AKSNodeClasses, EC2NodeClasses and ComputeClasses. These objects
decide spot or on-demand, machine families, CPU and memory ceilings and how
eagerly nodes are consolidated, so a wrong value can leave pods Pending or
put workloads on spot capacity. It never grants delete: deleting a
NodePool drains every node it created at once, and the agent refuses such a
delete itself, whatever else allows it. No cloud credential is involved; your
cluster’s own autoscaler creates and removes the machines.
Custom resources: an explicit list
Section titled “Custom resources: an explicit list”The agent writes a custom resource only if you name it.
rbac.write.tier1.customResources is a list of {apiGroup, resources}
entries, empty by default. Empty means the agent writes no custom
resources.
Each entry grants get, list, watch, create, update, patch and
delete on exactly the listed resources of exactly that group. The chart’s own
values file carries this recommended starting list, for teams that edit these
kinds through Seylo:
rbac: write: tier1: customResources: - apiGroup: cert-manager.io resources: [certificates] - apiGroup: external-secrets.io resources: [externalsecrets] - apiGroup: monitoring.coreos.com resources: [servicemonitors, podmonitors] - apiGroup: keda.sh resources: [scaledobjects] - apiGroup: networking.istio.io resources: [virtualservices] - apiGroup: argoproj.io resources: [rollouts]Keep only the entries you use.
SecretProviderClasses are not on this list. Their create and patch is a rule of its own in tier 1. See what a namespace copy needs.
What the chart refuses
Section titled “What the chart refuses”helm install, helm upgrade and helm template refuse to render, so nothing
reaches the cluster, when the list contains:
| Entry | Helm’s error includes |
|---|---|
The old value true |
rbac.write.tier1.customResources is now a LIST of {apiGroup, resources[]} entries, not true. |
apiGroup: "" (the core group) |
rbac.write.tier1.customResources[0].apiGroup is empty. The empty group is the CORE group (pods, secrets, namespaces...), whose kinds the tiers grant narrowly. |
apiGroup: "*" |
rbac.write.tier1.customResources[0].apiGroup is "*": "*" matches every API group, the core group and RBAC included, which is the wildcard this list replaced. |
A built-in group, such as apps |
rbac.write.tier1.customResources[0].apiGroup "apps" is a built-in Kubernetes group. |
"*" in resources |
rbac.write.tier1.customResources[0].resources contains "*" for cert-manager.io: "*" (or an empty name) would cover every resource of the group, including ones installed later. |
An entry with no resources |
rbac.write.tier1.customResources[0].resources must be a non-empty list of resource plurals of cert-manager.io, such as [certificates]. |
The [0] is the position of the offending entry, and the group is yours. Each
message goes on to say what to write instead. For true it ends:
Name what the agent may write, for example in a values file: rbac.write.tier1.customResources: [{apiGroup: cert-manager.io, resources: [certificates]}]; or --set rbac.write.tier1.customResources=null to grant none.A "*" anywhere in the group name is refused, not only on its own.
The built-in groups refused are apps, batch, rbac.authorization.k8s.io,
apiextensions.k8s.io, admissionregistration.k8s.io, storage.k8s.io,
certificates.k8s.io, coordination.k8s.io, node.k8s.io,
scheduling.k8s.io, policy, networking.k8s.io, autoscaling,
discovery.k8s.io, flowcontrol.apiserver.k8s.io, apiregistration.k8s.io,
events.k8s.io, metrics.k8s.io, authentication.k8s.io,
authorization.k8s.io, resource.k8s.io, internal.apiserver.k8s.io,
storagemigration.k8s.io, core and v1. Their kinds are granted, narrowly,
by the tiers.
The list is checked even while tier 1 is off, so a refused entry cannot wait in a values file for someone to switch tier 1 back on.
When Seylo refuses a custom-resource write
Section titled “When Seylo refuses a custom-resource write”A write to a custom resource that is not on the list is refused. The message
begins PERMISSION MISSING, not an empty or failed result:, names the verb and
the resource, and names the exact entry to add. For a KEDA ScaledObject it
includes:
scaledobjects.keda.sh is a custom resource, which the agent may write only when it is listed: add {apiGroup: keda.sh, resources: [scaledobjects]} to rbac.write.tier1.customResources in the agent's Helm values (helm upgrade --reuse-values -f <values file>). A "*" entry is refused; name the resource.Add that entry to your values file and run the upgrade. Use
helm upgrade seylo-agent oci://ghcr.io/seylo-inc/charts/seylo-agent --namespace seylo-agent --reset-then-reuse-values -f <values file>
rather than the --reuse-values the message names: --reuse-values also keeps
the old chart’s defaults, which can make an upgrade fail. On Helm older than
3.14, see upgrade the agent. Nothing was changed by
the refused write.
Cluster-wide vs namespace-scoped
Section titled “Cluster-wide vs namespace-scoped”| Cluster-wide | Namespace-scoped | |
|---|---|---|
| Binding | A ClusterRole + ClusterRoleBinding for the base grant and for each tier that is on |
A Role + RoleBinding for the base grant and each tier that is on, in each managed namespace |
| Blast radius | Every namespace in the cluster | Exactly the listed namespaces, enforced by Kubernetes |
| Namespace discovery | The agent lists namespaces itself | Impossible — watchNamespaces is required |
| Create namespace / namespace wizard | Works | Does not work |
| Auditing it yourself | Read the agent’s ClusterRoles, one per tier: kubectl get clusterrole -l app.kubernetes.io/name=seylo-agent |
kubectl get rolebinding -A | grep agent lists the complete reachable set |
| Choose it when | You own the cluster and want Seylo to onboard namespaces | Someone else owns the cluster and wants a hard, provable boundary |
Seylo additionally enforces the scope itself: a command naming a namespace outside the watched list is refused before any API call. So even a compromised Seylo server cannot widen the blast radius beyond what you configured — defence in depth on top of Kubernetes RBAC.
A namespace list on a cluster-wide install
Section titled “A namespace list on a cluster-wide install”A cluster-wide install has no namespace list by default. Seylo’s namespace
policy, which you change in the portal, then decides where Seylo acts. A new or
copied namespace needs no helm upgrade.
You can still fix a list into a cluster-wide install. On the cluster’s Agent
details page, tick
Also limit the agent to a fixed list of namespaces, inside the cluster, or
set watchNamespaces with Helm. The agent intersects what Seylo asks for with
that list. Seylo cannot widen the list remotely, by design. A mistaken or
compromised Seylo server can narrow what the agent does, but never extend it
past what you installed.
The trade-off is a helm upgrade for every namespace outside the list. See
a fixed namespace list on a cluster-wide install.
From agent 0.2.6, the agent reads each namespace in its list and reports only the ones that actually exist. A listed namespace that does not exist yet is not shown as present. A namespace copy creates it, or you can remove it from the list. Where the agent’s grant cannot read a Namespace object, as with a namespace-scoped install, it still reports the name, and Seylo treats it as unknown rather than present.
The one real consequence of namespace scoping
Section titled “The one real consequence of namespace scoping”A namespace-scoped install cannot read Nodes, so the agent cannot scrape the
kubelet itself. Without a metrics store in the cluster, the default kubelet
metrics source therefore collects nothing for that cluster.
Node is a cluster-scoped object. A Role cannot grant nodes, nodes/proxy
or nodes/metrics at all — not as a configuration choice, but because
Kubernetes RBAC does not permit a namespaced Role to reference a cluster-scoped
resource. No amount of editing the manifest changes this.
Still works, in full: inventory, the dashboard matrix, drift and environment comparison, history, releases, image patches, rollout restarts, manifest edits, ConfigMap and env edits, rollout status and follow-through, dry-run validation, approvals, audit, scheduled deployments, and pod-failure alerts (those come from events and pod status, both namespaced).
Does not work without a metrics store: node-proxy metrics — so no CPU, memory, network I/O or CPU throttling figures, and therefore no history and no right-sizing recommendations. These are reported as unknown, never as zero; the Insights page names the reason rather than drawing an empty chart.
Also unavailable, whatever you install: creating and listing namespaces, and the namespace wizard.
What to do instead, in order of preference
Section titled “What to do instead, in order of preference”-
Install the
seylo-observabilitychart and run an agent later than 0.2.4. On the defaultkubeletsource, such an agent reads usage from the chart’s metrics store instead of scraping the kubelet, so a namespace-scoped agent gets CPU, memory, network and throttling, with history, and its ownRolestays exactly as narrow as it was. The store still answers only for the agent’s namespaces. Restart counts and OOM kills come from the Kubernetes API as before. Agents 0.2.4 and earlier keep scraping the kubelet, so they gain nothing from the store here. Admin → Integrations → Observability stores shows the install command.The trade-off: the chart’s metrics store scrapes the kubelet with its own read-only
ClusterRoleon nodes. The agent does not gain that permission, but the cluster owner has to install the chart, and it holds cluster-wide read on nodes. The chart is also what managed vCPU is measured from. -
Set the cluster’s metrics source to
prometheusand give the agent an in-cluster Prometheus URL. Best coverage — CPU, memory, network and throttling with history — and it needs no cluster-scoped RBAC. The agent scrapes it from inside your cluster; Seylo never connects to it. -
Set the source to
metrics-server. Works with namespaced RBAC for pod metrics, so CPU and memory work. There is no network I/O or throttling from this source for anyone, namespaced or not. -
Grant the cluster-scoped variant, if the cluster owner is willing. This is the only way for the agent to scrape the kubelet itself, and it is also what the namespace wizard needs.
-
Accept metrics-free management. Everything in the “still works” list is unaffected. This is a perfectly reasonable posture for a cluster whose owner wants a provable, namespace-shaped boundary.
The chart prints all of this in its helm install output when you choose
namespace scoping.
Browsing arbitrary resource kinds
Section titled “Browsing arbitrary resource kinds”The base grant covers only what releases, environment edits, restarts and metrics need. Browsing arbitrary kinds — StatefulSets, DaemonSets, CronJobs, Jobs and any CRD — needs broad read, so it is a deliberate opt-in, off by default. With it off, the Resources page says exactly which permission is missing rather than showing an empty list.
Kubernetes RBAC has no deny rule and cannot express “everything except secrets”, so there are only two honest choices:
| Mode | What it grants | Trade-off |
|---|---|---|
wildcard (default) |
get/list/watch on everything, including CRDs installed later |
Includes secrets, in every namespace the binding reaches. Seylo’s refusal to expose values is enforced in code, not by RBAC |
enumerated |
The core group without secrets, plus the API groups you list |
This block adds no Secret read. Cost: a CRD group must be listed before it can be browsed |
Both modes are read-only. No write verb is ever granted by this block.
What a namespace copy needs
Section titled “What a namespace copy needs”A namespace copy from an agent-managed cluster reads and writes through the agent. From agent 0.2.6, the default grants cover everything a copy writes:
- tier 1 writes Deployments, StatefulSets, DaemonSets, CronJobs, Jobs, Services, Ingresses, NetworkPolicies, PodDisruptionBudgets, ConfigMaps, Secrets, ServiceAccounts and HPAs;
- the base grant reads SecretProviderClasses
(
secrets-store.csi.x-k8s.io) and PersistentVolumeClaims, and tier 1 creates and patches them, through a rule of its own. It never deletes either. Deleting a claim can delete the real disk, so that stays in the destructive tier.
These are two named kinds, not a broad grant. rbac.genericRead stays the
separate opt-in it was, and a copy does not need it. Nor does it need
rbac.write.destructive or anything in customResources.
Two cases still stop a copy that mounts a SecretProviderClass or a claim:
| Case | What happens | Fix |
|---|---|---|
| An agent older than 0.2.6 | Its default grant cannot read either kind, so the plan blocks the workload and names the upgrade | Upgrade the agent to 0.2.6 |
Posture read-only or deploy, or rbac.write.tier1.enabled=false |
The agent reads them but cannot create them, because tier 1 is off | Turn tier 1 on, create the object in the target yourself, or skip the workload |
On a copy between two agent-managed clusters, the target’s agent does the writing, so the target needs 0.2.6 too.
When a permission is missing
Section titled “When a permission is missing”A write the agent cannot perform is not refused opaquely. The error names the missing verb, the resource, and the exact setting that turns it on. Seylo also reads the agent’s real grant back and shows it in the portal, so you can see what the agent may do before something is refused rather than after.