Security and permissions
The permissions the KubeOn agent, hub and billing connections use, in clusters and in cloud accounts.
On this page
In the cluster
apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata: name: kubeon-agent-readerrules: - apiGroups: [""] resources: [nodes, pods, namespaces, persistentvolumes, persistentvolumeclaims, services] verbs: [get, list, watch] - apiGroups: [apps] resources: [deployments, statefulsets, daemonsets, replicasets] verbs: [get, list, watch] - apiGroups: [batch] resources: [jobs, cronjobs] verbs: [get, list, watch] - apiGroups: [storage.k8s.io] resources: [storageclasses] verbs: [get, list, watch] - apiGroups: [metrics.k8s.io] resources: [nodes, pods] verbs: [get, list] - apiGroups: [karpenter.sh] # NodePools, on Karpenter clusters resources: [nodepools, nodeclaims] verbs: [get, list, watch] - apiGroups: [autoscaling] resources: [horizontalpodautoscalers] verbs: [get, list, watch] - apiGroups: [argoproj.io] # where workloads are deployed from resources: [applications, rollouts] verbs: [get, list, watch] - apiGroups: [kustomize.toolkit.fluxcd.io, helm.toolkit.fluxcd.io, source.toolkit.fluxcd.io] resources: [kustomizations, helmreleases, gitrepositories] verbs: [get, list, watch]No write verbs, no Secrets, no ConfigMaps. The agent pod runs as uid 1000 with a read-only root filesystem, no privilege escalation and every capability dropped.
In the cluster: the KubeOn controller (opt-in)
Off by default. Its own service account and token, with one ClusterRole per feature you grant. See write access for what each feature holds.
In the cluster: the GPU optimizer (opt-in)
Only clusters where you install the GPU optimizer get these permissions. They are listed in full, with the reason for each rule, in install the GPU optimizer. The optimizer reads no Secrets except its own token, and edits ConfigMaps only in the GPU Operator's namespace.
AWS
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AthenaQuery", "Effect": "Allow", "Action": [ "athena:StartQueryExecution", "athena:GetQueryExecution", "athena:GetQueryResults", "athena:StopQueryExecution", "athena:GetWorkGroup" ], "Resource": "arn:aws:athena:us-east-1:111122223333:workgroup/primary" }, { "Sid": "GlueCatalogRead", "Effect": "Allow", "Action": [ "glue:GetDatabase", "glue:GetDatabases", "glue:GetTable", "glue:GetTables", "glue:GetPartition", "glue:GetPartitions" ], "Resource": [ "arn:aws:glue:us-east-1:111122223333:catalog", "arn:aws:glue:us-east-1:111122223333:database/cur_database", "arn:aws:glue:us-east-1:111122223333:table/cur_database/*" ] }, { "Sid": "CurBucketRead", "Effect": "Allow", "Action": ["s3:GetObject", "s3:ListBucket", "s3:GetBucketLocation"], "Resource": ["arn:aws:s3:::acme-cur-exports", "arn:aws:s3:::acme-cur-exports/*"] }, { "Sid": "AthenaResultsReadWrite", "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:AbortMultipartUpload", "s3:ListBucket", "s3:GetBucketLocation" ], "Resource": ["arn:aws:s3:::acme-athena-results", "arn:aws:s3:::acme-athena-results/*"] }, { "Sid": "STSAccess", "Effect": "Allow", "Action": "sts:GetCallerIdentity", "Resource": "*" } ]}| Role | Permissions |
|---|---|
| Agent | None. It authenticates to KubeOn with its cluster token and needs no IAM role |
| Hub task role | Assume the billing role; Athena and Glue read; read the CUR bucket; read and write the Athena results bucket; read and delete snapshots; KMS on the platform key; its own secrets |
Azure
| Identity | Role assignments |
|---|---|
| KubeOn billing identity | Storage Blob Data Reader on the export storage account, Cost Management Reader on the scope |
| Agent | None. No workload identity or role assignment |
Google Cloud
| Service account | Roles |
|---|---|
| KubeOn billing | roles/bigquery.dataViewer on the export dataset, roles/bigquery.jobUser in the billing project |
| Agent | None. No service account or Workload Identity binding |
Encryption
Databases, caches, buckets and backups are encrypted at rest with UnitVane-managed keys, and every connection uses TLS 1.2 or later. Integration credentials are encrypted before they are stored.