User manual

This user manual describes how to install and use the Mondoo Operator.

User manual

Upgrading from Previous Versions

When upgrading from operator versions prior to v12.x, the following changes apply:

Automatic Cleanup: The operator automatically cleans up resources from previous versions:

ScanAPI resources: The ScanAPI Deployment, Service, and token Secret are automatically removed. The new architecture uses direct CronJob-based scanning with cnspec.
Admission webhook resources: If you had admission webhooks configured, the ValidatingWebhookConfiguration, webhook Deployment, Service, and TLS Secret are automatically removed. See admission-migration-guide.md for details.

No action required: Existing MondooAuditConfig resources continue to work. The operator will transition your scanning workloads to the new CronJob-based architecture automatically.

Resource Watcher Changes: If you have resourceWatcher.enable: true:

New rate limiting: A minimum scan interval of 2 minutes is now enforced by default to prevent excessive scanning.
High-priority resources by default: The watcher now only monitors Deployments, DaemonSets, StatefulSets, and ReplicaSets by default (previously watched all resources including Pods and Jobs). To restore the previous behavior, set watchAllResources: true in your configuration.

Configuring the Operator

For cluster-wide settings like proxies, registry mirrors, and metrics, see the MondooOperatorConfig Guide.

Mondoo Operator Installation

Install the Mondoo Operator using kubectl, Helm, or Operator Lifecycle Manager.

Installing with kubectl

Follow this step to set up the Mondoo Operator using kubectl and a manifest file.

Precondition: kubectl with cluster admin access

To install with kubectl, apply the operator manifests:

kubectl apply -f https://github.com/mondoohq/mondoo-operator/releases/latest/download/mondoo-operator-manifests.yaml

curl -sSL https://github.com/mondoohq/mondoo-operator/releases/latest/download/mondoo-operator-manifests.yaml > mondoo-operator-manifests.yaml
kubectl apply -f mondoo-operator-manifests.yaml

Installing with Helm

Follow these steps to set up the Mondoo Operator using Helm.

Preconditions:

kubectl with cluster admin access
helm 3

Add the Helm repo:

helm repo add mondoo https://mondoohq.github.io/mondoo-operator
helm repo update

Deploy the operator using Helm:

helm install mondoo-operator mondoo/mondoo-operator --namespace mondoo-operator --create-namespace

Customization

In case you set the Chart name to a different name, this will break parts of the operator, unless you also set:

fullnameOverride=mondoo-operator

Installing with Operator Lifecycle Manager (OLM)

Follow these steps to set up the Mondoo Operator using Operator Lifecycle Manager (OLM):

Preconditions:

kubectl with cluster admin access
operator-lifecycle-manager installed in the cluster (see the OLM QuickStart)
operator-sdk installed locally

Verify that operator-lifecycle-manager is up:

operator-sdk olm status | echo $?
0
INFO[0000] Fetching CRDs for version "v0.20.0"
INFO[0000] Fetching resources for resolved version "v0.20.0"
INFO[0001] Successfully got OLM status for version "v0.20.0"

Install the Mondoo Operator bundle:

kubectl create namespace mondoo-operator
operator-sdk run bundle ghcr.io/mondoohq/mondoo-operator-bundle:latest --namespace=mondoo-operator

Verify that the operator is properly installed:
```
kubectl get csv -n operators
```

Configuring the Mondoo Secret

Follow these steps to configure the Mondoo Secret:

Create a new Mondoo service account to report assessments to Mondoo Platform.

Store the service account json into a local file creds.json. The creds.json file should look like this:

{
  "mrn": "//agents.api.mondoo.app/spaces/<space name>/serviceaccounts/<Key ID>",
  "space_mrn": "//captain.api.mondoo.app/spaces/<space name>",
  "private_key": "-----BEGIN PRIVATE KEY-----\n....\n-----END PRIVATE KEY-----\n",
  "certificate": "-----BEGIN CERTIFICATE-----\n....\n-----END CERTIFICATE-----\n",
  "api_endpoint": "https://api.mondoo.com"
}

Store the service account as a Secret in the Mondoo namespace:

kubectl create secret generic mondoo-client --namespace mondoo-operator --from-file=config=creds.json

Creating a MondooAuditConfig

Once the Secret is configured, configure the operator to define the scan targets:

Create mondoo-config.yaml:

apiVersion: k8s.mondoo.com/v1alpha2
kind: MondooAuditConfig
metadata:
  name: mondoo-client
  namespace: mondoo-operator
spec:
  mondooCredsSecretRef:
    name: mondoo-client
  # Optional: route assets to a specific space (useful with org-level Service Accounts)
  # spaceId: "your-space-1234"
  kubernetesResources:
    enable: true
  nodes:
    enable: true

Apply the configuration:
```
kubectl apply -f mondoo-config.yaml
```

Filter Kubernetes objects based on namespace

To exclude specific namespaces add this to your MondooAuditConfig:

...
spec:
...
  filtering:
    namespaces:
      exclude:
        - kube-system

When you only want to scan specific namespaces:

...
spec:
...
  filtering:
    namespaces:
      include:
        - app1
        - backend2
        - ...

Scanning External Clusters

The Mondoo Operator can scan remote Kubernetes clusters from a central installation. This is useful for:

Scanning clusters where you cannot or do not want to install the operator
Centralizing security scanning in a management cluster
Scanning development or staging clusters from a single location

Creating a kubeconfig Secret

First, create a kubeconfig file that has access to the remote cluster. The kubeconfig should have read-only access to the Kubernetes API resources you want to scan.

Create a kubeconfig file for the remote cluster:

# Example: Export kubeconfig from your current context
kubectl config view --minify --flatten > remote-kubeconfig.yaml

Create a Secret containing the kubeconfig:

kubectl create secret generic prod-kubeconfig \
  --namespace mondoo-operator \
  --from-file=kubeconfig=remote-kubeconfig.yaml

Note: The Secret must have a key named kubeconfig containing the kubeconfig content.

Configuring external cluster scanning

Add the externalClusters field to your MondooAuditConfig:

apiVersion: k8s.mondoo.com/v1alpha2
kind: MondooAuditConfig
metadata:
  name: mondoo-client
  namespace: mondoo-operator
spec:
  mondooCredsSecretRef:
    name: mondoo-client
  kubernetesResources:
    enable: true  # Scan local cluster (optional)
    externalClusters:
      - name: production
        kubeconfigSecretRef:
          name: prod-kubeconfig
      - name: staging
        kubeconfigSecretRef:
          name: staging-kubeconfig

Each external cluster will have its own CronJob created with the appropriate kubeconfig mounted.

Per-cluster configuration

You can customize settings for each external cluster:

externalClusters:
  - name: production
    kubeconfigSecretRef:
      name: prod-kubeconfig
    # Custom schedule for this cluster
    schedule: "0 */2 * * *"  # Every 2 hours
    # Cluster-specific namespace filtering
    filtering:
      namespaces:
        exclude:
          - kube-system
          - monitoring
    # Enable container image scanning for this cluster
    containerImageScanning: true
    # Private registry credentials for this cluster
    privateRegistriesPullSecretRef:
      name: prod-registry-creds

Configuration options:

Field	Description
`name`	Unique identifier for the cluster (used in CronJob names)
`kubeconfigSecretRef`	Reference to Secret containing kubeconfig
`schedule`	Override the default scan schedule (cron format)
`filtering`	Namespace include/exclude specific to this cluster
`containerImageScanning`	Enable container image scanning for this cluster
`privateRegistriesPullSecretRef`	Registry credentials for private images

Authentication methods

The operator supports five authentication methods for external clusters. Choose the one that best fits your security requirements:

1. Kubeconfig (most flexible)

Use a kubeconfig file stored in a Secret. This works with any cluster and authentication method supported by kubectl.

externalClusters:
  - name: production
    kubeconfigSecretRef:
      name: prod-kubeconfig

2. Service Account Token

Use a Kubernetes service account token with CA certificate. Simpler than a full kubeconfig.

externalClusters:
  - name: production
    serviceAccountAuth:
      server: "https://prod-cluster.example.com:6443"
      credentialsSecretRef:
        name: prod-sa-credentials  # Secret with "token" and "ca.crt" keys

Create the credentials Secret:

kubectl create secret generic prod-sa-credentials \
  --namespace mondoo-operator \
  --from-file=token=./token \
  --from-file=ca.crt=./ca.crt

3. Workload Identity Federation (cloud-native)

Use cloud-native identity federation with no static credentials. Supports GKE, EKS, and AKS.

GKE example:

externalClusters:
  - name: gke-prod
    workloadIdentity:
      provider: gke
      gke:
        projectId: my-gcp-project
        clusterName: production-cluster
        clusterLocation: us-central1-a
        googleServiceAccount: scanner@my-gcp-project.iam.gserviceaccount.com

EKS example:

externalClusters:
  - name: eks-prod
    workloadIdentity:
      provider: eks
      eks:
        region: us-west-2
        clusterName: production-cluster
        roleArn: arn:aws:iam::123456789012:role/MondooScannerRole

AKS example:

externalClusters:
  - name: aks-prod
    workloadIdentity:
      provider: aks
      aks:
        subscriptionId: 12345678-1234-1234-1234-123456789012
        resourceGroup: my-resource-group
        clusterName: production-cluster
        clientId: abcdef12-3456-7890-abcd-ef1234567890
        tenantId: fedcba98-7654-3210-fedc-ba9876543210

4. SPIFFE/SPIRE (zero-trust)

Use SPIFFE/SPIRE for zero-trust authentication with auto-rotating X.509 certificates.

externalClusters:
  - name: spiffe-cluster
    spiffeAuth:
      server: "https://remote-cluster.example.com:6443"
      trustBundleSecretRef:
        name: remote-cluster-ca  # Secret with "ca.crt" key
      # Optional: custom socket path (default: /run/spire/sockets/agent.sock)
      # socketPath: "/run/spire/sockets/agent.sock"

Important: HostPath permissions required

SPIFFE authentication requires mounting the SPIRE agent socket from the host filesystem using a HostPath volume. This may require:

Pod Security Admission: Configure the namespace with a policy that allows HostPath volumes

Pod Security Policy (deprecated): Allow HostPath in the PSP

OpenShift SCC: Use an SCC that permits HostPath mounts

Ensure your cluster's security policies allow HostPath volumes for the mondoo-operator namespace before using SPIFFE authentication.

Note: Certificate TTL consideration

SPIFFE certificates are fetched once at the start of each scan job and are not rotated during the scan. SPIFFE SVIDs typically have a 1-hour TTL by default. For most K8s resource scans that complete within minutes, this is sufficient. If your scans consistently exceed the SVID TTL, consider increasing the TTL in your SPIRE server configuration or using a different authentication method.

5. HashiCorp Vault (dynamic credentials)

Use HashiCorp Vault's Kubernetes secrets engine to dynamically generate short-lived service account tokens for scanning external clusters. This avoids managing static credentials.

Prerequisites:

Vault server with Kubernetes auth method configured for the operator's cluster
Vault Kubernetes secrets engine configured for the target cluster
A Vault role that allows the operator's pod service account to authenticate
A Vault credentials role that generates service account tokens on the target cluster

externalClusters:
  - name: vault-cluster
    vaultAuth:
      server: "https://target-cluster.example.com:6443"
      vaultAddr: "https://vault.example.com:8200"
      authRole: mondoo-scanner        # Vault K8s auth role
      credsRole: target-cluster-creds  # Vault K8s secrets engine role
      # Optional fields:
      # authPath: "auth/kubernetes"    # Vault K8s auth mount path (default)
      # secretsPath: "kubernetes"      # Vault K8s secrets engine mount (default)
      # kubernetesNamespace: scanning  # Target namespace for generated token
      # ttl: "1h"                      # Requested token TTL
      # caCertSecretRef:               # Secret with Vault CA cert (ca.crt key)
      #   name: vault-ca-cert
      # targetCACertSecretRef:         # Secret with target cluster CA cert (ca.crt key)
      #   name: target-ca-cert

Note: Credential refresh timing

When VaultAuth is configured, the operator automatically reconciles at half the requested TTL (clamped between 10 minutes and 1 hour) to refresh credentials before they expire. For example, a ttl: "1h" results in a 30-minute refresh interval. If no TTL is specified, the operator defaults to refreshing every 30 minutes.

Container Image Scanning

The Mondoo Operator can scan container images running in your cluster for vulnerabilities and security issues.

Enable container image scanning in your MondooAuditConfig:

apiVersion: k8s.mondoo.com/v1alpha2
kind: MondooAuditConfig
metadata:
  name: mondoo-client
  namespace: mondoo-operator
spec:
  mondooCredsSecretRef:
    name: mondoo-client
  containers:
    enable: true
    schedule: "0 0 * * *"  # Daily at midnight

Warning

GKE Autopilot ephemeral storage limit: GKE Autopilot's Default compute class caps ephemeral storage at 2Gi per pod. Container image scanning pulls and analyzes images locally, which can exceed this limit when scanning large or numerous images. If scan jobs are evicted with Pod ephemeral local storage usage exceeds the total limit, switch to the Scale-Out or Balanced compute class by adding the cloud.google.com/compute-class annotation to the pod template, or use a GKE Standard cluster instead.

spec:
  containers:
    resources:
      requests:
        ephemeral-storage: "10Gi"
      limits:
        ephemeral-storage: "10Gi"

Creating a secret for private image scanning

To allow the Mondoo operator to scan private images, it needs access to image pull secrets for these private registries.

Single registry secret

Create a secret with the name mondoo-private-registries-secrets within the same namespace as your MondooAuditConfig:

kubectl create secret docker-registry mondoo-private-registries-secrets \
  --namespace mondoo-operator \
  --docker-server=registry.example.com \
  --docker-username=user \
  --docker-password=password

Then reference it in your MondooAuditConfig:

apiVersion: k8s.mondoo.com/v1alpha2
kind: MondooAuditConfig
spec:
  scanner:
    privateRegistriesPullSecretRef:
      name: mondoo-private-registries-secrets
  containers:
    enable: true

Multiple registry secrets

When you need credentials from multiple sources (e.g., managed by different teams or external secret operators like External Secrets or Vault), use privateRegistriesPullSecretRefs:

apiVersion: k8s.mondoo.com/v1alpha2
kind: MondooAuditConfig
spec:
  scanner:
    privateRegistriesPullSecretRefs:
      - name: team-a-registry-creds
      - name: team-b-registry-creds
      - name: external-secrets-managed-creds
  containers:
    enable: true

The operator automatically merges credentials from all specified secrets. If the same registry appears in multiple secrets, the last one takes precedence.

You can also use both fields together - all secrets will be merged:

spec:
  scanner:
    privateRegistriesPullSecretRef:
      name: default-registry-creds
    privateRegistriesPullSecretRefs:
      - name: additional-registry-creds

Default secret name

If no secret reference is specified, the operator looks for a secret named mondoo-private-registries-secrets by default.

RBAC considerations

The operator's default RBAC only allows reading secrets with specific names. If you use a custom secret name, extend RBAC so that the ServiceAccount mondoo-operator-k8s-resources-scanning has permission to read the secret.

You can find examples of creating Docker registry secrets here.

Private image scanning with Workload Identity Federation

Instead of managing static image pull secrets, you can use Workload Identity Federation (WIF) to authenticate to cloud container registries using the pod's cloud identity. The operator creates a dedicated Kubernetes ServiceAccount with the appropriate cloud annotations and adds an init container that obtains short-lived registry credentials at runtime.

This approach supports GCR, Artifact Registry, ECR, and ACR.

GKE (Artifact Registry / GCR):

spec:
  containers:
    enable: true
    workloadIdentity:
      provider: gke
      gke:
        projectId: my-gcp-project
        clusterName: my-cluster          # required by schema
        clusterLocation: us-central1     # required by schema
        googleServiceAccount: scanner@my-gcp-project.iam.gserviceaccount.com

Prerequisites:

GKE cluster with Workload Identity enabled
Google Service Account with roles/artifactregistry.reader (or roles/storage.objectViewer for GCR)
IAM policy binding the GSA to the Kubernetes ServiceAccount mondoo-client-cr-wif in the operator namespace

EKS (ECR):

spec:
  containers:
    enable: true
    workloadIdentity:
      provider: eks
      eks:
        region: us-west-2
        clusterName: my-cluster          # required by schema
        roleArn: arn:aws:iam::123456789012:role/MondooRegistryReader

Prerequisites:

EKS cluster with IRSA (IAM Roles for Service Accounts) enabled
IAM role with ecr:GetAuthorizationToken, ecr:BatchGetImage, ecr:GetDownloadUrlForLayer, and ecr:BatchCheckLayerAvailability permissions
IRSA trust policy allowing the Kubernetes ServiceAccount mondoo-client-cr-wif in the operator namespace

AKS (ACR):

spec:
  containers:
    enable: true
    workloadIdentity:
      provider: aks
      aks:
        subscriptionId: 12345678-1234-1234-1234-123456789012  # required by schema
        resourceGroup: my-resource-group                       # required by schema
        clusterName: my-cluster                                # required by schema
        clientId: abcdef12-3456-7890-abcd-ef1234567890
        tenantId: fedcba98-7654-3210-fedc-ba9876543210
        loginServer: myregistry.azurecr.io

Prerequisites:

AKS cluster with Workload Identity enabled
User-assigned managed identity with AcrPull role on the ACR
Federated identity credential trusting the Kubernetes ServiceAccount mondoo-client-cr-wif in the operator namespace

RBAC for the WIF ServiceAccount

The WIF ServiceAccount (mondoo-client-cr-wif) needs the same cluster read permissions as the default scanner ServiceAccount to discover container images running in the cluster. Add it to the scanning ClusterRoleBinding via the Helm chart:

helm upgrade mondoo-operator mondoo/mondoo-operator \
  --set "k8SResourcesScanning.extraClusterRoleBindingSubjects[0].name=mondoo-client-cr-wif" \
  --set "k8SResourcesScanning.extraClusterRoleBindingSubjects[0].namespace=mondoo-operator"

Or in values.yaml:

k8SResourcesScanning:
  extraClusterRoleBindingSubjects:
    - name: mondoo-client-cr-wif
      namespace: mondoo-operator

Routing assets to a specific space with `spaceId`

By default, scanned assets are sent to the space associated with the service account credentials. The spaceId field lets you override this, routing assets to any space the service account has access to. This is especially useful with org-level service accounts, which have access to all spaces in the organization.

Use cases

Multi-space scanning: A single operator with multiple MondooAuditConfig resources, each routing to a different space.
External cluster scanning: Scan remote clusters and route each cluster's assets to its own space.
Simplified credential management: Use one org-level service account instead of managing per-space service accounts.

Example: Two MondooAuditConfigs routing to different spaces

Create an org-level service account in the Mondoo Console and store it as a Secret:

kubectl create secret generic mondoo-client --namespace mondoo-operator --from-file=config=org-creds.json

Then create two MondooAuditConfig resources, each targeting a different space:

apiVersion: k8s.mondoo.com/v1alpha2
kind: MondooAuditConfig
metadata:
  name: mondoo-scanner
  namespace: mondoo-operator
spec:
  mondooCredsSecretRef:
    name: mondoo-client
  spaceId: "space-for-local-cluster"
  kubernetesResources:
    enable: true
  nodes:
    enable: true
---
apiVersion: k8s.mondoo.com/v1alpha2
kind: MondooAuditConfig
metadata:
  name: mondoo-target
  namespace: mondoo-operator
spec:
  mondooCredsSecretRef:
    name: mondoo-client
  spaceId: "space-for-remote-cluster"
  kubernetesResources:
    enable: true
    externalClusters:
      - name: remote-cluster
        kubeconfigSecretRef:
          name: remote-kubeconfig

Both configs share the same org-level service account but route assets to different spaces. The operator creates a derived config Secret for each MondooAuditConfig that has spaceId set, injecting the target space into the scanner configuration.

Installing Mondoo into multiple namespaces

You can deploy the mondoo client into multiple namespaces with just a single operator running inside the cluster.

We assume you already have the operator running inside the default namespace. Now you want to send the data from a different namespace into another Mondoo Space. To do so, follow these steps:

Create an additional Space in Mondoo
Create a Mondoo Service Account for this space
Create the new namespace in Kubernetes:

kubectl create namespace 2nd-namespace

Create a Kubernetes Service Account in this namespace:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: mondoo-operator-k8s-resources-scanning
  namespace: 2nd-namespace

Bind this Service Account to a Cluster Role which was created during the installation of the operator:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: k8s-resources-scanning
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: mondoo-operator-k8s-resources-scanning
subjects:
  - kind: ServiceAccount
    name: mondoo-operator-k8s-resources-scanning
    namespace: 2nd-namespace

Add the Mondoo Service Account as a secret to the namespace as described here
Create a MondooAuditConfig in 2nd-namespace as described here
(Optional) In case you want to separate which Kubernetes namespaces show up in which Mondoo Space, you can add filtering.

After some seconds, you should see that the operator picked up the new MondooAuditConfig and starts creating objects.

Adjust the scan interval

You can adjust the interval for scans triggered via a CronJob. Edit the MondooAuditConfig to adjust the interval:

kubectl -n mondoo-operator edit mondooauditconfigs.k8s.mondoo.com mondoo-client

  kubernetesResources:
    enable: true
    schedule: 41 * * * *

You can adjust the schedule for the following components:

Kubernetes Resources Scanning
Container Image Scanning
Node Scanning

Real-time Resource Watcher (Opt-in)

The Resource Watcher is an opt-in feature that provides real-time scanning of Kubernetes resources as they change, rather than waiting for the scheduled CronJob scans.

Breaking Change (v1.x+): If you previously had resourceWatcher.enable: true without specifying resourceTypes, the default watched resources changed from all resource types to only high-priority resources (Deployments, DaemonSets, StatefulSets, ReplicaSets). To restore the previous behavior, set watchAllResources: true.

Enabling the Resource Watcher

To enable the resource watcher, add the following to your MondooAuditConfig:

apiVersion: k8s.mondoo.com/v1alpha2
kind: MondooAuditConfig
metadata:
  name: mondoo-client
  namespace: mondoo-operator
spec:
  mondooCredsSecretRef:
    name: mondoo-client
  kubernetesResources:
    enable: true
    resourceWatcher:
      enable: true  # Opt-in: must be explicitly enabled

Default Behavior

When enabled with default settings, the resource watcher:

Watches only high-priority resources: Deployments, DaemonSets, StatefulSets, and ReplicaSets
Rate limits scans: Minimum 2 minutes between scans to prevent excessive scanning
Batches changes: 10-second debounce interval to batch rapid changes before scanning
Complements scheduled scans: The hourly CronJob continues to run for full cluster coverage

Configuration Options

Option	Default	Description
`enable`	`false`	Must be set to `true` to enable the resource watcher
`minimumScanInterval`	`2m`	Minimum time between scans (rate limit)
`debounceInterval`	`10s`	Time to wait after last change before triggering a scan
`watchAllResources`	`false`	When `true`, watches all resources including Pods, Jobs, CronJobs
`resourceTypes`	(auto)	Explicit list of resource types to watch (overrides `watchAllResources`)

Example: Custom Configuration

apiVersion: k8s.mondoo.com/v1alpha2
kind: MondooAuditConfig
metadata:
  name: mondoo-client
  namespace: mondoo-operator
spec:
  mondooCredsSecretRef:
    name: mondoo-client
  kubernetesResources:
    enable: true
    resourceWatcher:
      enable: true
      minimumScanInterval: 5m    # Scan at most every 5 minutes
      debounceInterval: 30s      # Wait 30s after last change
      watchAllResources: true    # Include Pods, Jobs, etc.

Example: Watch Specific Resource Types

apiVersion: k8s.mondoo.com/v1alpha2
kind: MondooAuditConfig
metadata:
  name: mondoo-client
  namespace: mondoo-operator
spec:
  mondooCredsSecretRef:
    name: mondoo-client
  kubernetesResources:
    enable: true
    resourceWatcher:
      enable: true
      resourceTypes:
        - deployments
        - services
        - ingresses

Why High-Priority Resources by Default?

By default, the resource watcher only monitors stable workload resources (Deployments, DaemonSets, StatefulSets, ReplicaSets) because:

Pods are ephemeral: Pod changes are frequent but already covered by scanning their parent resources
Jobs are transient: Job resources change constantly in active clusters
Reduces noise: Fewer unnecessary scans means less resource consumption

If you need to monitor all resources, set watchAllResources: true or specify explicit resourceTypes.

Configure resources for the operator and its components

Configure resources for the operator-controller

To change resources for the mondoo-operator-controller-manager, you need to change the Deployment:

kubectl -n mondoo-operator edit deployment mondoo-operator-controller-manager

The mondoo-operator-controller-manager has predefined requests and limits. Depending on your cluster size, something other than these might work better. During editing, search for the defaults in the manifest:

        resources:
          limits:
            cpu: 200m
            memory: 140Mi
          requests:
            cpu: 100m
            memory: 70Mi

Increase them as required.

Configure resources for the different scanning components

The mondoo-operator-controller-manager manages the other Deployments and CronJobs needed to scan your cluster. If the provided requests and limits do not match your cluster size, increase them as needed. For components, do not edit the Deployments or CronJobs directly. The mondoo-operator-controller-manager will revert your changes. Instead, edit the MondooAuditConfig:

kubectl -n mondoo-operator edit mondooauditconfigs.k8s.mondoo.com mondoo-client

You can change the resources for different components in the config:

spec:
...
  containers:
    resources: {}
...
  nodes:
    enable: true
    resources: {}
  scanner:
    image: {}
    privateRegistriesPullSecretRef: {}
    replicas: 1
    resources: {}
    serviceAccountName: mondoo-operator-k8s-resources-scanning
...

The resources field accepts the Kubernetes resource definitions:

spec:
...
  containers:
    resources: {}
...
  nodes:
    enable: true
    resources:
      limits:
        cpu: 1
        memory: 1Gi
      requests:
        cpu: 500m
        memory: 200Mi
  scanner:
    image: {}
    privateRegistriesPullSecretRef: {}
    replicas: 1
    resources: {}
    serviceAccountName: mondoo-operator-k8s-resources-scanning
...

After you saved the changes, the mondoo-operator-controller-manager will adjust the corresponding Deployment or CronJob.

Uninstalling the Mondoo operator

Before uninstalling the Mondoo operator, be sure to delete all MondooAuditConfig and MondooOperatorConfig objects. You can find any in your cluster by running:

kubectl get mondooauditconfigs.k8s.mondoo.com,mondoooperatorconfigs.k8s.mondoo.com -A

Uninstalling the operator with kubectl

Run:

kubectl delete -f https://github.com/mondoohq/mondoo-operator/releases/latest/download/mondoo-operator-manifests.yaml

Uninstalling the operator with Helm

Run:

helm uninstall mondoo-operator

Uninstalling the operator with Operator Lifecycle Manager (OLM)

Run:

operator-sdk olm uninstall mondoo-operator

Cleanup failed uninstalls

Under normal circumstances, the above steps clean up the complete operator installation. In rare cases, the namespace gets stuck in terminating state. This most likely happens because of finalizers.

This can happen when the operator-controller isn't running correctly during uninstall.

To clean up the namespace:

Find objects that are still present in the namespace:

kubectl api-resources --verbs=list --namespaced -o name | xargs -n 1 kubectl get --show-kind --ignore-not-found -n mondoo-operator

Check the remaining objects for finalizers, e.g., the MondooAuditConfig:

kubectl -n mondoo-operator get mondooauditconfigs.k8s.mondoo.com mondoo-client -o jsonpath='{.metadata.finalizers}'

Remove the finalizers from the object:

kubectl -n mondoo-operator patch --type=merge mondooauditconfigs.k8s.mondoo.com mondoo-client -p '{"metadata":{"finalizers":null}}'

The namespace should now automatically clean up after a short time.

FAQ

I do not see the service running, only the operator. What should I do?

Check that the CRD is properly registered with the operator:

kubectl get crd
NAME                           CREATED AT
mondooauditconfigs.k8s.mondoo.com   2022-01-14T14:07:28Z

Make sure a configuration for the Mondoo Client is deployed:

kubectl get mondooauditconfigs -A
NAME                  AGE
mondoo-client        2m44s

How do I edit an existing operator configuration?

Run:

kubectl edit  mondooauditconfigs -n mondoo-operator

Why is there a deployment marked as unschedulable?

For development testing, you can see the allocated resources for the Mondoo Client:

spec:
  mondooCredsSecretRef: mondoo-client
  scanner:
    resources:
      limits:
        cpu: 500m
        memory: 900Mi
      requests:
        cpu: 100m
        memory: 20Mi
  kubernetesResources:
    enable: true
  nodes:
    enable: true

Why are (some of) my nodes unscored?

In some cases a node scan can require more memory than initially allotted. You can check whether that is the case by running:

kubectl get pods -n mondoo-operator

Look for pods in the form of <mondooauditconfig-name>-node-<node-name>-hash. For example, if your MondooAuditConfig is called mondoo-client and you have a node called node01, you should be able to find a pod mondoo-client-node-node01-<hash>.

If the pod is crashing and restarting, it's most probably running out of memory and terminating. You can verify that by looking into the pod's status:

kubectl get pods -n mondoo-operator mondoo-client-node-node01-<hash> -o yaml

If you need to increase the resource limits for node scanning, change your MondooAuditConfig:

kubectl edit -n mondoo-operator mondooauditconfig mondoo-client

Search for the nodes: section and specify the new limits there. It should look like this:

spec:
  nodes:
    enable: true
    resources:
      limits:
        cpu: 200m
        memory: 200Mi

How can I trigger a new scan?

The operator runs a full cluster scan and node scans hourly. If you need to manually trigger those scans there are two options:

Option A: Create a job from the existing cron job

Locate the cron job you want to trigger:

kubectl get cronjobs -n mondoo-operator

Create a new job from the existing cron job. To trigger a new cluster scan, the command is:

kubectl create job -n mondoo-operator my-job --from=cronjob/mondoo-client-k8s-scan

A job called my-job starts the scan immediately.

Option B: Turn scanning off and then on again

Edit the MondooAuditConfig:

kubectl edit -n mondoo-operator mondooauditconfig mondoo-client

Disable scanning by changing enable: true to enable: false:

spec:
  kubernetesResources:
    enable: false
  nodes:
    enable: false

Make sure the scan cron jobs are deleted before proceeding:

kubectl get cronjobs -n mondoo-operator

Edit the MondooAuditConfig again and re-enable scanning:

spec:
  kubernetesResources:
    enable: true
  nodes:
    enable: true

The scan cron jobs will be re-created and their initial run will occur within the next minute.

FilesExpand file tree

user-manual.md

Latest commit

History

user-manual.md

File metadata and controls

User manual

Upgrading from Previous Versions

Configuring the Operator

Mondoo Operator Installation

Installing with kubectl

Installing with Helm

Customization

Installing with Operator Lifecycle Manager (OLM)

Configuring the Mondoo Secret

Creating a MondooAuditConfig

Filter Kubernetes objects based on namespace

Scanning External Clusters

Creating a kubeconfig Secret

Configuring external cluster scanning

Per-cluster configuration

Authentication methods

1. Kubeconfig (most flexible)

2. Service Account Token

3. Workload Identity Federation (cloud-native)

4. SPIFFE/SPIRE (zero-trust)

5. HashiCorp Vault (dynamic credentials)

Container Image Scanning

Creating a secret for private image scanning

Single registry secret

Multiple registry secrets

Default secret name

RBAC considerations

Private image scanning with Workload Identity Federation

RBAC for the WIF ServiceAccount

Routing assets to a specific space with spaceId

Use cases

Example: Two MondooAuditConfigs routing to different spaces

Installing Mondoo into multiple namespaces

Adjust the scan interval

Real-time Resource Watcher (Opt-in)

Enabling the Resource Watcher

Default Behavior

Configuration Options

Example: Custom Configuration

Example: Watch Specific Resource Types

Why High-Priority Resources by Default?

Configure resources for the operator and its components

Configure resources for the operator-controller

Configure resources for the different scanning components

Uninstalling the Mondoo operator

Uninstalling the operator with kubectl

Uninstalling the operator with Helm

Uninstalling the operator with Operator Lifecycle Manager (OLM)

Cleanup failed uninstalls

FAQ

I do not see the service running, only the operator. What should I do?

How do I edit an existing operator configuration?

Why is there a deployment marked as unschedulable?

Why are (some of) my nodes unscored?

How can I trigger a new scan?

Routing assets to a specific space with `spaceId`