Zero Kubernetes Secrets Architecture¶

cloudtaser eliminates Kubernetes Secrets from the trust chain -- not only for customer workloads, but for its own operational secrets. The operator, webhook TLS certificates, broker tokens, and enrollment credentials all live in OpenBao and are loaded directly into memory at runtime.

Why¶

cloudtaser's core promise is that secrets never touch etcd. Storing its own operational secrets in Kubernetes Secrets would undermine that promise in several ways:

Dogfooding. If cloudtaser protects customer secrets by keeping them out of K8s Secrets, it should do the same for its own. An operator that stores its webhook TLS key in a Kubernetes Secret is one kubectl get secret -o yaml away from compromise.
etcd as an attack vector. etcd backups, etcd snapshots, and etcd encryption-at-rest key management are all surfaces where operational secrets could leak. Removing them from etcd eliminates the surface entirely.
Unified audit trail. When all secrets (customer and operational) go through OpenBao, every access is recorded in the OpenBao audit log. There is no second audit path to maintain.
Sovereignty for operational secrets. cloudtaser's own TLS certificates and broker tokens deserve the same EU-sovereign storage as customer data. If OpenBao is in the EU, so are the operator's secrets.

How It Works¶

Authentication¶

The operator authenticates to OpenBao using the Kubernetes auth method. It presents its ServiceAccount JWT token to OpenBao, which validates it against the cluster's TokenReview API. No Kubernetes Secret is involved -- the ServiceAccount token is projected into the pod by the kubelet automatically.

Operator Pod
  |
  |  1. Read projected ServiceAccount JWT from /var/run/secrets/kubernetes.io/serviceaccount/token
  |
  v
Vault (K8s auth method)
  |
  |  2. Validate JWT via TokenReview API against the cluster
  |
  v
Return vault token (short-lived, scoped to cloudtaser-operator policy)

First Install¶

On first startup, when no secrets exist in OpenBao yet, the operator bootstraps itself:

Generates a self-signed CA and webhook TLS certificate
Generates broker mTLS certificates (server, client, CA)
Generates a random broker authentication token
Writes all of these to OpenBao at secret/cloudtaser/system/*
Loads them into memory and begins serving

No Kubernetes Secret is created at any point.

Subsequent Starts¶

On subsequent startups, the operator:

Authenticates to OpenBao via Kubernetes auth
Reads existing secrets from secret/cloudtaser/system/*
Loads them into memory
Begins serving the webhook and broker with the loaded credentials

Webhook TLS¶

The webhook TLS certificate is served from memory using Go's tls.Config.GetCertificate callback. There is no file mount and no Kubernetes Secret containing the certificate. The API server's MutatingWebhookConfiguration is patched with the CA bundle on every operator startup.

tlsConfig := &tls.Config{
    GetCertificate: func(*tls.ClientHelloInfo) (*tls.Certificate, error) {
        return operatorCertManager.GetCurrent(), nil
    },
}

Certificate Rotation¶

The operator checks certificate expiry periodically (default: every hour). When a certificate is within 30 days of expiry:

Generates a new certificate signed by the existing CA
Writes the new certificate to OpenBao
Updates the in-memory certificate (the GetCertificate callback returns the new one immediately)
Patches the MutatingWebhookConfiguration if the CA changed

No restart required. No Kubernetes Secret rotation.

High Availability¶

When the operator runs with multiple replicas (operator.ha: true), all replicas need access to the same broker token so they can authenticate wrapper connections consistently. OpenBao provides this naturally:

All replicas read the shared broker token from secret/cloudtaser/system/broker-token
Leader election determines which replica actively serves the webhook
The standby replicas have the same credentials loaded and can take over instantly

This replaces the pattern of storing shared state in a Kubernetes Secret and watching for changes.

OpenBao Paths¶

All operator secrets are stored under a single OpenBao path prefix:

Path	Contents
`secret/cloudtaser/system/webhook-tls`	Webhook server certificate, private key, and CA certificate
`secret/cloudtaser/system/broker-tls`	Broker mTLS certificates: server cert + key, client cert + key, CA certificate
`secret/cloudtaser/system/broker-auth`	Broker authentication token (used by wrappers to authenticate to the broker)
`secret/cloudtaser/system/broker-token`	Shared broker token for HA replicas (ensures consistent wrapper auth across failover)
`secret/cloudtaser/system/enrollment`	SaaS platform enrollment token (only present when connected to cloudtaser Platform)

All paths use the KV v2 secrets engine. The operator's OpenBao policy grants CRUD access to secret/data/cloudtaser/system/* and secret/metadata/cloudtaser/system/*.

The One Exception¶

OpenBao bootstrap credentials -- unseal keys and root token -- cannot be stored in OpenBao they unlock. This is an inherent constraint of any OpenBao-based system.

During cloudtaser-cli source install + cloudtaser-cli source configure, these credentials are:

Output once to the terminal
Must be stored externally in an HSM, on paper (Shamir shares), or in an external KMS
Deleted from Kubernetes after retrieval (if they were temporarily stored during automated bootstrap)

Unseal keys are the one secret cloudtaser cannot protect

OpenBao unseal keys and initial root token are the single exception to the zero-K8s-secrets model. Treat them with the highest level of care. Do not store them in the same cloud account as OpenBao. Do not store them in a Kubernetes Secret. The source install command outputs them exactly once.

Fallback Mode¶

For environments where OpenBao is not available at operator startup, the operator supports falling back to in-memory secrets:

operator:
  secretBackend: inmemory  # default: "vault"

Or via environment variable:

CLOUDTASER_SECRET_BACKEND=inmemory

When secretBackend is set to inmemory, the operator generates certificates and holds them in process memory. They do not persist across restarts but avoid storing secrets in etcd.

When to use fallback mode:

Local development with kind, minikube, or k3d where no OpenBao is available
Environments where OpenBao is not reachable at operator startup (OpenBao deployed after the operator)
Gradual migration from older installations transitioning to OpenBao-based storage

Fallback mode weakens the security model

With secretBackend: inmemory, operational secrets are not persisted and are lost on pod restart. The operator must regenerate certificates each time it starts. Use OpenBao mode in production for durability and centralized audit.

Setup¶

The cloudtaser-cli source configure command configures OpenBao for operator secret storage:

cloudtaser-cli source configure \
  --openbao-addr https://vault.eu.example.com \
  --token hvs.YOUR_ADMIN_TOKEN

This creates:

OpenBao policy cloudtaser-operator -- grants CRUD on secret/data/cloudtaser/system/* and secret/metadata/cloudtaser/system/*
Kubernetes auth role cloudtaser-operator -- bound to the operator's ServiceAccount (cloudtaser-operator in namespace cloudtaser-system)

After running this command, deploy or restart the operator with secretBackend: vault (the default). The operator will authenticate, bootstrap its secrets on first run, and serve from memory on subsequent runs.

Preview first

Add --dry-run to see what will be created without making changes:

cloudtaser-cli source configure --dry-run \
  --openbao-addr https://vault.eu.example.com \
  --token hvs.YOUR_ADMIN_TOKEN

Security Model¶

The following diagram shows the trust chain with OpenBao-based secret storage. The key property is that no Kubernetes Secret appears anywhere in the chain.

flowchart TD
    subgraph "Kubernetes Cluster"
        SA["ServiceAccount Token<br/>(projected by kubelet)"]
        OP["cloudtaser Operator"]
        WH["Webhook Server<br/>(TLS from memory)"]
        BR["Auth Broker<br/>(mTLS from memory)"]
        API["K8s API Server"]
    end

    subgraph "EU Vault"
        KA["K8s Auth Method"]
        SS["secret/cloudtaser/system/*"]
        AL["Audit Log"]
    end

    SA -->|"1. JWT token"| OP
    OP -->|"2. Login with JWT"| KA
    KA -->|"3. Validate via TokenReview"| API
    KA -->|"4. Return vault token"| OP
    OP -->|"5. Read/write secrets"| SS
    SS -->|"6. Load into memory"| OP
    OP --> WH
    OP --> BR
    SS -.->|"All access logged"| AL

    style SA fill:#e8f5e9,stroke:#2e7d32
    style SS fill:#e3f2fd,stroke:#1565c0
    style AL fill:#fff3e0,stroke:#e65100

What this eliminates:

Traditional approach	Zero K8s Secrets approach
Webhook cert stored in `cloudtaser-operator-certs` K8s Secret	Webhook cert stored in OpenBao, served from memory
Broker token stored in a K8s Secret	Broker token stored in OpenBao, loaded into memory
etcd contains cloudtaser TLS private keys	etcd contains no cloudtaser secrets
`kubectl get secret` exposes operator credentials	`kubectl get secret` returns nothing cloudtaser-related
Cert rotation requires Secret update + rollout	Cert rotation is in-memory, no restart needed
Audit requires K8s audit policy configuration	All access logged in OpenBao audit log automatically

Trust boundaries:

The operator trusts the kubelet to project a valid ServiceAccount token
The operator trusts OpenBao to authenticate and authorize correctly
OpenBao trusts the Kubernetes TokenReview API to validate ServiceAccount tokens
No trust is placed in etcd, Kubernetes Secrets, or the cloud provider's control plane

Architecture Overview -- how all cloudtaser components fit together
Enterprise Deployment Architecture -- CLI-as-orchestrator model for air-gapped and multi-cluster environments
Security Model -- full threat model and trust boundaries
Troubleshooting: Operator OpenBao connectivity -- diagnosing OpenBao connection issues
Helm Values -- configuring operator.secretBackend and OpenBao settings