Breaking Changes & Migration Guide (2026)¶
This page documents all breaking behavioral changes shipped across CloudTaser components that require attention during upgrade. Each entry includes the affected versions, impact, and step-by-step migration instructions.
How to use this guide¶
- Identify which components you are upgrading
- Check the relevant sections below for breaking changes
- Follow the migration steps before upgrading in production
Upgrade order
When multiple components have breaking changes in the same release wave, upgrade in this order: Helm → Operator → Bridge (onprem) → Beacon → Wrapper → CLI. Upgrading out of order may cause admission webhook rejections or pairing failures.
CLI (cloudtaser-cli)¶
target connect: --bridge-endpoint flag¶
Versions affected: v0.18.x+
GitHub reference: cli#549
Impact: The target connect command now accepts --bridge-endpoint to wire up P2P
reverse mode. Without this flag the operator falls back to the legacy direct-vault
connectivity path, which is no longer the default in new installs. If you have automated
scripts calling target connect they will continue to work, but will not enable P2P
mode and beacon pairing will not occur.
Migration:
Add --bridge-endpoint pointing to your beacon relay when connecting a cluster in P2P mode:
# Before
cloudtaser target connect \
--vault-addr https://vault.eu.example.com \
--cluster-id my-cluster
# After (P2P mode)
cloudtaser target connect \
--vault-addr https://vault.eu.example.com \
--cluster-id my-cluster \
--bridge-endpoint beacon.example.com:443
Leave the flag absent only for air-gapped or direct-vault topologies where the cluster
has line-of-sight to vault. For all other deployments --bridge-endpoint is required.
target migrate: routes through broker admin proxy¶
Versions affected: v0.18.x+
GitHub reference: cli#550
Impact: target migrate now prefers routing secret migration traffic through the
bridge admin proxy rather than contacting vault directly from the CLI host. The
--vault-token flag (and VAULT_TOKEN env var / --vault-token-file) is deprecated
but still accepted and still required — the command errors if no token is provided
via any of these methods. Removing --vault-token without an alternative will break
your target migrate invocations.
A future release will introduce broker-proxy auth as a replacement, at which point direct token passing will be removed.
Migration:
No immediate action is required. Your existing --vault-token invocations continue to
work. Plan to migrate to broker-proxy auth when it becomes available:
# Current (deprecated but still works — still required)
cloudtaser target migrate \
--vault-token s.xxxxxxxx \
--namespace my-app
# Preferred: use --vault-token-file to avoid shell history leakage
cloudtaser target migrate \
--vault-token-file /run/secrets/vault-token \
--namespace my-app
# Future (when broker-proxy auth lands — not yet available)
# cloudtaser target migrate \
# --namespace my-app
Ensure the bridge (cloudtaser-onprem) is running and reachable before running
target migrate. See the Bridge migration section below for
required bridge configuration changes.
Operator (cloudtaser-operator)¶
Webhook failurePolicy=Fail enforced by tamper controller¶
Versions affected: v0.13.x+
GitHub reference: operator#817
Impact: The tamper controller now actively enforces failurePolicy=Fail on the
mutating admission webhook. If any component (another operator, GitOps tooling, manual
kubectl edit) resets failurePolicy to Ignore, the tamper controller will
immediately restore it. Pods submitted during a CA rotation window will be rejected
rather than admitted uninstrumented.
Migration:
- Coordinate CA rotations with your team — no pods can be scheduled during the rotation window.
- If you use GitOps (ArgoCD, Flux) to manage webhook objects, remove those objects from your GitOps scope or add an ignore annotation, since the tamper controller owns the webhook configuration:
# ArgoCD: exclude the MutatingWebhookConfiguration from managed resources
# in your Application spec:
spec:
ignoreDifferences:
- group: admissionregistration.k8s.io
kind: MutatingWebhookConfiguration
name: cloudtaser-webhook
- Pre-drain nodes or use PodDisruptionBudgets to manage in-flight pods during planned CA rotation windows.
New env vars: CLOUDTASER_VAULT_NAMESPACE / CLOUDTASER_SECRETSTORE_NAMESPACE¶
Versions affected: v0.13.x+
GitHub reference: operator#818
Impact: Two new environment variables are available on the operator pod. They are optional but if your vault uses namespaces (Vault Enterprise / OpenBao namespaces) and you previously passed the namespace via the vault address or a custom annotation, you must now set these explicitly.
Migration:
Add the new env vars to your operator Helm values or Deployment spec if you use vault namespaces:
# values.yaml
operator:
env:
CLOUDTASER_VAULT_NAMESPACE: "eu-prod" # vault namespace for secret reads
CLOUDTASER_SECRETSTORE_NAMESPACE: "eu-prod" # secret store namespace
If you do not use vault namespaces, no action is required.
Webhook rule scope narrowed to NamespacedScope¶
Versions affected: v0.13.x+
GitHub reference: operator#819
Impact: The mutating webhook rule scope has changed from * (all resources) to
NamespacedScope. Cluster-scoped resources (e.g. ClusterRole, PersistentVolume,
Node) will no longer trigger the webhook. This is a security improvement but may
surface as unexpected behavior if you relied on the webhook intercepting cluster-scoped
objects.
Migration:
No action required for standard workload-namespace usage. If you have custom tooling that expected the webhook to mutate cluster-scoped objects, update that tooling — those mutations were no-ops in previous versions anyway, as CloudTaser only instruments pod-level workloads.
CloudTaserPolicy CRD: required vaultPaths field¶
Versions affected: v0.13.x+
GitHub reference: operator#820
Impact: The CloudTaserPolicy CRD now requires a vaultPaths field. Existing
CloudTaserPolicy objects created before this version that lack vaultPaths will fail
validation after the CRD is upgraded and will need to be recreated.
Migration:
Before upgrading the operator CRDs, audit your existing CloudTaserPolicy objects:
For each policy missing vaultPaths, add the field specifying the allowed vault paths
this policy permits:
apiVersion: cloudtaser.io/v1alpha1
kind: CloudTaserPolicy
metadata:
name: my-app-policy
namespace: production
spec:
vaultPaths: # now required
- "secret/data/my-app/*"
- "secret/data/shared/db-creds"
selector:
matchLabels:
app: my-app
Apply the updated policies before upgrading the operator to prevent reconciliation failures.
Wrapper (cloudtaser-wrapper)¶
CLOUDTASER_VAULT_* env vars renamed to CLOUDTASER_SECRETSTORE_*¶
Versions affected: v0.2.x+
GitHub reference: wrapper#414
Impact: Environment variables previously named CLOUDTASER_VAULT_* that configure
the secret store connection have been renamed to CLOUDTASER_SECRETSTORE_* to support
multiple secret store backends. The old names still work but emit deprecation warnings
in wrapper logs. They will be removed in a future release.
Migration:
Update any direct env var references in pod annotations or templates:
| Old name | New name |
|---|---|
CLOUDTASER_VAULT_ADDR |
CLOUDTASER_SECRETSTORE_ADDR |
CLOUDTASER_VAULT_TOKEN |
CLOUDTASER_SECRETSTORE_TOKEN |
CLOUDTASER_VAULT_CACERT |
CLOUDTASER_SECRETSTORE_CACERT |
CLOUDTASER_VAULT_NAMESPACE |
CLOUDTASER_SECRETSTORE_NAMESPACE |
CLOUDTASER_VAULT_ROLE |
CLOUDTASER_SECRETSTORE_ROLE |
If you set these variables via Helm values or operator annotations, the operator injects
the new names automatically once upgraded. Check your custom pod specs and init-container
overrides for any hardcoded CLOUDTASER_VAULT_* references and migrate them.
To silence the deprecation warnings immediately while phasing the rename across your estate, you can set both old and new names in parallel — the wrapper will prefer the new name.
Broker retry schedule: 5 s base / 60 s cap exponential backoff¶
Versions affected: v0.2.x+
GitHub reference: wrapper#415
Impact: The wrapper's retry schedule for broker connectivity failures has changed from a fixed interval to exponential backoff (5 s base, 60 s cap). If your monitoring alerting relies on a fixed retry cadence (e.g. "N retries in M seconds" thresholds), those alerts may fire differently under the new schedule.
Migration:
Update any monitoring rules or SLO dashboards that assume a fixed broker retry interval. The new schedule produces retry attempts at approximately: 5 s, 10 s, 20 s, 40 s, 60 s, 60 s, … (jittered). Adjust alert thresholds accordingly. No configuration change is required in the wrapper itself — the new schedule is applied automatically.
Beacon (cloudtaser-beacon)¶
--auth-key or --auth-key-file now required in production mode¶
Versions affected: v0.3.x+
GitHub reference: beacon#249
Impact: Starting this version, running the beacon without an auth key in any mode
other than --dev is a hard startup failure. Previously the beacon would start without
authentication and log a warning. Any deployment that relied on the unauthenticated
default will fail to start.
Migration:
Generate a strong auth key and configure it before upgrading:
# Generate a 32-byte key and base64-encode it
openssl rand -base64 32 > /etc/cloudtaser/beacon-auth-key
# Pass as a file (recommended — avoids shell history leakage)
cloudtaser-beacon --auth-key-file /etc/cloudtaser/beacon-auth-key
# Or pass inline (suitable for K8s env var injection)
cloudtaser-beacon --auth-key "$BEACON_AUTH_KEY"
Store the key in your vault or a K8s secret mounted at runtime. The bridge
(cloudtaser-onprem) must be configured with the matching key via
BEACON_AUTH_KEY environment variable. Helm chart users: set
beacon.authKey or beacon.authKeySecret in values.yaml.
/metrics endpoint moved to opt-in via --metrics-enabled¶
Versions affected: v0.3.x+
GitHub reference: beacon#250
Impact: The beacon's Prometheus /metrics endpoint is now disabled by default.
Existing Prometheus scrapers targeting the beacon will return 404 after the upgrade.
Migration:
If you scrape beacon metrics, add --metrics-enabled to the beacon startup flags:
Helm chart users:
Update your Prometheus scrape_configs or ServiceMonitor if the port has changed.
If you do not scrape beacon metrics, no action is required.
Bridge / onprem (cloudtaser-onprem)¶
AppRole auth replaces static VAULT_TOKEN¶
Versions affected: v0.5.x+
GitHub reference: onprem#469
Impact: The bridge still accepts VAULT_TOKEN (environment variable) and
--vault-token (flag), but now emits a deprecation warning at startup when either
is used. The preferred production auth method is AppRole via --vault-approle-role-id
and --vault-wrapped-token (wrapped token intro). Existing deployments using
VAULT_TOKEN will continue to work without modification — this is a soft migration
window, not a breaking change.
Migration:
No immediate action is required. Existing VAULT_TOKEN deployments will keep working
with a deprecation warning in bridge logs. To migrate to AppRole auth (recommended):
- Create an AppRole in your vault for the bridge:
# Enable AppRole auth if not already enabled
bao auth enable approle
# Create a policy for the bridge
bao policy write cloudtaser-bridge - <<EOF
path "cloudtaser/data/clusters/*" { capabilities = ["read", "list"] }
path "cloudtaser/data/bridge/*" { capabilities = ["read"] }
path "auth/approle/login" { capabilities = ["create"] }
EOF
# Create the role
bao write auth/approle/role/cloudtaser-bridge \
token_policies="cloudtaser-bridge" \
token_ttl="1h" \
token_max_ttl="4h"
# Retrieve the role ID
bao read -field=role_id auth/approle/role/cloudtaser-bridge/role-id
- Generate a wrapped secret ID for the bridge:
bao write -wrap-ttl=120s -f auth/approle/role/cloudtaser-bridge/secret-id
# Use the wrapping_token value as --vault-wrapped-token
- Update your bridge deployment:
# Before (deprecated — still works, logs a warning)
VAULT_TOKEN=s.xxxxxxxx cloudtaser-bridge --vault-addr https://vault.eu.example.com
# After (preferred — AppRole auth with automatic token renewal)
cloudtaser-bridge \
--vault-addr https://vault.eu.example.com \
--vault-approle-role-id <role-id> \
--vault-wrapped-token <wrapping-token>
- The bridge automatically renews its token via AppRole. Once you have confirmed
AppRole auth works, remove
VAULT_TOKENfrom secrets, Helm values, and environment files at your convenience.
--vault-skip-verify requires companion --vault-allow-insecure=true¶
Versions affected: v0.5.x+
GitHub reference: onprem#470
Impact: The --vault-skip-verify flag (TLS certificate verification bypass) now
requires the companion flag --vault-allow-insecure=true to take effect. Without
--vault-allow-insecure=true, passing --vault-skip-verify alone is a no-op and the
bridge will reject the vault TLS certificate if it cannot be verified.
Migration:
This change is intentional to prevent accidental TLS bypass via a single flag. If you
currently use --vault-skip-verify in a development or staging environment with a
self-signed vault certificate, update your startup command:
# Before
cloudtaser-bridge --vault-skip-verify
# After (both flags required to bypass TLS verification)
cloudtaser-bridge --vault-skip-verify --vault-allow-insecure=true
Production environments
--vault-skip-verify + --vault-allow-insecure=true must never be used in
production. Use a properly signed certificate from a trusted CA instead. If your
vault certificate is self-signed, add your CA to the bridge's trust store via
--vault-ca-cert.
Helm chart (cloudtaser-helm)¶
operator.flagd.enabled introduced (default OFF, requires bridgeEndpoint)¶
Versions affected: v1.x+
GitHub reference: helm#1087
Impact: A new operator.flagd.enabled value controls the feature flag sidecar.
It defaults to false. If you previously deployed flagd alongside the operator via
a separate manifest, the Helm chart will not manage it until you opt in. No behavioral
change for users who did not use flagd.
Migration:
To enable the flagd sidecar via Helm, set both values:
operator:
flagd:
enabled: true
bridgeEndpoint: "beacon.example.com:443" # required when flagd is enabled
Omitting bridgeEndpoint when flagd.enabled: true is a validation error.
ebpf.requireFullEnforcement defaults to true; may crashloop on Tier 2 kernels¶
Versions affected: v1.x+
GitHub reference: helm#1088
Impact: The ebpf.requireFullEnforcement Helm value now defaults to true. On
kernels that do not support full LSM enforcement (Tier 2 kernels: standard GKE COS,
default Debian, most managed K8s distributions without custom kernel flags), the eBPF
daemonset will crashloop rather than silently fall back to detect-only mode.
This is a deliberate security posture choice — silent degradation from enforcement to detection was indistinguishable from a misconfiguration or attack. Crashlooping surfaces the issue immediately.
Migration:
Option A (recommended): upgrade your kernel to Tier 1.
See Kernel Compatibility for supported kernels
with full LSM enforcement.
Option B: explicitly opt out of full enforcement on Tier 2 kernels.
Only acceptable for development or evaluation clusters where the security tradeoff is understood:
Security implication
Setting requireFullEnforcement: false on a Tier 2 kernel means the eBPF layer
runs in detect-only mode. /proc/environ, ptrace, and process_vm_readv attacks
are logged but not blocked. Do not use this setting in production for regulated
workloads.
Check your current kernel tier:
Kernels with CONFIG_LSM including bpf at position 1 in the LSM order are Tier 1.
GKE COS nodes (kernel 5.15/6.1 default images) are Tier 2 unless you use the
COS containerd-optimized image with custom LSM flags.
Summary checklist¶
Use this checklist before each upgrade:
| Component | Breaking change | Action required |
|---|---|---|
| CLI | --bridge-endpoint flag |
Add flag to target connect in P2P mode |
| CLI | target migrate auth |
--vault-token deprecated, plan migration to broker-proxy auth |
| Operator | failurePolicy=Fail enforcement |
Exclude webhook from GitOps scope |
| Operator | CLOUDTASER_VAULT_NAMESPACE env |
Add if using vault namespaces |
| Operator | Webhook NamespacedScope |
Verify no cluster-scoped mutation deps |
| Operator | CloudTaserPolicy.vaultPaths |
Add vaultPaths to all existing policies |
| Wrapper | CLOUDTASER_VAULT_* rename |
Update to CLOUDTASER_SECRETSTORE_* |
| Wrapper | Broker retry backoff | Update monitoring alert thresholds |
| Beacon | Auth key required | Generate and configure --auth-key-file |
| Beacon | /metrics opt-in |
Add --metrics-enabled to scraping deployments |
| Bridge | AppRole auth (preferred) | VAULT_TOKEN deprecated; migrate to AppRole when ready |
| Bridge | --vault-allow-insecure=true |
Add companion flag alongside --vault-skip-verify |
| Helm | operator.flagd.enabled |
Set bridgeEndpoint if enabling flagd |
| Helm | ebpf.requireFullEnforcement: true |
Upgrade kernel to Tier 1 or opt out explicitly |