Creating connectivity across systems is painful.
Ask me how I know! It might be a scheduled job that needs data from a database two clusters over, an AI agent that has to call an internal tool in a different cloud account, or a CI runner trying to reach a database nobody is allowed to expose. The workload is different every time; the wall in front of it never is.
The usual fix is to start punching holes. Open a port here, stand up an ingress there, bolt on a VPN, write the firewall rules, and pray the network topology never shifts underneath you. Every hole is a standing invitation, and you own it forever. I gave up on that years ago; these days I lean on OpenZiti to wire my systems together instead.
AI only makes it worse. An agent spins up to handle a single request and reaches for some internal tool or model endpoint it was never explicitly granted. A few minutes later it's gone. Nobody set that access up ahead of time, because nobody knew the agent would exist.
What's worked for me is to make a workload's identity the thing you trust, not its place on the network. Give each workload a cryptographic identity that lives only as long as the workload does, and let it reach the specific services it's allowed to over an encrypted overlay. There's nothing on the open network to find and connect to, and when the pod goes away, its access goes with it. That's OpenZiti, an open-source zero-trust networking platform from NetFoundry, and the rest of this post is how I put it to work in Kubernetes, where each pod gets its own identity when it starts and it disappears when it stops.
THE IDEA: CONNECTIVITY THAT LIVES AND DIES WITH THE POD
Each Kubernetes workload gets its own X.509 certificate-based identity and connects to services through an encrypted overlay network. Ingress and egress can live anywhere, and neither side has to expose a service to reach the other. The workload doesn't need to know where the service is hosted, and the service doesn't need to know where the workload is running. Forget IPs and ports; this is a world of services and identities.
The identity is generated at pod startup and discarded when the pod terminates. There's no admin provisioning identities ahead of time. No enrollment tokens to distribute. The workloads self-provision through 3rd Party CA Auto Enrollment, a feature where any client presenting a certificate signed by a trusted CA can automatically create its own identity in the OpenZiti network.
The example: a Kubernetes CronJob that runs every 10 minutes, reaches a web API, Postgres, and Redis through the OpenZiti overlay, regardless of where those services are hosted, and leaves no reusable credentials behind.
WHY THIS IS SECURE
Before diving into implementation, here's why this architecture matters.
No dependency on host networking
This is the critical design point. The OpenZiti sidecar proxy and all service intercept happen at the pod level, not the host level. The sidecar runs as an unprivileged container inside the pod. It doesn't touch the host's network stack, doesn't require NET_ADMIN capabilities, doesn't install iptables rules on the node, and doesn't need host networking enabled.
This means:
No privileged DaemonSets intercepting traffic at the node level
No host-level network changes that affect other tenants on the same node
No shared network namespace between your workload and the host
Complete pod-level isolation, with the OpenZiti overlay scoped to the pod's own network namespace
The sidecar binds OpenZiti services to 127.0.0.1 inside the pod. Traffic between the app container and the sidecar never leaves the pod's loopback interface. Traffic between the sidecar and the OpenZiti overlay is mTLS-encrypted. Nothing is visible on the node's network.
Similarly, on the hosting side, the OpenZiti router that terminates overlay traffic and forwards it to backend services (Postgres, Redis, etc.) runs as a pod in the cluster, not as a host-level process. Both ends of the connection are container-scoped.
No long-lived credentials
The certificate and identity JSON exist only in memory-backed emptyDir volumes. When the pod terminates, they're gone. There's nothing on disk to exfiltrate, no secret in etcd to leak, no credential to rotate on a schedule.
No shared identity
Every job run creates its own identity: ephemeral-job-20260415-195300. If a workload is compromised, the blast radius is one pod, one run, one identity, and that identity stops being useful the moment the pod terminates. This matters even more for AI agents, which increasingly execute model-generated tool calls and code: when a workload's behavior isn't fully predictable, keeping its identity ephemeral and narrowly scoped bounds what a compromised or misbehaving run can reach.
No exposed services anywhere
The backend services (web API, Postgres, Redis) are accessed exclusively through the OpenZiti overlay. They have no exposed ports, no listening sockets on the cluster network, no ingress rules, no load balancer endpoints. They are not addressable by anything outside the overlay: not by other pods in the same namespace, not by other services in the same cluster, not by anything on the underlying network. They are completely dark. The only way to reach them is with a valid OpenZiti identity and the right policy.
Policy-driven access, not network-driven
Access isn't determined by network topology or namespace boundaries. It's determined by the role assigned during enrollment (#ephemeral-workloads) and the OpenZiti service policies that reference that role. Change the policy, and every future workload gets the updated access, no redeployment needed.
Choosing the right authentication method
OpenZiti supports multiple authentication mechanisms for identities. I evaluated three approaches for ephemeral Kubernetes workloads before settling on one:
External JWT Signers (Kubernetes service account tokens)
OpenZiti can authenticate identities using JWTs from external identity providers. In Kubernetes, every pod gets a projected service account token, a short-lived JWT signed by the cluster's OIDC issuer. I could configure OpenZiti to trust this issuer and let workloads authenticate directly with their service account token.
This is the lowest-infrastructure option: no extra CA, no cert generation. The claimsProperty on the External JWT Signer maps a JWT claim (like sub) to the identity's externalId, and enrollToTokenEnabled means the workload keeps re-authenticating with fresh tokens.
For SDK-native applications, this is a strong option. I ruled this out for the demo because the auto-enrollment flow for External JWT Signers requires SDK-level support that may not be fully implemented in the ziti-tunnel sidecar image.
3rd Party CA with OTT (One-Time Token) Enrollment
This approach pre-creates identities in the controller and distributes a one-time enrollment JWT per workload. The workload presents both the JWT and a client certificate signed by a registered 3rd Party CA to complete enrollment.
This is more secure than pure OTT (the controller validates the cert against the CA), but it still requires an admin to create each identity ahead of time. That defeats the purpose for ephemeral workloads where pods come and go unpredictably.
3rd Party CA with Auto Enrollment (what I chose)
Auto CA enrollment eliminates the admin provisioning step entirely. You register a CA with the controller once, and any client presenting a certificate signed by that CA can create its own identity on the fly. The controller assigns a role and name automatically based on the CA configuration.
The 3rd Party CA's externalIdClaim feature also supports SPIFFE integration: it can extract a SPIFFE ID from the certificate's SAN URI so that workloads running SPIRE can enroll with their SVID certificates directly. This is a viable option if your workloads are within a single cluster where SPIRE is already deployed, but it doesn't extend across cluster boundaries the way a shared CA does.
This is the right fit for ephemeral workloads because:
No per-workload provisioning by an admin
Identity creation is driven by the workload itself at startup
The CA controls who can enroll (only certs signed by the registered CA)
The CA configuration controls what access they get (role attributes assigned at enrollment)
The CA can be any PKI the controller trusts: cert-manager, Vault, or your own internal CA
Works with the existing ziti-tunnel sidecar without SDK changes
Architecture
Everything happens within the pod's own network namespace. The host node sees none of this traffic. The OpenZiti endpoint that provides ingress to the backend services (an edge router or a hosting tunneler) can run as a Kubernetes pod, or anywhere else on the network.
How the pieces fit together
The CA: cert-manager manages the signing authority
I use cert-manager to manage a self-signed RSA CA inside the cluster. This CA exists solely to sign ephemeral workload certificates:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: ziti-ephemeral-ca
spec:
isCA: true
commonName: ziti-ephemeral-workloads
secretName: ziti-ephemeral-ca-keypair
privateKey:
algorithm: RSA
size: 2048
The trust anchor: OpenZiti's 3rd Party CA Auto Enrollment
The CA is registered with the OpenZiti controller as a trusted 3rd Party CA with auto-enrollment enabled:
ziti edge create ca ephemeral-k8s-workloads ca.pem \
--auth \
--autoca \
--role-attributes "ephemeral-workloads" \
--identity-name-format 'ephemeral-job-[commonName]'
This tells the controller: "any client that presents a certificate signed by this CA can automatically create an identity. Give it the role ephemeral-workloads."
That role is the key to authorization. OpenZiti service policies grant #ephemeral-workloads access to specific services, and nothing else. The workload gets exactly the access it needs, determined by the CA it enrolled through, not by a manually assigned permission.
The init container: self-provisioning identity
Each pod starts with an init container that generates a unique certificate, enrolls with the OpenZiti controller, and writes an identity file to a shared memory-backed volume:
initContainers:
- name: enroll
image: openziti/ziti-cli:1.6.15
command: ["sh", "-c"]
args:
- |
# Generate a unique client cert signed by the CA
ziti pki create client \
--pki-root /certs/pki \
--ca-name ephemeral-ca \
--client-name "$(date -u +%Y%m%d-%H%M%S)" \
--client-file workload \
--expire-limit 1
# Auto-enroll: controller sees the CA signature
# and creates an identity automatically
curl -s --cert $CERT --key $KEY --cacert $CA \
-X POST "${ZITI_CTRL}/edge/client/v1/enroll?method=ca" \
-H "Content-Type: application/json" -d '{}'
# Build identity JSON for the sidecar
jq -n --arg ztAPI "${ZITI_CTRL}/edge/client/v1" \
--arg cert "$(cat $CERT)" \
--arg key "$(cat $KEY)" \
--arg ca "$(cat $CA)" \
'{ztAPI: $ztAPI, id: {cert: $cert, key: $key, ca: $ca}}' \
> /ziti-identity/identity.json
No admin API token. No pre-created identity. The workload's certificate is its proof of authorization.
The sidecar: pod-scoped, zero-privilege proxy
The sidecar runs OpenZiti's tunnel in proxy mode, binding each OpenZiti service to a port on the pod's loopback interface:
containers:
- name: ziti-tunnel
image: openziti/ziti-tunnel
args:
- "proxy"
- "ziti.webapi:80"
- "ziti.postgres:5432"
- "ziti.redis:6379"
This is where the host-independence matters most. OpenZiti offers several intercept modes for Kubernetes:
Node-level DaemonSet (ziti-edge-tunnel run): intercepts at the host network level, requires privileged pods and host networking
Transparent proxy sidecar (tproxy): intercepts within the pod but requires NET_ADMIN to set up iptables TPROXY rules
Proxy mode sidecar (proxy): binds services to loopback ports, requires no capabilities at all
I chose proxy mode because it's the least-privilege option. The sidecar is an unprivileged container with no special capabilities. It doesn't modify iptables. It doesn't touch the host network. All intercept is scoped entirely to the pod's loopback interface.
Friendly DNS names are mapped to 127.0.0.1 via Kubernetes hostAliases, so the app container connects to ziti.redis:6379 as if it were a normal service:
hostAliases:
- ip: "127.0.0.1"
hostnames:
- "ziti.webapi"
- "ziti.postgres"
- "ziti.redis"
What the app sees
The app container has no OpenZiti SDK, no special configuration. It makes standard network calls to friendly names and gets real responses through the zero-trust overlay:
===== ziti.webapi (HTTP) =====
{
"headers": {
"Host": "ziti.webapi",
"User-Agent": "curl/8.7.1"
},
"url": "http://ziti.webapi/get"
}
===== ziti.postgres (TCP) =====
Connected: TCP session established on port 5432
===== ziti.redis (PING) =====
Response: +PONG
===== Summary =====
All 3 services reached through OpenZiti overlay.
This identity was ephemeral.
Three services, three different protocols, all reached through the encrypted overlay without touching the host network. The app doesn't know the difference.
Cleaning up
Ephemeral identities accumulate in the OpenZiti controller. A daily cleanup job handles this: a separate Kubernetes CronJob that runs at 02:00 UTC, authenticates to the OpenZiti management API, and deletes any ephemeral-job-* identities older than 24 hours:
==> Cutoff time: 2026-04-14T20:00:12Z
Deleting: ephemeral-job-20260414-100000 (created 2026-04-14T10:00:03Z)
Deleting: ephemeral-job-20260414-101000 (created 2026-04-14T10:10:02Z)
Keeping: ephemeral-job-20260415-195300 (created 2026-04-15T19:53:03Z)
==> Cleanup complete.
When to use this pattern
This pattern fits any workload where pods are created and destroyed frequently and need authenticated access to backend services:
AI agents and agentic workloads: agents and inference jobs that spin up to handle a task and need scoped access to tools, model and inference endpoints, vector stores, and internal APIs, without being handed a shared, standing credential that outlives the task
Kubernetes CronJob workloads: each run gets its own identity scoped to exactly the services it needs
CI/CD runners: build agents that need to reach internal APIs without sharing a static token
Autoscaled deployments: new replicas self-provision without waiting for an admin to create credentials
Preview environments: spin up an ephemeral environment with its own isolated identity and tear it down cleanly
Batch processing and jobs: short-lived compute that should never share credentials with other runs
Try it yourself
The complete demo (manifests, setup script, and cleanup job) is available on GitHub at mguthrie88/openziti-ephemeral-identity-demo. It runs on any Kubernetes cluster with cert-manager and OpenZiti installed.
What you need:
What you get:
A Kubernetes CronJob that self-provisions zero-trust identities every 10 minutes
Three services (web API, Postgres, Redis) reachable only through the encrypted overlay
Pod-scoped networking with no host-level dependencies
Automatic cleanup of expired identities
A pattern you can adapt for your own workloads
Own it, or hand it off
Run your own overlay. OpenZiti is fully open source. Stand up your own controller and routers, register your CA, and everything in this post is yours end to end: no vendor in the loop, no per-seat pricing, no lock-in.
Or let someone else run the network. Operating your own overlay is real, ongoing work: controllers to upgrade, routers to scale, uptime to own. When you'd rather not, NetFoundry offers the same OpenZiti network as a managed, hosted service. The pattern above doesn't change; you just stop being the one paged at 3am when a router falls over.
Star the project. If this was useful, give OpenZiti a star on GitHub. It's a small thing, and it helps us put open-source zero-trust networking in front of more people.
