Kubernetes Threat Hunting: Container Escape, Lateral Movement, and Cluster Defense

Kubernetes clusters are complex enough that a single misconfigured field in a Pod spec can collapse isolation boundaries that took significant effort to build. Unlike a compromised Linux server — where the blast radius is roughly one host — a cluster compromise can mean control over every workload, credential, and secret across all namespaces.

This article traces the attack path from initial pod access to control plane compromise, with two objectives: understanding how each stage works so you can build detection for it, and identifying the specific controls that break the chain at each step. The perspective throughout is defensive — understanding the attack path is how you build effective threat hunts.

Stage 1: Initial Access and Pod Enumeration

The entry point is typically an application vulnerability — an RCE in a public-facing service, a deserialization flaw, or an SSRF that reaches the EC2/GCP metadata endpoint and retrieves credentials. The attacker lands in a shell inside a running pod.

The first enumeration steps are predictable:

# Confirm container environment
cat /proc/1/cgroup
ls /run/.containerenv 2>/dev/null

# Check for service account token
ls /var/run/secrets/kubernetes.io/serviceaccount/
cat /var/run/secrets/kubernetes.io/serviceaccount/token

# Check for elevated capabilities
cat /proc/self/status | grep CapEff
# Non-zero value indicates capabilities beyond default

# Network reachability to API server
curl -sk https://kubernetes.default.svc/api --header \
  "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)"

On service account tokens: Since Kubernetes 1.24, the default service account token uses projected volume tokens with bounded lifetime — they’re not long-lived static JWTs anymore. More importantly, the default service account in a well-configured cluster has minimal permissions. The risk is over-privileged service accounts, not the existence of the token itself.

To assess what a service account can do:

kubectl auth can-i --list \
  --as=system:serviceaccount:<namespace>:<serviceaccount>

This is the fastest way to enumerate a service account’s actual permissions. A service account that can list secrets cluster-wide, create pods, or access exec on pods is a significant finding. Service accounts with cluster-admin binding are unfortunately common in clusters where Helm charts have been deployed without reviewing their RBAC requirements.

MITRE ATT&CK for Containers — T1613 (Container and Resource Discovery): This enumeration phase maps to reconnaissance of the cluster’s workload topology and permission boundaries.

Stage 2: Container Escape via Misconfigured Security Context

Containers are isolated processes using Linux namespaces and cgroups. They don’t have their own kernel — they share the host’s. The isolation holds until a security context misconfiguration removes it.

The hostPath escape: Mounting the host root filesystem into a container and running as root gives an attacker direct access to the node:

# The misconfiguration that breaks container isolation
spec:
  containers:
  - name: app
    securityContext:
      runAsUser: 0  # root — makes the following dangerous
    volumeMounts:
    - mountPath: /host
      name: host-root
  volumes:
  - name: host-root
    hostPath:
      path: /

With the host filesystem accessible and the container running as root, the attacker can read node credentials, write cron jobs, or use nsenter to execute commands in the host’s namespaces:

# Enter host namespaces — more reliable than chroot for full escape
nsenter --target 1 --mount --uts --ipc --net --pid -- bash

Note: chroot /host_root /bin/bash is sometimes cited but less complete than nsenter — it doesn’t switch namespaces, only the filesystem root. Modern container runtimes with seccomp and AppArmor profiles may restrict nsenter even from root containers, which is why runtime security hardening matters.

The privileged container escape: A pod with privileged: true has all capabilities and can mount the host’s block devices:

# List block devices visible from privileged container
fdisk -l

# Mount host disk
mkdir /mnt/host && mount /dev/sda1 /mnt/host

Breaking these escapes: The primary control is Pod Security Admission (PSA), which replaced the deprecated PodSecurityPolicy in Kubernetes 1.25. PSA enforces the Kubernetes Pod Security Standards at admission time:

# Namespace-level enforcement
apiVersion: v1
kind: Namespace
metadata:
  name: production
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted

The restricted profile blocks hostPath volumes, privileged containers, root users, and most capability grants. The enforce label blocks non-compliant pods at admission; audit and warn log violations without blocking, useful for rollout.

For more granular policy, OPA/Gatekeeper or Kyverno provide admission webhooks with custom policies — useful for organization-specific rules that PSA’s fixed profiles don’t cover.

MITRE ATT&CK for Containers — T1611 (Escape to Host).

Stage 3: Node-Level Lateral Movement

Once on the worker node, the attacker’s target is the kubelet’s kubeconfig, which contains client certificates for authenticating to the API server as the node identity:

# Common locations
cat /etc/kubernetes/kubelet.conf
cat /var/lib/kubelet/kubeconfig

The system:node:<node-name> identity is subject to the NodeAuthorizer, which restricts node access to resources relevant to pods running on that node — it can’t list all secrets cluster-wide. But it can do meaningful reconnaissance:

kubectl --kubeconfig /etc/kubernetes/kubelet.conf \
  get pods --all-namespaces -o wide

This reveals the full workload topology: which pods are running where, their IP addresses, and which namespaces exist. From the node, the attacker can also directly access the container runtime socket to inspect running containers or spawn new ones with elevated permissions:

# If containerd socket is accessible
ctr --namespace k8s.io containers list

Network policies are the primary control for limiting what a compromised node can reach. A well-configured Cilium or Calico policy that restricts pod-to-pod and pod-to-control-plane traffic significantly constrains lateral movement options. Most clusters deploy network plugins but leave network policies empty — an allow-all default.

Stage 4: Control Plane Targeting and etcd

etcd is Kubernetes’s backing store for all cluster state: every Deployment, ConfigMap, Secret, and Service Account token. Secrets in etcd are base64-encoded by default — not encrypted. Encryption at rest (EncryptionConfiguration) is available since Kubernetes 1.13 and is enabled by default in managed services like GKE and EKS with specific configurations, but self-managed clusters frequently leave it disabled.

etcd listens on port 2379, protected by mTLS. The certificates required are on the control plane node:

# Certificates needed for etcd access
/etc/kubernetes/pki/etcd/ca.crt
/etc/kubernetes/pki/etcd/server.crt
/etc/kubernetes/pki/etcd/server.key

An attacker who reaches the control plane node — through a misconfigured pod running there, through a stolen backup, or through network-level access without proper isolation — can read all secrets:

ETCDCTL_API=3 etcdctl \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  get /registry/secrets/ --prefix --keys-only

Defenses at this stage:

• Enable encryption at rest for Secrets specifically (EncryptionConfiguration with AES-GCM or KMS provider)
• etcd should be network-isolated — only the API server should reach port 2379
• VPC/network-level controls or Kubernetes NetworkPolicy (for clusters where etcd is reachable through the pod network) should enforce this
• Regular audit of what can reach the control plane network segment

Threat Hunting: Kubernetes Audit Logs

The Kubernetes API server logs every request with structured data. These logs are the primary detection source for API-level attacks. The key fields: verb, resource, user.username, sourceIPs, responseStatus.code.

Enable audit logging with a policy that captures the events relevant to threat hunting:

apiVersion: audit.k8s.io/v1
kind: Policy
rules:
  # Capture all exec operations
  - level: RequestResponse
    resources:
    - group: ""
      resources: ["pods/exec", "pods/attach"]
  # Capture secret access
  - level: Metadata
    resources:
    - group: ""
      resources: ["secrets"]
  # Capture service account token requests
  - level: Request
    resources:
    - group: "authentication.k8s.io"
      resources: ["tokenreviews"]
  # Capture namespace and RBAC changes
  - level: RequestResponse
    resources:
    - group: "rbac.authorization.k8s.io"
      resources: ["clusterrolebindings", "rolebindings"]

Hunt 1 — Interactive exec into production pods:

# jq query against audit log JSON
cat audit.log | jq -r '
  select(
    .verb == "create" and
    .objectRef.resource == "pods" and
    .objectRef.subresource == "exec" and
    (.requestObject.command // [] | any(. == "bash" or . == "sh"))
  ) |
  [.requestReceivedTimestamp, .user.username,
   .objectRef.namespace, .objectRef.name,
   (.requestObject.command | join(" "))] |
  @tsv
'

Developers execing into production pods should be rare and ideally require an approved change ticket. Any instance outside an incident response window is worth investigating.

Hunt 2 — Default service account doing cluster-wide reconnaissance:

cat audit.log | jq -r '
  select(
    (.user.username | startswith("system:serviceaccount:")) and
    (.user.username | endswith(":default")) and
    .verb == "list" and
    (.objectRef.resource == "secrets" or
     .objectRef.resource == "namespaces" or
     .objectRef.resource == "pods") and
    (.objectRef.namespace == null or .objectRef.namespace == "")
  ) |
  [.requestReceivedTimestamp, .user.username,
   .verb, .objectRef.resource] | @tsv
'

The default service account doing cluster-wide list operations is almost always automated recon.

Hunt 3 — Node identity behaving like a user:

cat audit.log | jq -r '
  select(
    (.user.username | startswith("system:node:")) and
    (
      .verb == "list" or
      (.verb == "get" and .objectRef.resource == "secrets" and
       # NodeAuthorizer blocks cross-node secret reads, but log the attempt
       .responseStatus.code == 403)
    )
  ) |
  [.requestReceivedTimestamp, .user.username,
   .verb, .objectRef.resource, .objectRef.namespace,
   .responseStatus.code] | @tsv
'

A kubelet making API requests for resources outside its own node’s pods is an anomaly. A 403 response is still a detection signal — the NodeRestriction admission controller blocked it, but the attempt in the log is evidence of compromise.

Hunt 4 — RBAC escalation:

cat audit.log | jq -r '
  select(
    .verb == "create" and
    .objectRef.resource == "clusterrolebindings" and
    (.requestObject.roleRef.name == "cluster-admin" or
     .requestObject.roleRef.name | contains("admin"))
  ) |
  [.requestReceivedTimestamp, .user.username,
   .requestObject.subjects[].name,
   .requestObject.roleRef.name] | @tsv
'

Unexpected cluster-admin binding creation is a critical alert regardless of who’s creating it.

Runtime Detection with Falco

Audit logs cover API-level activity. For file system, process, and syscall-level activity inside containers, Falco (CNCF project) provides real-time detection using eBPF or kernel module hooks.

Relevant Falco rules for the attack path above:

# Container spawning a shell — covers initial RCE and exec-based access
- rule: Terminal shell in container
  desc: Detects shell spawned in a container
  condition: >
    spawned_process and container and
    shell_procs and
    not user_expected_terminal_shell_in_container_conditions
  output: >
    Shell spawned in a container (user=%user.name container=%container.name
    image=%container.image.repository shell=%proc.name
    parent=%proc.pname cmdline=%proc.cmdline)
  priority: WARNING

# nsenter or mount used for escape
- rule: Launch Sensitive Mount Container
  desc: Detects container launched with sensitive host mounts
  condition: >
    container_started and container and
    (container.mount.dest[/proc*] != "" or
     container.mount.dest[/sys*] != "" or
     container.mount.dest[/host*] != "")
  output: >
    Sensitive mount in container (user=%user.name name=%container.name
    image=%container.image.repository mounts=%container.mounts)
  priority: ERROR

# Reading kubelet credentials from node filesystem
- rule: Read sensitive file untrusted
  desc: Detects reading of kubelet kubeconfig
  condition: >
    open_read and
    fd.name in (/etc/kubernetes/kubelet.conf,
                /var/lib/kubelet/kubeconfig,
                /etc/kubernetes/pki/etcd/server.key) and
    not proc.name in (kubelet, kubeadm)
  output: >
    Sensitive file read (user=%user.name command=%proc.cmdline
    file=%fd.name container=%container.name)
  priority: CRITICAL

Falco alerts integrate with SIEM, Slack, PagerDuty, or any webhook target through its output plugins.

Defense Checklist

The controls that break the attack chain at each stage:

The common thread: most of these controls exist in Kubernetes and are either disabled by default or require explicit configuration. Enabling PSA’s restricted profile in production namespaces, enforcing network policies that default-deny, and shipping audit logs to a SIEM are the highest-leverage starting points for a cluster that currently has none of these in place.

For deeper coverage: the CIS Kubernetes Benchmark provides scored controls across all cluster components, and kube-bench automates the assessment against those benchmarks.