Learn from common Kubernetes attack paths
Turn recurring Kubernetes incident patterns into practical controls for initial access, workload isolation, identity, software supply chains, detection, and response.
Kubernetes incidents differ in detail, but the attack paths repeat. An exposed service, vulnerable application, stolen credential, malicious image, or excessive permission provides initial access. The attacker then searches for identities, reachable services, secrets, control-plane authority, persistence, and profitable compute.
Teams learn more by translating those patterns into controls than by memorizing individual breach stories. The useful question is: which link in the path would prevent, contain, or expose the same behavior in our clusters?
Model the recurring attack paths
Use a simple chain when reviewing a cluster:
| Stage | Common opportunity | Defensive objective |
|---|---|---|
| Initial access | Public workload, exposed API, vulnerable dependency, stolen credential | Reduce exposure and patch quickly |
| Execution | Malicious image, command execution, writable package or model artifact | Verify inputs and constrain runtime |
| Discovery | ServiceAccount tokens, environment data, DNS, cloud metadata | Limit available information and identity |
| Lateral movement | Open east-west networking and shared credentials | Segment workloads and authenticate peers |
| Privilege escalation | Broad RBAC, privileged Pods, host mounts, node credentials | Enforce least privilege and admission policy |
| Impact | Data theft, cryptomining, sabotage, resource exhaustion | Detect behavior, cap resources, and respond |
Threat modeling should include ML-specific inputs. A model, notebook, dataset utility, or generated package can execute code even when the container image passed its vulnerability scan.
Reduce initial access
Inventory internet-facing load balancers, routes, management endpoints, and firewall rules. Require authentication at application and platform boundaries, remove abandoned services, and keep Kubernetes, node images, add-ons, and application dependencies within supported patch windows.
Separate health, metrics, profiling, and administrative endpoints from public application traffic. Limit request sizes, methods, timeouts, and rates at the edge. Vulnerability management should prioritize reachable components and exploitable paths rather than treating every package finding as equivalent.
Protect the software and artifact supply chain
Build images from controlled sources, scan them, generate provenance, sign approved releases, and deploy immutable digests. Restrict registry write access and alert on overwritten tags or newly public repositories.
Apply comparable controls to code archives, plugins, serialized models, checkpoints, and datasets that can influence execution. Validate origin and integrity before registration or promotion. Run untrusted conversion and inspection steps in an isolated environment with no production credentials and restricted egress.
An admission control should verify policy at deployment time, while continuous scanning catches newly disclosed vulnerabilities in images that are already running.
Limit identity and runtime authority
Use dedicated ServiceAccounts, disable automatic token mounting where it is unnecessary, and bind the smallest set of Kubernetes permissions. Treat permission to create Pods as sensitive because it may allow access to credentials, nodes, or privileged runtime features.
Apply the Pod Security Standards to prevent privileged containers, unsafe capabilities, host namespace sharing, and sensitive host mounts by default. Run as a non-root user, use a read-only root filesystem where practical, and set CPU, memory, and ephemeral-storage limits.
Use workload identity for cloud access. Do not let ordinary Pods inherit a node role through instance metadata. Short-lived, workload-scoped credentials reduce both blast radius and the value of stolen secrets.
Segment workloads and dependencies
Adopt default-deny network policies for protected namespaces, then permit only required DNS, registry, artifact, telemetry, API, and peer-service paths. Separate production, development, public serving, interactive notebooks, and untrusted evaluation workloads.
Network isolation needs tests. Verify that intended dependencies work and that a compromised Pod cannot reach the Kubernetes API, node services, metadata endpoints, databases, or unrelated namespaces without authorization.
Detect behavior across layers
Collect Kubernetes audit events, cloud API activity, workload and node runtime signals, DNS requests, network flows, admission decisions, and image provenance. High-value detections include:
- new privileged or host-networked Pods;
- unexpected
exec,attach, or port-forward activity; - access to Secrets or cluster-scoped resources by a new identity;
- metadata-service requests from application namespaces;
- connections to mining pools, scanning patterns, or unusual destinations;
- sudden CPU saturation without corresponding workload progress;
- deletion or weakening of audit, admission, or network controls.
Correlate alerts around a workload and timeline. Ten isolated alerts are less useful than one incident showing the image, identity, network activity, API changes, and affected data path.
Prepare containment before an incident
Define how to stop new scheduling, isolate a namespace or node, revoke workload and cloud credentials, block destinations, preserve logs and volatile evidence, and redeploy from a trusted source. Practice the runbook without relying on the compromised cluster for every action.
Afterward, fix the enabling condition and the missing detection. Review other clusters, images, credentials, and tenants that share the same pattern. The lesson is complete only when the control is deployed and verified.
Polyaxon gives ML operations stable project and run context across ephemeral Pods. That context helps responders identify the component, code, inputs, image, artifacts, owner, and downstream runs associated with suspicious execution, while Kubernetes and cloud controls enforce containment.