Container security

Pick a small base and pin it

The base image is most of your attack surface. A full ubuntu or node base drags in a shell, a package manager, and hundreds of libraries your app never calls, and every one of those is code an attacker can use and a scanner will flag. Two moves cut it down: start from a slim or distroless base, and pin what you start from.

Distroless images (and slim variants) ship your app and its runtime, with no shell and no apt/apk. That does two things. A process that gets breached has no sh to spawn and no package manager to pull tools with, and the scanner has far fewer packages to find CVEs in, so its output is signal instead of noise.

Pinning is about which bytes you actually run. A tag like node:22 or :latest is mutable: the publisher can move it to a new image at any time, so two builds a week apart can produce different bases with no change on your side. Pinning to a digest freezes the exact content.

FROM node:latest              # mutable: the bytes behind this change under you
FROM node:22-slim@sha256:...  # digest pin: the exact same base every build

Wrong ":latest is good because I always get the newest patched packages." You also get whatever else moved, with no review, and your build stops being reproducible. Pin the digest and update it on purpose (a bot like Dependabot or Renovate can raise the PR), so a base change is a diff you approve, not a surprise.

Ship the artifact, not the toolchain

A multi-stage build compiles in a fat image and copies only the built artifact into a small runtime image. containers-deployment covers the mechanics; the security point is what you leave behind. Compilers, npm/build caches, dev dependencies, and your source tree stay in the build stage and never reach production, so they are not there to be exploited or scanned.

FROM node:22 AS build
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM gcr.io/distroless/nodejs22   # no shell, no package manager
WORKDIR /app
COPY --from=build /app/dist ./dist
COPY --from=build /app/node_modules ./node_modules
CMD ["dist/server.js"]

The runtime stage has no build tools and no source, so it is smaller and there is less to attack.

Do not run as root

By default a container process runs as root (uid 0), and root inside the container is a real uid 0 on the host kernel the container shares. If the app is compromised, or a mount is misconfigured, or a kernel bug lets the process break out, root inside is a far bigger blast radius than an unprivileged user. The senior notes go into the escape paths; the floor is: set a non-root user.

RUN useradd --uid 10001 app
USER 10001          # numeric uid, so Kubernetes can verify it is non-root

Use a numeric uid, not a name. Kubernetes checks runAsNonRoot against the image's UID metadata without launching the container, so a username it cannot resolve to a number fails that check. Drop the two together: USER in the image and runAsNonRoot: true in the pod, so a base that forgot the USER line cannot silently run as root.

Scan the image, but do not trust a green result

Scanners (Trivy, Grype, and the rest) read the packages installed in the image and match their versions against a CVE feed. That is genuinely useful and you should gate on it. It is also a narrow check, and knowing its limits is what separates a real answer from a checkbox.

A clean scan means "no package version I know about matches a CVE in my feed today." It does not mean safe. The scanner cannot see a zero-day (no CVE yet), cannot find a bug in your own application code, and cannot tell whether a flagged CVE is in code your app ever reaches. It also goes stale: the same image can be green today and red tomorrow when the feed updates, with no change to the image.

The practical fallout is noise. A busy image throws hundreds of low-relevance CVEs, many of them in packages you never call or that the distro marked "will not fix," and engineers who see hundreds of findings start ignoring all of them. Filtering the unfixable (trivy --ignore-unfixed) and shrinking the base so there is less to flag are how you keep the signal readable. Scanning tells you about known problems in what you shipped; the next section, signing and provenance, is about trusting that you shipped what you think you did.

Why root in a container is the thing to fix

The whole reason the hardening below matters is that a container is not a security boundary the way a VM is. It shares the host kernel (containers-deployment and linux-fundamentals cover the namespace and cgroup mechanics), so isolation is only as strong as that kernel, and the process's power on the host is set by which uid and which capabilities it runs with.

Root inside the container is uid 0 against the shared kernel. If the process is compromised, three paths turn that into host access: a kernel vulnerability that lets it escape the namespaces, a misconfigured mount (a hostPath volume, a bind-mounted docker socket, /proc from the host) that reaches the host filesystem directly, or a container that was run --privileged and never had its isolation applied in the first place. In all three, the difference between "attacker is uid 0" and "attacker is an unprivileged uid with no capabilities" is the difference between owning the node and being stuck in a box with nothing useful in it.

So the goal is least privilege at the process level, and it stacks: run as a non-root uid, drop the capabilities you do not use, forbid gaining new ones, and make the root filesystem read-only. Each one is cheap and each one removes a rung an attacker would climb.

The securityContext that actually hardens a pod

Here is the shape worth memorizing, because interviewers ask you to harden a manifest live:

securityContext:
  runAsNonRoot: true            # kubelet refuses to start if the image runs as uid 0
  runAsUser: 10001
  readOnlyRootFilesystem: true  # writes go to mounted volumes only
  allowPrivilegeEscalation: false  # sets no_new_privs: no setuid escalation
  capabilities:
    drop: ["ALL"]               # start from zero
    add: ["NET_BIND_SERVICE"]   # add back only what the app needs
  seccompProfile:
    type: RuntimeDefault        # runtime's syscall filter, blocks the dangerous ones

The capability line is where seniors separate. The kernel splits root's power into distinct capabilities (capabilities(7)), so you almost never need "root," you need one or two operations. Binding a port below 1024 is CAP_NET_BIND_SERVICE. The default container runtime already grants a handful (CHOWN, SETUID, SETGID, NET_RAW, DAC_OVERRIDE, and more), several of which help an attacker, so drop: ["ALL"] then adding back the one you need is strictly better than the default.

Wrong "The container needs a privileged operation, so run it privileged: true." Privileged mode grants close to the full capability set plus host device access and turns off much of the isolation, an enormous blast radius for one operation. Grant the single capability instead. And never reach for CAP_SYS_ADMIN: the man page literally warns "don't choose CAP_SYS_ADMIN if you can possibly avoid it" because it accumulated so many unrelated powers (mount, namespace ops, and more) that having it is close to being root.

runAsNonRoot has a nuance that trips deploys. The kubelet validates it by reading the image's configured UID, not by launching the container and reading /etc/passwd, so a Dockerfile that sets USER to a name the kubelet cannot resolve to a number fails with CreateContainerConfigError: container has runAsNonRoot and image will run as root. Set a numeric uid in the image. User namespaces are the next step up: they remap container root to an unprivileged host uid so uid 0 inside is nobody outside, and Kubernetes exposes this via hostUsers: false (the namespace remapping mechanism itself belongs to linux-fundamentals).

Signing, SBOM, and provenance are three different guarantees

Scanning tells you about known vulnerabilities in what you built. It does not tell you the image running in production is the one your pipeline produced, or how it was built. That is a separate problem, and people collapse three distinct things into "we secure the supply chain," so keep them apart:

• A signature says this exact digest came from an identity you trust.
• An SBOM (software bill of materials) says what is inside the image, package by package, so when the next CVE drops you can answer "are we affected" without rescanning everything.
• Provenance says how and by whom the image was built, as tamper-evident metadata.

Sigstore's cosign signs without a long-lived key. It gets an ephemeral key, Fulcio issues a short-lived certificate binding that key to your OIDC identity (a CI workflow, a Google or GitHub login), the signing event is recorded in the Rekor transparency log, and the signature is stored in the registry next to the image by digest. Nothing secret is stored at rest.

Wrong "We run cosign verify, so we only accept our own images." Fulcio issues certs to anyone with an OIDC identity, so a Fulcio cert alone only proves some identity signed the image, not which. cosign 2.0+ will not perform keyless verification unless you pass --certificate-identity and --certificate-oidc-issuer (it errors that one is required for verification in keyless mode): it forces you to name the identity and issuer you trust instead of accepting any signer. The identity and issuer are the control:

cosign verify \
  --certificate-identity=https://github.com/org/repo/.github/workflows/release.yml@refs/heads/main \
  --certificate-oidc-issuer=https://token.actions.githubusercontent.com \
  registry.example.com/app@sha256:...

SLSA frames the provenance side as build levels, useful as a maturity ladder even if you never certify: L1 is provenance exists (unsigned, catches mistakes), L2 adds a hosted build platform and a signed provenance (tamper-evidence after the build), L3 adds a hardened, isolated builder so one build cannot influence another or reach the signing key (tamper-resistance during the build). cosign attaches both the SBOM and the SLSA provenance as in-toto attestations bound to the image digest, verified with cosign verify-attestation.

Make it mandatory at admission, not per-manifest

A hardened securityContext that lives in one team's YAML is a suggestion. The next manifest, or the next team, ships without it and nothing stops them. Enforcement belongs at the cluster edge.

Pod Security Admission (GA since Kubernetes 1.25) applies the Pod Security Standards by namespace label. There are three levels and three modes, and you set them independently so you can watch before you block:

metadata:
  labels:
    pod-security.kubernetes.io/enforce: restricted  # reject violating pods
    pod-security.kubernetes.io/warn: restricted      # warn the client
    pod-security.kubernetes.io/audit: restricted     # annotate the audit log

Baseline blocks the known-bad: no privileged, no host namespaces, no hostPath, no adding dangerous capabilities. Restricted is the hardening profile and adds the rest of the list above: runAsNonRoot required, allowPrivilegeEscalation: false, capabilities dropped to ALL (only NET_BIND_SERVICE may be added back), a non-Unconfined seccomp profile, and a restricted set of volume types. Start a namespace at warn: restricted to surface every violation without breaking deploys, then flip enforce once the manifests are clean.

Built-in PSA is coarse by design (three fixed levels, no custom rules, no image checks). For anything finer you reach for a policy engine (Kyverno, OPA/Gatekeeper) as a validating admission webhook: it can require images from an allowed registry, require a digest instead of a tag, and, wired to cosign, reject any image whose signature or provenance does not verify. That last check is what closes the loop the scanner cannot: a mutable tag can be repointed at a malicious image after CI passed, and only verifying the signed digest at admission catches it.