Docker#

• Start time ~50 ms
• Kernel shared with host
• Sees namespaces
• Uses cgroups

A container is not a small virtual machine, and the difference matters the first time something goes wrong. A VM boots its own kernel on virtualised hardware. A container is an ordinary Linux process that the kernel has been asked to answer questions differently for. Two features do all of the work: namespaces change what the process can see — its own process table, its own filesystem root, its own network interfaces — and cgroups limit what it can consume.

That is the whole trick, and every practical consequence follows from it. Containers start in milliseconds because nothing is being booted. A Linux image cannot run on a Windows kernel, because there is no kernel inside the image to run it. And a container that gets root and finds a kernel vulnerability is on the host, because the host's kernel is the one it was using all along.

• Strength — the dependency argument ends The image carries the runtime, the libraries and the system packages, so "works on my machine" becomes "works, because it is the same filesystem". Nothing else you can do to a project buys as much reproducibility for as little effort.
• Weakness — isolation is about tidiness, not security Root in a container is root on the host if anything escapes, and a namespace does not contain a kernel bug. Run as a non-root user, and treat a container boundary as a guard rail rather than a wall.

• Image read-only, shared
• Container + 1 writable layer
• On rm writable layer gone
• To keep data use a volume

An image is a stack of read-only layers, each one the filesystem diff produced by a single Dockerfile instruction and addressed by the hash of its contents. A container is that stack plus one thin writable layer on top. Writes use copy-on-write: modifying a file that came from an image layer copies it upward first, so the image itself is never touched and a hundred containers can share the same layers on disk.

The consequence developers care about is the last one in the strip above. The writable layer belongs to that container and dies with it, and containers are replaced constantly — on every rebuild locally, on every deploy in production. Anything written to it is temporary by design.

• Strength — sharing makes pulls cheap Ten services built on the same base image store that base once and pull it once. It is also why pushing a new build of your app usually uploads a few megabytes rather than the whole image.
• Weakness — layers are append-only, so nothing is ever really deleted Removing a file in a later layer only adds a marker hiding it. The bytes stay in the earlier layer and anyone who can pull the image can extract them — which is why a secret copied in and then deleted is still a leaked secret.

• Instructions 1 layer each
• Order by rate of change
• CMD exec form
• USER non-root

A Dockerfile is a recipe read from top to bottom, where each instruction produces a layer. Most of the craft is in two decisions: what goes in, and in what order. Everything else is a handful of instructions you will use every day.

Here is a Dockerfile that would pass review, annotated with the reason for each decision. It is worth reading closely — sections 4 to 6 are essentially explanations of why it is written this way.

# syntax=docker/dockerfile:1.7

# Pinned by digest: a tag can be rebuilt upstream, a digest cannot change.
FROM python:3.12-slim@sha256:2f9a3c1e0b7d5a4c8e2f1b6d3a9c7e5f0b8d2a4c6e1f3b5d AS runtime

# Fail fast and unbuffered, so logs appear immediately in docker logs.
ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    PIP_NO_CACHE_DIR=1

WORKDIR /app

# System packages change rarely, so they go near the top. Install and clean
# in ONE RUN - a later "rm" cannot shrink an earlier layer.
RUN apt-get update \
 && apt-get install -y --no-install-recommends libpq5=15.* \
 && rm -rf /var/lib/apt/lists/*

# The single highest-value line in the file: copy ONLY the dependency
# manifest, so editing your source does not invalidate the install below.
COPY requirements.lock .
RUN pip install --require-hashes -r requirements.lock

# Source last, because it changes on every commit.
COPY . .

# A real user, created here rather than assumed to exist.
RUN useradd --system --uid 10001 app && chown -R app:app /app
USER 10001

EXPOSE 8080
# Exec form: your process is PID 1 and receives SIGTERM directly (§11).
CMD ["python", "-m", "app"]

.dockerignore - as important as the Dockerfile, and usually missing
───────────────────────────────────────────────────────────────────
The build context is everything sent to the builder before the first
instruction runs. Without this file you are uploading your .git history
on every build, and any COPY . . bakes it into the image.

.git
.gitignore
node_modules
__pycache__
*.pyc
.venv
.env
.env.*
.pytest_cache
.mypy_cache
dist
build
coverage
*.md
tests

Why each matters
  .git         often larger than the application itself
  node_modules reinstalled inside; copying breaks native modules
  .env         the single most common way a secret enters an image
  __pycache__  changes constantly, invalidating COPY for no reason
  tests        not needed at run time; more code for a scanner to flag

• Cache key instruction + layers below
• One miss invalidates everything below
• Rule stable first, volatile last

Each layer's cache key is the instruction text plus the digest of everything beneath it. For COPY, the checksum of the copied files is part of the key. That one sentence explains every fast build and every slow one: invalidating a layer invalidates every layer below it, so instructions must be ordered from least to most frequently changing.

• Strength — a correctly ordered file is dramatically faster for free Moving one line usually takes a four-minute rebuild to four seconds, on every commit, for every developer and every CI run. There are very few changes with that ratio of effort to payoff.
• Weakness — the cache is per machine unless you export it A fresh CI runner has no cache and rebuilds everything, which is why builds are fast locally and slow in CI. BuildKit can push the cache to the registry (--cache-to) so runners share it.

• Typical saving 5–10×
• Build tools left behind
• distroless no shell

A multi-stage build lets you compile in one image and ship another. The build stage can contain a compiler, development headers, test dependencies and build-time credentials; the final stage contains only what runs. You get three benefits at once and they compound: faster pulls on every deploy and every scale-up, a much shorter vulnerability report, and a container that is a far worse place for an attacker to land because there is no compiler and no package manager in it.

• Strength — the guarantee is structural Build-time material cannot reach production by accident, because the final stage starts from a clean base and you name exactly what crosses. That is stronger than remembering to delete things.
• Weakness — a tiny image is harder to debug Distroless has no shell, so docker exec … sh fails and so does kubectl exec -it … -- bash. Keep a :debug variant of the same image, or use ephemeral debug containers that attach a toolbox alongside.

• Manifest a range
• Lockfile exact + hashes
• ENV secret permanent
• Secret mount leaves nothing

A build is only reproducible if its inputs are. The manifest (package.json, pyproject.toml) records your intent — usually a range like ^4.17.0. The lockfile records the resolution: the exact version and hash of every package in the transitive graph. Commit the lockfile and install from it, or two builds of the same commit will contain different code and every test result becomes an opinion about a program you did not ship.

The second half of this section is the one that causes real incidents. A private package index, a licence key, a token for a protected artifact — these are needed during the build and must not survive it. ENV keeps the value in the image configuration permanently. ARG does not persist as an environment variable but is recorded in the build history, which docker history prints without pulling the layers at all. And deleting the file in a later layer does nothing, because layers are append-only.

# syntax=docker/dockerfile:1.7        <- required for the mounts below

FROM python:3.12-slim@sha256:2f9a3c... AS build
WORKDIR /app

COPY requirements.lock .

# --mount=type=cache keeps the wheel cache OUTSIDE the layer: fast rebuilds
# without shipping the cache. --require-hashes refuses to install anything
# not pinned by hash, turning a supply-chain surprise into a build failure.
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install --require-hashes --prefix=/install -r requirements.lock

# A private index needs a token. --mount=type=secret exposes it to THIS
# command only; it is never written to a layer or to the build history.
RUN --mount=type=secret,id=index_token \
    PIP_INDEX_URL="https://$(cat /run/secrets/index_token)@pypi.acme.dev" \
    pip install --prefix=/install acme-internal==2.1.0

# ---- and how you pass it in ----
#   docker build --secret id=index_token,env=INDEX_TOKEN .
#   docker build --secret id=index_token,src=./token.txt .

# Verify nothing leaked - this prints the build history WITHOUT pulling:
#   docker history --no-trunc myapp | grep -i -E 'token|secret|password'

# Generate a lockfile with hashes from a loose manifest.
pip-compile --generate-hashes --output-file requirements.lock requirements.in

# Install from the lock in CI - never let the install re-resolve.
pip install --require-hashes -r requirements.lock   # Python
npm ci                                              # Node: fails if lock
                                                    # and manifest disagree
go mod download && go mod verify                    # Go: checks go.sum

# What is in the graph, and who pulled it in?
pipdeptree --reverse --packages urllib3
npm ls lodash --all

# THE ANTI-PATTERNS, for completeness:
#   ENV NPM_TOKEN=...          -> in the image config, forever
#   ARG NPM_TOKEN              -> in docker history, readable without a pull
#   COPY .npmrc . && RUN ... && RUN rm .npmrc
#                              -> the bytes are still in the earlier layer

• Strength — a locked build is a build you can reason about Same commit, same bytes. That is what makes "it passed CI" a statement about the artifact you are about to run rather than about a similar one.
• Weakness — pinning without updating is its own risk A lockfile guarantees you install the same thing every time, including the same known-vulnerable version. Pair it with automated weekly updates, batched so that review is possible.

• -p host:container
• EXPOSE documentation only
• localhost is the container
• Container to container by name

Each container gets its own network namespace: its own interfaces, its own routing table and its own port space. That is why a service listening on port 5000 inside a container is not on your machine's port 5000, and why the first thing everyone does is get a connection refused on an application that is running perfectly.

• Strength — ports stop colliding Five services can all listen on 8080 internally and be published on five different host ports, or on none at all. You never again edit a config file because two projects both wanted 3000.
• Weakness — the indirection is invisible in error messages ECONNREFUSED 127.0.0.1:5432 looks identical whether the database is down or simply in a different namespace, and the second is far more likely. Check where before checking whether.

• Named volume survives
• Bind mount dev only
• Writable layer lost on replace
• down -v deletes volumes

There are exactly three places a container can write, and choosing the wrong one is behind most of the confusing behaviour people hit in their first month. The writable layer is the default and dies with the container. A named volume is managed by the runtime and outlives the container. A bind mount is a host directory grafted into the container, which is how live-reload development works and is tied to one machine's filesystem.

• Strength — state becomes an explicit decision Because the default is "everything is lost", you are forced to say what matters. Teams that internalise this write services that survive being restarted, which is the same property that makes rolling deploys possible.
• Weakness — a bind mount hides what was underneath Mounting your project over /app also replaces the dependencies installed during the build with whatever is on your laptop — compiled for the wrong platform. Mask the dependency directory with an anonymous volume.

• Goal one command to run
• depends_on order, not readiness
• Fix + healthcheck

Compose describes a set of containers, a network and some volumes in one file. Its value for a developer is narrow and large: one command produces a working system, with the real database and the real message broker rather than mocks, and a new colleague is productive in minutes rather than after a page of setup instructions.

The trap is depends_on. It reads like "wait until the database is ready" and it means "start the database's container first" — which is roughly a hundred milliseconds of head start against a database that needs several seconds to accept connections.

• Strength — parity with production where it matters The same image, the same database engine and version, the same environment variable names. What it cannot give you is orchestration behaviour — probes, resource limits, rolling updates — so those bugs are only findable in a real cluster.
• Weakness — the file rots quietly A service is added to production and not to Compose, a version diverges, a seed script stops matching the schema. Running compose up in CI on a schedule is the cheapest way to notice before a new joiner does.

• Unit of deploy one image per service
• They find each other by service name
• Shared database not microservices
• New cost the network

Containers made microservices practical, which is why the two words are so often used together — and why so many teams end up with the costs of a distributed system and none of the benefits. The mechanical part is genuinely easy and you have already seen all of it: one image per service, one container each, one user-defined network, and they call each other by name. The hard part is not mechanical at all.

Start from what actually changes. In a single process, a call from the order module to the payment module is a function call: it cannot half-succeed, it cannot time out, and the compiler checks it. Move payments into its own container and that same call becomes a network request that can be slow, can be retried into a duplicate charge, and whose contract is now a convention rather than a type. You have converted a compile-time problem into a run-time one, and everything below is the consequence of that trade.

The packaging rules follow directly, and they are the part Docker is responsible for. One service, one image, one repository of truth for its dependencies. Do not build a single image containing three services selected by an environment variable — that image cannot be deployed or rolled back per service, which removes the only reason you split them. And each service gets its own configuration through its own environment variables, so that the same image runs unchanged in every environment.

• Strength — independent deployability is the whole point If a change to search can be built, tested, released and rolled back without coordinating with anyone, the split has paid for itself. That is the property to test a proposed boundary against, and it is the only one that matters.
• Weakness — a shared database undoes all of it If two services write the same tables, they cannot change schema independently, cannot be released independently and cannot fail independently. That is a distributed monolith: every cost of both models and the benefits of neither.

• docker stop SIGTERM, then 10 s
• Then SIGKILL
• Shell form swallows signals
• Exec form your app is PID 1

Stopping a container is a two-step conversation. The runtime sends SIGTERM and waits ten seconds; if the process is still alive it sends SIGKILL, which cannot be caught, blocked or handled. Whether you get a clean shutdown or a killed process comes down to one question: does the signal reach your code?

It very often does not, because of a syntax detail. CMD python app.py — shell form — is rewritten to /bin/sh -c "python app.py", making the shell PID 1 and your application its child. A plain sh running one command does not forward signals, so your handler never runs, every stop takes the full ten seconds, and every deploy drops the requests that were in flight.

• Strength — the fix is one line and it pays off in production Exec form plus a SIGTERM handler is what makes a rolling deploy invisible to users. The same code path is exercised on every single deploy, so it is well tested by accident.
• Weakness — PID 1 has no default handlers and no reaper Signals with default actions are ignored for PID 1, and orphaned grandchildren become zombies that nothing cleans up. If your process spawns children, add an init: docker run --init, or tini as the entrypoint.

• Push uploads only new layers
• Tag mutable pointer
• Digest immutable
• latest not a version

A registry stores manifests and blobs. A manifest is a small JSON document listing the config and the layers, each by digest; the image digest is the hash of that manifest. Because layers are content-addressed, a push uploads only the digests the registry does not already have — which is why the first push of an image is slow and every subsequent one moves a few megabytes.

• Strength — the digest is a fact "What is running in production?" has exactly one answer when the deployment references myapp@sha256:9f2c…, and rollback becomes "deploy the previous digest", which is always available because it is still in the registry.
• Weakness — tags invite the opposite latest is not a version, it is the tag applied when you give none. Two containers started an hour apart can be on different code with the same tag, and nothing anywhere will tell you.

• 137 OOM killed
• 127 command not found
• 139 segfault
• 143 terminated

The instinct when a container will not start is to edit the Dockerfile and rebuild. Resist it for ninety seconds: the exit code and the logs almost always name the problem, and rebuilding blind turns a two-minute diagnosis into an afternoon.

• User non-root
• Root filesystem read-only
• Config injected
• Logs stdout

Everything to this point makes the image work. This section is the short list that makes it acceptable to run somewhere you cannot log into. None of it is difficult; all of it is routinely missing, and each item removes a specific failure you would otherwise meet in production.

# Everything the checklist asks for, as flags - so you can see the shape
# before it becomes YAML in an orchestrator.
docker run -d --name api \
  --user 10001:10001 \
  --read-only \
  --tmpfs /tmp:rw,noexec,nosuid,size=64m \
  --cap-drop ALL \
  --security-opt no-new-privileges \
  --memory 512m --cpus 1 \
  --restart unless-stopped \
  --stop-timeout 45 \
  -e DATABASE_URL \
  -e LOG_LEVEL=info \
  -p 8080:8080 \
  ghcr.io/acme/api@sha256:9f2c1ab...

# Verify the hardening actually applied.
docker exec api id                       # uid=10001 - not root
docker exec api touch /oops              # Read-only file system
docker exec api capsh --print | head -2  # no capabilities

# Scan before it ever reaches a registry.
trivy image --severity HIGH,CRITICAL --ignore-unfixed \
  --exit-code 1 ghcr.io/acme/api:"$GIT_SHA"

# The same requirements expressed where they will actually live.
# Recognising this shape is most of what Kubernetes asks of you.
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 10001
    fsGroup: 10001
  containers:
    - name: api
      image: ghcr.io/acme/api@sha256:9f2c1ab...
      securityContext:
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true
        capabilities: { drop: ["ALL"] }
      resources:
        requests: { cpu: 250m, memory: 512Mi }
        limits:   { memory: 512Mi }
      env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef: { name: api-db, key: url }
      volumeMounts:
        - { name: tmp, mountPath: /tmp }     # required with a read-only root
      readinessProbe:
        httpGet: { path: /readyz, port: 8080 }
  terminationGracePeriodSeconds: 45
  volumes:
    - name: tmp
      emptyDir: { medium: Memory, sizeLimit: 64Mi }

• A container is a process Namespaces limit what it sees, cgroups limit what it uses, and the kernel is the host's. Fast to start, cheap to replace, not a security boundary.
• Layer order is a cache key Stable things first, volatile things last, and copy the dependency manifest before the source. One line of ordering is usually the difference between a four-second and a four-minute rebuild.
• Build fat, ship thin Multi-stage keeps compilers, test dependencies and build secrets out of the image that runs. Smaller pull, smaller attack surface, shorter scan report.
• Assume it will be deleted Data in volumes or a database, logs to stdout, config from the environment. Then restart, rebuild, rollback and scale are all the same boring operation.
• The traps COPY . . above the install. Secrets in ENV or ARG. Shell-form CMD. Binding to 127.0.0.1. Deploying :latest. depends_on without a healthcheck. Every one looks harmless on the day it is written.
• The limits Docker packages and runs one container. It does not give you rollouts, self-healing, service discovery across machines or autoscaling — which is what the next course is about.

# ---- build ----
docker build -t app:dev .
docker build --target build -t app:test .        # stop at an earlier stage
docker buildx build --platform linux/amd64,linux/arm64 -t app:v1 --push .

# ---- run ----
docker run --rm -it -p 8080:8080 --env-file .env app:dev
docker run --rm -it --entrypoint sh app:dev      # look inside, do not start
docker compose up -d --build
docker compose down                              # -v ALSO deletes volumes

# ---- inspect ----
docker ps -a                                     # exit codes live here
docker logs -f --tail 100 api
docker exec -it api sh
docker stats --no-stream
docker history --no-trunc app:dev                # secrets hide here
docker inspect app:dev --format '{{json .Config}}' | jq

# ---- clean up ----
docker system df
docker system prune -af --filter 'until=168h'    # not volumes
docker volume prune                              # unused volumes only

Decision table
──────────────
Where does this data go?
  needed after the container is replaced?  -> volume / database / object store
  scratch, regenerable?                    -> writable layer or tmpfs
  a log line?                              -> stdout

How do I reach it?
  from the host / a browser?               -> -p host:container
  from another container?                  -> service name on the real port
  from inside the same container?          -> localhost

Which base image?
  need a shell and apt?                    -> debian-slim
  static binary (Go, Rust)?                -> distroless/static or scratch
  smallest possible, no debugging?         -> distroless
  Alpine?                                  -> only if musl libc is fine;
                                              expect surprises with wheels,
                                              DNS resolution and native deps

What identifies this build?
  for a human reading a changelog          -> :v1.9.0
  for tracing back to a commit             -> :9f2c1ab
  for anything that deploys or verifies    -> @sha256:...

1. The Kubernetes crash course — the natural sequel, also written for developers: what happens to your container once something else is responsible for running it.
2. The DevOps crash course — where containers sit in the wider delivery loop: pipelines, deployment strategies, observability and reliability.
3. All DevOps courses — the catalogue page for this topic.
4. The Operating Systems crash course — namespaces, cgroups, signals and the page cache from the kernel's side, which makes container behaviour stop being surprising.
5. The Dockerfile reference and the BuildKit documentation are unusually good primary sources — particularly the pages on cache mounts, secret mounts and multi-platform builds, all of which are under-used.