Learn Docker
From First Principles to Deep Dive

“the application” and “the machine running the application.”

The original problem: an application is more than its source code

Suppose a program works on a developer laptop. To run it elsewhere, you also need the right:

• operating-system libraries;
• language runtime;
• package versions;
• configuration;
• filesystem layout;
• system users and permissions;
• startup command.

Traditional deployment copied the code and then tried to reconstruct its environment. That is where “works on my machine” comes from.

flowchart LR
    SRC["Source code"] --> DEV["Developer machine
Python 3.x · libfoo v2
special config"]
    SRC --> PROD["Production server
Python 3.y · libfoo v1
different config"]
    DEV --> OK["works"]
    PROD --> FAIL["fails"]

The source was identical. The execution environment was not.

Before containers: three common answers

Containers did not make these approaches useless. They introduced a smaller deployment unit: package the userspace an application needs, then run it while sharing the host kernel.

The five pains Docker addresses

Docker is strongest when the image becomes the unit of delivery:

flowchart LR
    CODE["Source + Dockerfile"] --> BUILD["Build once"]
    BUILD --> IMG["Image digest
sha256:abc..."]
    IMG --> TEST["Test"]
    IMG --> STAGE["Stage"]
    IMG --> PROD["Production"]

The important word is same. Rebuilding separately for production can produce a different artifact even from the same commit.

First Principle #1 — Package the environment, not the machine

An image normally contains:

• application files;
• runtime libraries and executables;
• installed dependencies;
• default environment metadata;
• a default startup command.

It does not normally contain:

• a kernel;
• physical or virtual hardware;
• production secrets;
• durable application data;
• environment-specific identity.

flowchart TB
    subgraph IMAGE["IMAGE · portable userspace"]
        APP["application"]
        DEPS["dependencies"]
        ROOTFS["root filesystem"]
        META["metadata + startup contract"]
    end
    IMAGE --> K1["Linux kernel on host A"]
    IMAGE --> K2["Linux kernel in Docker Desktop VM"]
    IMAGE --> K3["Linux kernel on host C"]

This is the portability bargain: the image carries userspace, while the runtime supplies a compatible kernel and infrastructure.

The portability boundary

“Runs anywhere” is useful shorthand, not a law of physics.

• A Linux image needs a Linux kernel. Docker Desktop supplies one through a VM.
• CPU architecture matters. An amd64 image does not natively execute on arm64.
• Host-mounted files, devices, kernel features, and privileged operations reduce

portability.

• External services and configuration still have to exist.

Good container design keeps these assumptions explicit.

First Principle #2 — An image is a recipe result; a container is a process

People often use image and container interchangeably. Do not.

flowchart LR
    D["Dockerfile"] -->|docker build| I["Image
myapp:1.0"]
    I -->|docker run| C1["Container A"]
    I -->|docker run| C2["Container B"]
    I -->|docker run| C3["Container C"]

Three containers can start from one image. Their writes and processes are independent. Deleting one container does not delete the image or the other containers.

First Principle #3 — Immutable image, replaceable container

You can enter a container and edit files. That does not make it a sound deployment method.

flowchart TD
    subgraph MUT["Mutable repair"]
        M1["container is wrong"] --> M2["exec in and patch it"]
        M2 --> M3["unknown state
cannot reproduce reliably"]
    end
    subgraph IMM["Image replacement"]
        I1["image is wrong"] --> I2["change source or Dockerfile"]
        I2 --> I3["build a new image"]
        I3 --> I4["replace container"]
    end

The operational rule is:

**Do not repair a deployed container. Repair the inputs, build a new image, and

replace the container.**

This makes rollback, audit, scaling, and debugging much more predictable.

Runtime state belongs outside the replaceable container:

• configuration enters through environment variables, files, or a config service;
• secrets enter at runtime, not in image layers;
• durable data goes to a volume or external service;
• logs go to stdout/stderr or an external logging system.

First Principle #4 — Isolation is not virtualization

A container is a normal host process with a restricted view of the system.

flowchart TB
    subgraph VM["VIRTUAL MACHINES"]
        HYP["Hypervisor"]
        G1["Guest OS + kernel
app A"]
        G2["Guest OS + kernel
app B"]
        HYP --> G1
        HYP --> G2
    end

    subgraph CT["CONTAINERS"]
        K["One host kernel"]
        C1["isolated process A
userspace A"]
        C2["isolated process B
userspace B"]
        K --> C1
        K --> C2
    end

Sharing a kernel is why containers start quickly and use less memory than full VMs. It is also why their isolation boundary is different. A kernel vulnerability can matter across containers; a privileged container can deliberately cross boundaries.

Use the right tool:

• choose containers for packaging, speed, density, and process-level isolation;
• choose VMs when you need separate kernels, stronger tenant boundaries, or a

different operating system;

• commonly use both: containers inside VMs.

First Principle #5 — Docker records intent, but it is not magic

Docker can reproduce what you declare. It cannot reproduce hidden assumptions.

The Dockerfile, Compose file, image metadata, and deployment configuration form a contract. The more complete the contract, the smaller the gap between environments.

Build time vs runtime — the boundary to memorize

flowchart LR
    subgraph BUILD["BUILD TIME"]
        B1["source"] --> B2["Dockerfile instructions"]
        B2 --> B3["immutable image"]
    end
    subgraph RUN["RUN TIME"]
        R1["image"] --> R2["configuration + secrets
network + storage + limits"]
        R2 --> R3["container process"]
    end
    B3 --> R1

Build-time choices should be the same everywhere: compiled code, packages, runtime, and default command. Runtime choices vary by environment: credentials, endpoints, replica identity, resources, and durable storage.

If a production secret appears in a Dockerfile, the boundary is already broken.

Recap

Docker makes an application environment into a repeatable artifact. The artifact is an image; running it creates a container. The container is an isolated process, not a miniature VM. Treat images as immutable, containers as replaceable, and runtime state as external. Docker improves consistency only to the extent that you declare the application’s real requirements.

whose view and resource access are shaped by kernel features.

Start with an ordinary process

When you start a program on Linux, the kernel gives it:

• a process ID;
• virtual memory;
• file descriptors;
• credentials;
• CPU time;
• access to files, devices, and networks according to permissions.

Without isolation, processes see the same system-wide process tree, network interfaces, hostname, and mounts. Containers change what a group of processes can see and consume.

flowchart TD
    IMG["Image root filesystem"] --> RUNTIME["Container runtime"]
    RUNTIME --> NS["Namespaces
what can it see?"]
    RUNTIME --> CG["cgroups
how much can it use?"]
    RUNTIME --> SEC["Capabilities + seccomp + LSM
what may it do?"]
    NS --> PROC["isolated application process"]
    CG --> PROC
    SEC --> PROC

The filesystem comes from the image. The isolation comes primarily from the kernel. Docker assembles these pieces behind a convenient API.

Namespaces — control what a process can see

Linux namespaces give a process a scoped view of global resources.

Namespaces do not make a process unreal. From the host, it is still visible:

inside container: process believes it is PID 1
outside on host:  the same process might be PID 28417

Both observations are true in different PID namespaces.

Network namespace example

A container can bind to port 80 without occupying host port 80 because the container has its own network namespace. Publishing 8080:80 adds a forwarding path from host port 8080 to container port 80.

cgroups — control what a process can consume

Namespaces answer “what can you see?” Control groups answer “how much can you use?”

cgroups can account for and constrain resources such as:

• memory;
• CPU time and CPU weight;
• number of processes;
• block I/O;
• device access, depending on cgroup version and configuration.

flowchart LR
    HOST["Host capacity
8 CPU · 16 GiB"] --> C1["Container A cgroup
2 CPU · 1 GiB"]
    HOST --> C2["Container B cgroup
4 CPU · 8 GiB"]
    HOST --> SYS["Host processes
remaining capacity"]

Important distinctions:

• a memory limit is a hard ceiling; exceeding it can trigger an OOM kill;
• a CPU quota constrains time; excess demand usually causes throttling;
• a CPU share/weight controls relative priority during contention;
• no limit usually means the container may compete for most host resources.

Isolation without limits prevents some visibility but not resource exhaustion.

Root filesystem — image layers plus one writable layer

The process sees a root filesystem assembled from read-only image layers and a thin container-specific writable layer.

flowchart TB
    W["container writable layer"] --> L3["app layer"]
    L3 --> L2["dependencies layer"]
    L2 --> L1["base image layer"]
    L1 --> VIEW["merged filesystem view seen by process"]
    W --> VIEW

If the process changes /app/config.json, the storage driver performs a copy-on-write operation. The underlying image remains unchanged. Delete the container and its writable layer disappears unless the data was mounted elsewhere.

Capabilities — split root into smaller privileges

Traditional Unix root can do almost anything. Linux capabilities split that power into units such as changing network configuration or signaling arbitrary processes.

Docker starts containers with a reduced capability set, but “root in a container” still deserves caution. You can reduce power further:

docker run --cap-drop=ALL --cap-add=NET_BIND_SERVICE nginx

The best capability is one the application never receives.

seccomp and Linux security modules

More layers can constrain the process:

• seccomp filters system calls;
• AppArmor or SELinux can restrict access using host security policy;
• no-new-privileges prevents gaining extra privilege through executables;
• a read-only root filesystem blocks writes outside declared mounts.

No single layer is “container security.” Security comes from overlapping boundaries.

OCI: why Docker is not the only runtime

The Open Container Initiative standardizes major pieces of the ecosystem:

• the image format;
• image distribution;
• the runtime specification.

This lets an image built with Docker run through other OCI-compatible tooling. Docker commonly uses lower-level components rather than directly creating every kernel primitive itself.

sequenceDiagram
    participant CLI as docker CLI
    participant D as dockerd
    participant C as containerd
    participant R as low-level OCI runtime
    participant K as Linux kernel

    CLI->>D: create/start container
    D->>C: manage container lifecycle
    C->>R: create OCI container
    R->>K: namespaces, cgroups, rootfs, process
    R-->>C: initial process started

The exact implementation can evolve; the durable mental model is layered: high-level API → lifecycle manager → OCI runtime → kernel.

PID 1 — the process with special responsibilities

The image’s command becomes the container’s initial process. Inside its PID namespace, it is PID 1.

PID 1 matters because it must:

1. receive and handle termination signals correctly;
2. reap orphaned child processes.

A shell wrapper can accidentally intercept signals:

# Shell form: starts through /bin/sh -c
CMD python app.py

# Exec form: Python becomes PID 1 directly
CMD ["python", "app.py"]

Prefer exec-form CMD and ENTRYPOINT. If the application does not reap children, run with Docker’s tiny init process:

docker run --init myapp:1.0

In Compose:

services:
  app:
    image: myapp:1.0
    init: true

Graceful shutdown

docker stop sends the configured stop signal, normally SIGTERM, waits for a grace period, then sends SIGKILL if the process has not exited.

sequenceDiagram
    participant U as Operator
    participant D as Docker
    participant P as PID 1
    U->>D: docker stop app
    D->>P: SIGTERM
    alt exits during grace period
        P-->>D: exit code
    else ignores or traps badly
        D->>P: SIGKILL after timeout
    end

Your application should stop accepting work, finish or release in-flight work within the allowed time, close connections, flush essential state, and exit.

Containers vs virtual machines

Docker Desktop on macOS and Windows is a useful demonstration: Linux containers still need a Linux kernel, so Desktop supplies one in a managed VM. The Docker CLI hides much of that indirection.

storage, networking, filesystems, a container manager, and the kernel.

The map

flowchart LR
    subgraph CLIENT["CLIENT SIDE"]
        CLI["docker CLI"]
        COMPOSE["docker compose"]
        CTX["Docker context"]
    end

    subgraph HOST["DOCKER HOST"]
        API["dockerd
Engine API"]
        BK["BuildKit
image builds"]
        CD["containerd
image + lifecycle"]
        OCI["OCI runtime
create process"]
        OBJ["images · containers
networks · volumes"]
        K["host kernel"]
    end

    subgraph REG["REGISTRY"]
        IDX["image manifests"]
        BLOBS["content-addressed layers"]
    end

    CLI --> API
    COMPOSE --> API
    CTX -.-> CLI
    API --> BK
    API --> CD
    API --> OBJ
    CD --> OCI --> K
    API <--> REG
    BK <--> REG

You do not need to memorize every implementation detail. Memorize the boundaries.

docker CLI — the client

The docker command parses your intent and sends an API request. It is not normally the component that runs the container.

docker version

The output has a Client section and a Server section. If the Client appears but the Server cannot be reached, the CLI is installed but no reachable Engine is available.

Contexts

A Docker context selects which daemon the CLI talks to:

docker context ls
docker context show

The target may be:

• a local Docker Engine;
• Docker Desktop’s managed Engine;
• a remote Engine reached over SSH or TLS;
• another compatible endpoint.

This leads to a critical rule:

Relative bind-mount paths and daemon-side resources belong to the Docker host, not

necessarily to the laptop where you typed the command.

dockerd — the API and object manager

The Docker daemon:

• exposes the Docker Engine API;
• manages containers, images, networks, and volumes;
• coordinates image pulls and pushes;
• delegates lower-level lifecycle work;
• applies daemon configuration and logging behavior.

The daemon often has significant host privilege. Access to its Unix socket is therefore highly sensitive.

/var/run/docker.sock

Mounting that socket inside a container effectively gives the container control over the daemon and, in a typical rootful setup, a path to control the host. Treat Docker socket access like administrative access.

containerd and the OCI runtime — lifecycle layers

At a simplified level:

• dockerd provides Docker’s API and higher-level object behavior;
• containerd manages images, snapshots, tasks, and container lifecycle;
• an OCI runtime such as runc creates the namespaced, constrained process;
• a shim helps keep the running container independent of the daemon’s immediate

process tree and manages I/O/lifecycle details.

The names matter less than the separation of concerns. If dockerd restarts, a well-configured system can keep existing container processes running.

BuildKit — the builder

BuildKit turns a build definition and build context into image content. It provides:

• dependency-aware build execution;
• build cache reuse;
• parallel work where possible;
• secret and SSH mounts that need not become layers;
• multi-platform build support through Buildx;
• external cache import/export.

docker build commonly uses BuildKit. docker buildx exposes advanced builder and output controls.

docker buildx ls
docker buildx build --help

Registry — distribution, not execution

A registry stores and serves image manifests and blobs. Docker Hub is one registry; private and cloud registries use the same broad model.

flowchart LR
    LOCAL["local image store"] -->|push| REG["registry"]
    REG -->|pull| HOSTA["host A"]
    REG -->|pull| HOSTB["host B"]
    REG -->|pull| HOSTC["host C"]

A registry repository is not a Git repository. It is a namespace containing image manifests referenced by tags or digests.

Docker objects

Compose is not a new runtime object. It is a client-side model that asks the Engine to create ordinary containers, networks, volumes, and related resources with predictable labels and names.

What happens during docker run nginx

sequenceDiagram
    autonumber
    participant U as User
    participant C as docker CLI
    participant D as dockerd
    participant R as Registry
    participant M as containerd/runtime
    participant K as Kernel

    U->>C: docker run nginx
    C->>D: create container from nginx
    D->>D: check local image store
    alt image missing
        D->>R: resolve tag and fetch manifest
        R-->>D: manifest + missing blobs
    end
    D->>D: prepare writable layer and network
    D->>M: create and start task
    M->>K: configure namespaces/cgroups, exec process
    K-->>M: process running
    M-->>D: status + I/O
    D-->>C: attach output
    C-->>U: container stdout/stderr

Important consequences:

• run is conceptually pull if needed + create + start;
• the tag is resolved before the process starts;
• the container receives a writable layer and runtime configuration;
• without -d, the CLI attaches to the process output;
• stopping the CLI and stopping the container are related only according to attach

and signal behavior.

Docker Desktop adds a VM boundary

On native Linux Engine:

CLI → dockerd → Linux kernel

On macOS or Windows running Linux containers:

CLI → Docker Desktop → Linux VM → dockerd/runtime → Linux kernel

This affects:

• bind-mount performance and file sharing;
• meaning of “host” networking;
• available memory/CPU, which may be capped by Desktop settings;
• where daemon data physically lives.

But the container mental model remains the same inside the Linux VM.

How to inspect the architecture yourself

# Client and server versions
docker version

# Engine, storage driver, cgroups, plugins, and security options
docker info

# Which endpoint the CLI currently targets
docker context show
docker context inspect "$(docker context show)"

# All Docker objects at a glance
docker system df

Do not publish unredacted docker info or context output blindly; endpoints, registry names, and environment details may be sensitive.

collection of content-addressed filesystem layers.

Image references: registry, repository, tag, digest

Consider:

ghcr.io/acme/payments:1.4

A fully immutable reference can include a digest:

ghcr.io/acme/payments@sha256:4f3c...

Tags are convenient pointers. Digests identify exact manifest content.

flowchart LR
    T1["tag: 1.4"] --> M1["manifest digest A"]
    TL["tag: latest"] --> M1
    TL -.->|"later moved"| M2["manifest digest B"]

latest is not “the newest image” discovered by Docker. It is merely the default tag name used when you omit one. A registry does not promise that it points to the newest, best, or safe artifact.

Practical tagging strategy

Push multiple useful references to the same release:

payments:1.4.2       human release
payments:1.4         moving minor-series pointer
payments:git-a1b2c3  source traceability

Deploy critical workloads by digest when exact reproducibility matters. Retain human-readable tags for discovery and operations.

The anatomy of an image

An OCI-style image includes:

1. a manifest describing platform-specific content;
2. a config object containing metadata such as environment, entrypoint, and

the ordered layer list;

1. compressed filesystem layer blobs;
2. sometimes an image index pointing to multiple platform manifests.

flowchart TD
    REF["repository:tag"] --> IDX["image index
optional multi-platform"]
    IDX --> AMD["linux/amd64 manifest"]
    IDX --> ARM["linux/arm64 manifest"]
    AMD --> CFG1["config"]
    AMD --> A1["layer A"]
    AMD --> A2["layer B"]
    ARM --> CFG2["config"]
    ARM --> B1["layer A'"]
    ARM --> B2["layer B'"]

When an image supports several platforms, the registry can serve a platform-specific manifest based on the pull request. This is why the same tag can work on an Intel server and an ARM laptop while delivering different layer content.

Inspect the selected platform:

docker image inspect nginx
docker buildx imagetools inspect nginx:alpine

Layers — filesystem changes stacked in order

Many Dockerfile instructions produce a filesystem layer. Imagine:

FROM debian:stable-slim
RUN apt-get update && apt-get install -y ca-certificates
COPY app /usr/local/bin/app

Conceptually:

flowchart TB
    L3["COPY app
adds /usr/local/bin/app"]
    L2["RUN apt-get...
adds packages and metadata"]
    L1["debian base filesystem"]
    L3 --> L2 --> L1

Layers are:

• ordered;
• content-addressed;
• read-only after the image is built;
• shared between images when their content is identical;
• transferred only when the target does not already have the blob.

This makes pulls and storage efficient. Ten images based on the same base do not need ten independent copies of identical base layers.

Deleting a file does not erase it from history

Suppose one layer writes a 500 MB file and a later layer deletes it:

RUN download-large-tool
RUN use-tool-and-delete-it

The final merged filesystem no longer shows the tool, but the earlier layer still contains its bytes. The image can remain roughly 500 MB larger.

Do creation and cleanup in one instruction when the temporary data does not belong in the final result:

RUN download-large-tool \
    && use-tool \
    && rm -rf /tmp/tool

Better still, use a multi-stage build so build tools never enter the final stage.

The same rule is crucial for secrets: deleting a copied credential in a later layer does not remove it from the earlier layer or cache.

Copy-on-write — how a container changes an image view

When a container changes a file inherited from the image, the storage driver copies the file into the container’s writable layer and modifies that copy.

sequenceDiagram
    participant P as Container process
    participant W as Writable layer
    participant I as Read-only image layers

    P->>I: read /etc/example.conf
    I-->>P: file contents
    P->>W: write /etc/example.conf
    W->>I: copy original up if needed
    W-->>P: modified file now shadows image file

The merged view looks like a normal filesystem, but the image below it never changes.

This writable layer is suitable for small ephemeral changes, not for database durability:

• its lifetime is tied to the container;
• copy-on-write can be inefficient for write-heavy workloads;
• it is harder to back up and manage than explicit storage.

Tags, image IDs, and digests are different identifiers

Useful commands:

docker image ls --digests
docker image inspect myapp:1.0
docker image history myapp:1.0
docker container inspect my-container

docker image history is helpful but not a complete security audit. Build metadata, base content, package databases, and layer contents require dedicated inspection.

Pull policy and local cache

For a standalone docker run, Docker normally uses a matching local image if present and pulls only when required by the selected policy.

Be explicit when freshness matters:

docker pull nginx:alpine
docker run --pull=always nginx:alpine
docker run --pull=never myapp:test

Even --pull=always does not make a mutable tag reproducible—it only resolves the tag again. A digest makes the artifact exact.

Image size: useful, but not the only goal

A smaller image can mean:

• less network transfer;
• faster cold starts;
• less storage;
• fewer packages and potentially less attack surface.

But “smallest possible” can harm debugging, compatibility, or maintainability. Favor:

1. a trusted and maintained base;
2. only required runtime content;
3. deterministic package installation;
4. multi-stage builds;
5. clear ownership and update policy.

Do not choose an unfamiliar minimal base merely to win a size contest.

the image contents, but also build speed, reproducibility, and leak risk.

The build has three primary inputs

flowchart LR
    DF["Dockerfile
instructions"]
    CTX["Build context
files builder may access"]
    OPT["Build options
platform, args, secrets, cache"]
    DF --> BK["BuildKit"]
    CTX --> BK
    OPT --> BK
    BK --> IMG["image or other output"]

Run:

docker build -t myapp:dev .

The final . is the build context, not decorative punctuation.

Build instructions cannot normally COPY arbitrary parent files outside the context. That boundary makes remote builders possible and limits what the build definition can access.

A production-minded Dockerfile

# syntax=docker/dockerfile:1

FROM python:3.13-slim

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

WORKDIR /app

COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install --no-compile -r requirements.txt

RUN groupadd --gid 10001 app \
    && useradd --uid 10001 --gid app --no-create-home app

COPY --chown=app:app . .

USER app
EXPOSE 8000

CMD ["python", "app.py"]

Read it as a narrative:

1. choose a base and start a build stage;
2. set runtime defaults;
3. choose the working directory;
4. copy the dependency manifest before frequently changing source;
5. install dependencies with a reusable cache mount;
6. create a non-root runtime identity;
7. copy application files with correct ownership;
8. document the listening port;
9. define the default process.

Dockerfile instructions — what each one means

Build time is not runtime

RUN python -m compileall /app  # happens while building
CMD ["python", "app.py"]       # happens when a container starts

No server started by RUN remains running in the final container. Build steps execute in temporary build containers/sandboxes and record their results.

Build context and .dockerignore

The builder only needs files relevant to the build. A .dockerignore might contain:

.git
.env
.venv
__pycache__/
*.pyc
node_modules/
dist/
coverage/
*.log
README.md

Benefits:

• smaller context transfers;
• fewer accidental cache invalidations;
• less chance of copying credentials;
• clearer build inputs.

.dockerignore is a safety layer, not a substitute for secret management. If a sensitive file exists in the context and the Dockerfile copies it, assume it may end up in cache or image history.

Inspect context-related build output with:

docker build --progress=plain -t myapp:dev .

Build cache — reuse until an input changes

BuildKit evaluates whether it can reuse the result of each build step. Once an early step changes, dependent later work may need to run again.

Bad cache order:

COPY . .
RUN pip install -r requirements.txt

Every source-code edit changes the COPY, forcing dependency installation again.

Better:

COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .

Now normal source edits preserve the expensive dependency layer.

flowchart TD
    A["COPY requirements.txt"] --> B["install dependencies"]
    B --> C["COPY application source"]
    C --> D["final image"]
    X["source file changes"] -.-> C
    Y["requirements change"] -.-> A

The design rule:

Put stable, expensive steps before volatile, cheap steps—while preserving correctness.

Cache invalidation is dependency invalidation

For COPY, relevant file content and metadata affect the result. For RUN, the instruction and its mounted inputs matter; BuildKit does not simply execute every command to ask whether the output would differ.

Force a clean build only when diagnosing or intentionally discarding cache:

docker build --no-cache --pull -t myapp:clean .

--no-cache is not a daily best practice. Correctly structured caching is.

Cache mounts — persistent working cache without image baggage

Package managers repeatedly download the same artifacts. A BuildKit cache mount keeps their cache available across builds without copying it into the final image layer:

RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

Other common targets:

# npm
RUN --mount=type=cache,target=/root/.npm npm ci

# apt package metadata/cache (exact options depend on base image)
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
    --mount=type=cache,target=/var/lib/apt,sharing=locked \
    apt-get update && apt-get install -y --no-install-recommends curl

Cache mounts improve speed. Lockfiles and version constraints provide reproducibility. They solve different problems.

Multi-stage builds — compile in one world, run in another

# syntax=docker/dockerfile:1

FROM golang:alpine AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod go mod download
COPY . .
RUN CGO_ENABLED=0 go build -trimpath -o /out/server ./cmd/server

FROM scratch AS runtime
COPY --from=build /out/server /server
USER 10001:10001
EXPOSE 8080
ENTRYPOINT ["/server"]

flowchart LR
    SRC["source"] --> BUILD["build stage
compiler + modules + tools"]
    BUILD --> BIN["/out/server"]
    BIN --> RUN["runtime stage
binary only"]
    TOOLS["compiler and cache"] -.->|"left behind"| BUILD

Benefits:

• smaller final image;
• fewer runtime packages;
• lower attack surface;
• clean separation between build and runtime requirements.

Build a particular stage for testing or debugging:

docker build --target build -t myapp:build-env .

Name stages. Numeric --from=0 references break easily when stages are reordered.

CMD vs ENTRYPOINT

Use exec-form JSON arrays to preserve argument boundaries and signal delivery.

ENTRYPOINT ["python", "-m", "myapp"]
CMD ["serve", "--port", "8000"]

The runtime command becomes:

python -m myapp serve --port 8000

Then:

docker run myapp:1.0 migrate

replaces CMD arguments and becomes:

python -m myapp migrate

Override an entrypoint for diagnostics:

docker run --rm --entrypoint sh myapp:1.0

If the image has no shell, use purpose-built debugging tooling or a debug build stage.

Secrets during builds

These are unsafe:

ARG TOKEN
ENV TOKEN=$TOKEN
RUN curl -H "Authorization: Bearer $TOKEN" ...

Build args and environment values can leak through metadata, provenance, logs, cache, or history.

Use BuildKit secret mounts:

RUN --mount=type=secret,id=repo_token \
    TOKEN="$(cat /run/secrets/repo_token)" \
    && fetch-private-dependency --token "$TOKEN"

Build:

docker build \
  --secret id=repo_token,src="$PWD/repo-token.txt" \
  -t myapp:dev .

The mounted file exists for that build instruction and is not copied into the output layer by the mount mechanism. Your command must still avoid printing or copying it.

For private Git access, use an SSH mount rather than copying keys:

RUN --mount=type=ssh git clone git@example.com:acme/private.git

Reproducibility vs updates

Two desirable goals pull in opposite directions:

• reproducible: the same inputs produce the same artifact;
• updated: rebuilds receive base-image and package security fixes.

Manage the tension deliberately:

• use application lockfiles;
• pin critical base images by digest;
• automate dependency and base-image update proposals;
• rebuild regularly;
• scan the resulting artifact;
• promote the exact tested digest.

A six-month-old perfectly reproducible image may be perfectly reproducible—and vulnerable.

Multi-platform builds

Buildx can create an image index with platform-specific manifests:

docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -t registry.example.com/acme/myapp:1.0 \
  --push .

Cross-platform builds may use emulation, cross-compilation, or multiple native builder nodes. Test on the real target architecture when behavior or performance is important.

Build review checklist

• Is the base maintained, minimal enough, and pinned according to policy?
• Is the build context intentionally small?
• Are dependency manifests copied before frequently changing source?
• Are package versions locked or constrained?
• Are temporary build tools excluded through multi-stage builds?
• Does the final image run as a non-root user?
• Are CMD/ENTRYPOINT in exec form?
• Are secrets supplied through secret/SSH mounts rather than args or copies?
• Does the image contain only runtime necessities?
• Can source commit, build provenance, and final digest be connected?

answer “how will this particular instance run?”

run is shorthand

These are conceptually equivalent:

docker run --name web -p 8080:80 nginx:alpine

docker create --name web -p 8080:80 nginx:alpine
docker start --attach web

docker create records runtime configuration and prepares the container.

docker start starts the already-configured container process.

If you need different ports, mounts, environment variables, or limits, you normally replace the container. docker start does not re-read new docker run flags.

The lifecycle state machine

stateDiagram-v2
    [*] --> Created: docker create
    Created --> Running: docker start
    Running --> Paused: docker pause
    Paused --> Running: docker unpause
    Running --> Exited: process exits / docker stop
    Exited --> Running: docker start
    Created --> Removed: docker rm
    Exited --> Removed: docker rm
    Running --> Removed: docker rm -f
    Removed --> [*]

The container is running only while its main process is running. Starting a background daemon inside and letting PID 1 exit causes the container to stop.

Inspect all states:

docker ps          # running containers
docker ps -a       # all containers
docker inspect web

Foreground, detached, interactive, and TTY

Common flags:

Examples:

# Foreground one-shot command
docker run --rm alpine:latest echo "hello"

# Interactive shell
docker run --rm -it alpine:latest sh

# Detached server
docker run -d --name web nginx:alpine

-it is a terminal convenience, not a requirement for every exec.

exec vs attach

docker exec -it web sh
docker attach web

• exec starts an additional process inside the existing container namespaces.
• attach connects your terminal to the existing main process I/O.

For diagnostics, exec is usually safer. With attach, terminal key sequences and signals can affect the main process.

Changes made through exec enter the container’s writable layer and vanish when the container is replaced. Record real fixes in source, the Dockerfile, or runtime configuration.

Environment and configuration

Set values:

docker run --rm \
  --env APP_MODE=development \
  --env-file ./app.env \
  myapp:dev

Inside the container:

APP_MODE=development

Cautions:

• environment variables are often visible through container inspection and process

tooling;

• .env files are configuration inputs, not inherently secure secret stores;
• validate required variables on application startup;
• avoid giant, untyped piles of configuration.

Image ENV values are defaults. Runtime --env values override them for that container.

Ports: listening is not publishing

Three separate facts are often confused:

1. the application listens on a container address and port;
2. image metadata may declare EXPOSE 8000;
3. runtime configuration may publish a host port with -p 8080:8000.

docker run -d \
  --name app \
  -p 127.0.0.1:8080:8000 \
  myapp:dev

EXPOSE does not create the mapping. -p does.

The application should usually listen on 0.0.0.0 inside its network namespace. If it listens only on 127.0.0.1 inside the container, forwarded traffic arriving through the container interface cannot reach it.

Logs: stdout and stderr are the contract

docker logs web
docker logs --follow --tail 100 --timestamps web

Docker captures the main process’s stdout/stderr through the configured logging driver. A log written only to /var/log/app.log inside the container will not automatically appear in docker logs.

Operational pattern:

• application writes structured logs to stdout/stderr;
• runtime logging driver collects them;
• external log system stores, searches, and retains them;
• log rotation or remote delivery prevents unbounded local disk use.

docker logs availability and behavior depend on the logging driver.

Inspecting a live container

# Full low-level configuration and state
docker inspect web

# Selected values
docker inspect \
  --format '{{.State.Status}} {{.State.ExitCode}} {{.State.OOMKilled}}' \
  web

# Processes
docker top web

# Live CPU, memory, network, and block I/O
docker stats web

# Files changed in writable layer
docker diff web

# Port mappings
docker port web

Inspection answers what Docker configured. Application-level health still requires application-aware checks.

Exit codes

The main process exit code becomes the container exit code.

Exit 137 often suggests SIGKILL, possibly an OOM kill or forced removal, but do not guess. Inspect:

docker inspect \
  --format 'exit={{.State.ExitCode}} oom={{.State.OOMKilled}} error={{.State.Error}}' \
  web

Correlate with daemon and host logs.

Stopping well

docker stop --timeout 20 web
docker kill --signal=SIGHUP web

stop is graceful first, forceful later. kill can send a selected signal immediately; with no signal option it sends SIGKILL.

Common shutdown failure:

CMD ["sh", "-c", "python app.py"]

The shell is PID 1 and may not forward SIGTERM correctly. Prefer:

CMD ["python", "app.py"]

If a wrapper is necessary, end it with exec:

exec python app.py

Restart policies

docker run -d --restart=unless-stopped --name app myapp:1.0

Restart policies handle process exit. They do not repair a deadlock while PID 1 stays alive, fix a corrupt dependency, distribute replicas across hosts, or replace an unhealthy container merely because its health status changed.

Use backoff and monitoring around crash loops. Fast infinite restart loops can hide the original error and consume resources.

Health checks

Add image-level health intent:

HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
  CMD ["python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=2)"]

Inspect:

docker ps
docker inspect --format '{{json .State.Health}}' app

Health states are:

• starting;
• healthy;
• unhealthy.

A useful health check is:

• local to the service;
• cheap;
• time-bounded;
• representative of the service’s ability to do useful work;
• independent enough not to create cascading failures.

Crucial nuance:

Docker Engine records health status, but a standalone unhealthy container is not

automatically restarted solely because it is unhealthy.

Compose can wait for a dependency marked service_healthy, and orchestrators can use health information according to their own policies.

Resource controls

docker run -d \
  --name app \
  --memory=512m \
  --memory-swap=512m \
  --cpus=1.5 \
  --pids-limit=200 \
  myapp:1.0

Why set limits:

• one process cannot consume all host memory;
• fork bombs have a process ceiling;
• capacity assumptions become explicit;
• noisy-neighbor effects shrink.

Why monitor after setting them:

• a too-low memory limit causes OOM kills;
• CPU throttling can cause latency without process failure;
• limits do not reserve physical capacity;
• Desktop’s VM may add a higher-level resource ceiling.

Test under realistic load. Limits chosen by guesswork merely convert one failure mode into another.

Cleanup and object lifetimes

docker stop web
docker rm web
docker image rm myapp:old
docker volume rm unused-volume
docker network rm unused-network

--rm is excellent for disposable commands:

docker run --rm myapp:dev run-tests

Prune commands are broad and can remove reusable cache or stopped resources:

docker system df
docker container prune
docker image prune
docker builder prune
docker system prune

Inspect first. Avoid --volumes unless you have positively identified disposable data.

“From which network namespace is this address being used?”

Four communication paths

flowchart LR
    B["Browser on host"] -->|"localhost:8080"| HP["host port 8080"]
    HP -->|"published mapping"| W["web container :80"]
    W -->|"http://api:8000"| A["api container :8000"]
    A -->|"outbound through bridge"| EXT["external service"]

localhost always means “this network namespace”

Inside the API container:

localhost:8000 = API container itself

It does not mean:

• the Docker host;
• the database container;
• your laptop;
• another Compose service.

This mistake is so common that it deserves a rule:

To reach another container, use its DNS name and container port, not localhost

and not its published host port.

The default bridge vs a user-defined bridge

If you run a container without --network, Docker attaches it to the default bridge on Linux Engine. Prefer a user-defined bridge for related applications:

docker network create app-net

docker run -d \
  --name api \
  --network app-net \
  myapi:dev

docker run --rm \
  --network app-net \
  curlimages/curl \
  http://api:8000/health

User-defined bridges provide:

• automatic DNS resolution by container name/alias;
• an explicit isolation boundary;
• easier connection/disconnection;
• per-network configuration.

Compose creates a user-defined project network by default.

What a bridge network creates

At a simplified level:

flowchart TB
    subgraph HOST["Docker host"]
        BR["Linux bridge
gateway address"]
        V1["virtual Ethernet end"]
        V2["virtual Ethernet end"]
        FW["routing / firewall / NAT"]
        BR --- V1
        BR --- V2
        BR --> FW
    end
    V1 --- C1["container A
eth0"]
    V2 --- C2["container B
eth0"]
    FW --> OUT["host network / internet"]

Each container receives its own network namespace and virtual interface. Docker connects the host-side interface to a bridge and supplies routing and DNS behavior. Exact implementation differs across platforms and configurations, but the namespace model remains useful.

DNS-based service discovery

Containers on the same user-defined network can reach:

http://api:8000
postgresql://db:5432/app
redis://cache:6379

Names are stable even when container IP addresses change. Use names in application configuration and allow reconnect/retry behavior.

Do not bake container IPs into configuration. Container replacement can assign a new address.

Inspect:

docker network inspect app-net
docker inspect --format '{{json .NetworkSettings.Networks}}' api

Publishing ports

Syntax:

HOST_IP:HOST_PORT:CONTAINER_PORT/PROTOCOL

Examples:

# Reachable on all host interfaces by default
docker run -p 8080:80 nginx

# Reachable only from the host loopback interface
docker run -p 127.0.0.1:8080:80 nginx

# Publish UDP
docker run -p 5353:53/udp dns-server

# Let Docker select an available host port
docker run -p 127.0.0.1::80 nginx

Publishing without a host IP commonly exposes the port on all host interfaces. Bind to 127.0.0.1 for local-only development unless remote access is intentional.

List the actual mapping:

docker port <container>

EXPOSE, expose, and ports

Containers on the same network can connect to a listening container port without publishing it to the host.

One request, end to end

For:

docker run -d --name web -p 127.0.0.1:8080:80 nginx

sequenceDiagram
    participant B as Browser
    participant H as Host localhost:8080
    participant D as Docker network rules
    participant C as Container eth0:80
    participant N as nginx process

    B->>H: TCP connect
    H->>D: match published port
    D->>C: forward to container address:80
    C->>N: deliver socket traffic
    N-->>C: HTTP response
    C-->>D: response packets
    D-->>H: translate connection
    H-->>B: HTTP response

If it fails, test each boundary instead of changing random flags:

1. Is the process running?
2. Is it listening on 0.0.0.0:80 inside the container?
3. Is the port published?
4. Is the host address correct?
5. Is a host firewall or another process interfering?

Reaching the host from a container

On Docker Desktop, this conventional name reaches host services:

host.docker.internal

On Linux Engine, support/configuration differs. A common explicit mapping is:

docker run \
  --add-host=host.docker.internal:host-gateway \
  myapp:dev

Better architecture often puts dependencies in the same declared network or at a real routable service endpoint, rather than relying on a special host alias.

Network drivers

For most single-host applications, use a user-defined bridge.

host networking changes the model: the container does not get an isolated network stack, and -p mappings are not meaningful. Desktop support and behavior have platform-specific constraints.

Network isolation

Put only services that must communicate on the same network.

services:
  proxy:
    networks: [front]
  api:
    networks: [front, back]
  db:
    networks: [back]

networks:
  front:
  back:
    internal: true

flowchart LR
    INTERNET["client"] --> PROXY["proxy"]
    subgraph FRONT["front network"]
        PROXY --> API["api"]
    end
    subgraph BACK["back network"]
        API --> DB["database"]
    end

The proxy has no reason to connect directly to the database. Network segmentation reduces unintended paths, though it does not replace authentication or encryption.

Debugging networking systematically

From the host:

docker ps
docker port api
docker logs api
docker network inspect app-net

From a diagnostic container on the same network:

docker run --rm -it --network app-net nicolaka/netshoot

Then check:

getent hosts api
curl -v http://api:8000/health
nc -vz db 5432

Prefer a separate diagnostic container when production images intentionally omit shells and network tools. Do not bloat every runtime image solely for emergency debugging.

Persistent state needs an explicitly different lifetime.

The writable layer is ephemeral by design

flowchart TD
    I["image layers
read-only and reusable"] --> C1["container 1 writable layer"]
    I --> C2["container 2 writable layer"]
    C1 -->|"docker rm"| G1["deleted"]
    C2 -->|"still exists"| G2["its data remains"]

Stopping and starting the same container preserves its writable layer. Removing and recreating the container does not.

This distinction explains a classic surprise:

docker stop db
docker start db     # same container: writable-layer data remains

docker rm db
docker run ...      # new container: old writable-layer data is absent

Do not mistake survival across stop for durable storage.

The three mount types

flowchart LR
    C["container path"]
    V["named volume
Docker manages location"] --> C
    B["bind mount
you choose host path"] --> C
    T["tmpfs
memory only"] --> C

Prefer the explicit --mount syntax in scripts and documentation:

docker run --mount type=volume,src=app-data,dst=/data myapp
docker run --mount type=bind,src="$PWD/config",dst=/app/config,readonly myapp
docker run --mount type=tmpfs,dst=/tmp,tmpfs-size=64m myapp

The shorter -v syntax is common, but its compact parsing can hide mistakes.

Named volumes

Create and inspect:

docker volume create app-data
docker volume inspect app-data
docker volume ls

Use:

docker run -d \
  --name db \
  --mount type=volume,src=app-data,dst=/var/lib/example \
  example-db:1.0

Benefits:

• lifetime independent of a particular container;
• Docker handles the host-side location;
• easy attachment to replacement containers;
• volume drivers can integrate external storage.

Limitations:

• “persistent” does not mean backed up;
• local volumes remain tied to one Docker host;
• concurrent writers require application/storage support;
• a volume preserves corruption just as faithfully as good data.

Bind mounts

Development example:

docker run --rm \
  --mount type=bind,src="$PWD",dst=/workspace \
  --workdir /workspace \
  node:alpine \
  npm test

Read-only configuration:

docker run \
  --mount type=bind,src="$PWD/nginx.conf",dst=/etc/nginx/nginx.conf,readonly \
  nginx

Bind mounts are direct host coupling:

• host path must exist for --mount;
• host permissions and ownership matter;
• the container can modify the host when mounted read-write;
• the path must be shared into Docker Desktop’s VM;
• remote daemon paths refer to the daemon host.

Mount the narrowest path with the least access necessary.

Mounting hides image content

Suppose the image contains /app/defaults/config.json. Mounting a volume at /app obscures the entire image directory for that container:

image /app contents      hidden by mount at /app
mounted /app contents    visible

This does not delete the image content. Remove the mount by recreating the container and the original files reappear.

Mount at narrow, intentional paths such as /app/config or /data.

Docker can populate a new empty named volume from existing image content at the mount destination; understand the image and volume driver behavior before using this as an initialization mechanism.

tmpfs

Use tmpfs when data should not survive the container and should avoid the writable layer:

docker run --rm \
  --mount type=tmpfs,dst=/run/secrets,tmpfs-size=16m,tmpfs-mode=0700 \
  myapp

Good uses:

• temporary scratch files;
• runtime-generated sensitive material;
• high-churn disposable state.

tmpfs consumes memory and may contribute to memory pressure. It is not a database backup strategy.

Ownership and permissions

Files are governed by numeric UID/GID. A username inside the container is merely a mapping to numbers in /etc/passwd.

Common failure:

host source owned by UID 501
container process runs as UID 10001
bind-mounted directory is not writable

Possible solutions:

• match or parameterize development UIDs where appropriate;
• set ownership during image build with COPY --chown;
• initialize volume ownership before dropping privilege;
• use read-only mounts when writes are unnecessary;
• avoid chmod 777, which discards the boundary instead of understanding it.

On Docker Desktop, filesystem sharing adds a translation layer, so behavior and performance may differ from native Linux.

Backing up a named volume

The correct backup method is application-specific. A database often needs a logical dump or snapshot coordination, not a blind file copy while it is writing.

For simple quiesced file data, a helper container can archive the mounted volume:

docker run --rm \
  --mount type=volume,src=app-data,dst=/data,readonly \
  --mount type=bind,src="$PWD/backups",dst=/backup \
  alpine \
  tar -czf /backup/app-data.tgz -C /data .

Restore only into an identified empty/replacement volume, validate the archive first, and test the recovery procedure. An untested backup is a theory.

Compose storage

services:
  db:
    image: postgres:17-alpine
    volumes:
      - db-data:/var/lib/postgresql/data
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql:ro
    tmpfs:
      - /tmp

volumes:
  db-data:

docker compose down removes project containers and networks but normally retains named volumes. This command removes the volume too:

docker compose down --volumes

Treat it as destructive if the volume contains data you care about.

Choosing storage

flowchart TD
    Q1{"Must data survive
container replacement?"}
    Q1 -->|No| Q2{"Should it stay
only in memory?"}
    Q2 -->|Yes| TMP["tmpfs"]
    Q2 -->|No| WL["writable layer"]
    Q1 -->|Yes| Q3{"Must the host/user edit
an exact path directly?"}
    Q3 -->|Yes| BIND["bind mount"]
    Q3 -->|No| Q4{"Single-host Docker-managed
storage sufficient?"}
    Q4 -->|Yes| VOL["named volume"]
    Q4 -->|No| EXT["external storage/service
or volume driver"]

For production databases, prefer the data platform and backup model appropriate to your reliability requirements. “It is in a Docker volume” is not a complete storage architecture.

services, networks, volumes, configuration, and relationships.

The problem Compose solves

One container is easy to start from memory. A useful application may need:

• an API built from local source;
• a database image;
• an internal network;
• a public port;
• a persistent volume;
• environment configuration;
• health checks and startup dependencies.

Imperative setup:

create network
create volume
build image
run database with 8 flags
wait for database
run API with 12 flags
remember every name for cleanup

Compose stores that intent in compose.yaml.

flowchart TD
    Y["compose.yaml"] --> C["docker compose"]
    C --> I["build/pull images"]
    C --> N["create project networks"]
    C --> V["create project volumes"]
    C --> S1["create/start app container"]
    C --> S2["create/start db container"]

Compose is a client of the same Docker Engine. It creates ordinary Docker objects and labels them as members of a project.

The Compose Specification

Modern Compose uses the unified Compose Specification. A top-level version: key is not required for current Compose files.

Core top-level elements:

name: example

services:
  app: {}
  db: {}

networks:
  front: {}
  back: {}

volumes:
  db-data: {}

secrets:
  db-password: {}

configs:
  proxy-config: {}

Most applications need only services, plus perhaps volumes and networks.

A complete example

name: notes

services:
  app:
    build:
      context: .
      target: runtime
    image: notes-app:dev
    init: true
    environment:
      APP_MODE: development
      DATABASE_URL: postgresql://notes@db:5432/notes
      DB_PASSWORD_FILE: /run/secrets/db-password
    secrets:
      - db-password
    depends_on:
      db:
        condition: service_healthy
        restart: true
    ports:
      - "127.0.0.1:8080:8000"
    networks:
      - front
      - back
    read_only: true
    tmpfs:
      - /tmp
    restart: unless-stopped

  db:
    image: postgres:17-alpine
    environment:
      POSTGRES_DB: notes
      POSTGRES_USER: notes
      POSTGRES_PASSWORD_FILE: /run/secrets/db-password
    secrets:
      - db-password
    volumes:
      - db-data:/var/lib/postgresql/data
    networks:
      - back
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U notes -d notes"]
      interval: 5s
      timeout: 3s
      retries: 10
      start_period: 10s
    restart: unless-stopped

networks:
  front:
  back:
    internal: true

volumes:
  db-data:

secrets:
  db-password:
    file: ./secrets/db-password.txt

What this declares:

• app is built locally and tagged;
• only app publishes a host port, and only on loopback;
• app reaches the database at DNS name db, port 5432;
• database data outlives database container replacement;
• the database password appears as a runtime file;
• the application waits for database health before creation;
• the back network has no direct external route;
• application root filesystem is read-only except for /tmp and declared mounts.

Local Compose file-backed secrets improve separation from images and environment variables, but they are not automatically an enterprise secret vault. Protect the source file and use a proper secret-management system where required.

Service is a model; container is an instance

In Compose, app is a service definition. Running the project creates one or more containers from it.

flowchart LR
    S["service: app"] --> C1["notes-app-1"]
    S --> C2["notes-app-2"]
    S --> C3["notes-app-3"]

Scale:

docker compose up -d --scale app=3

Do not publish the same fixed host port from every replica. Put a reverse proxy/load balancer in front or let Docker assign distinct host ports. Compose scaling remains single-Engine operation, not multi-host scheduling.

Project names create a namespace

Compose derives a project name from, in priority order, explicit options/configuration and commonly the directory name. The project name influences generated object names and labels.

Set it explicitly:

docker compose --project-name notes-dev up -d

Two copies can coexist if their host ports do not collide:

notes-alice_app_1
notes-bob_app_1

Avoid container_name unless an external constraint truly requires it. It defeats normal scaling and couples clients to a particular instance instead of a service.

The everyday command loop

# Parse, merge, interpolate, and render the effective model
docker compose config

# Build images
docker compose build

# Create/start and detach
docker compose up -d

# Status
docker compose ps

# Follow logs
docker compose logs -f --tail=100

# Run a one-off command using a service definition
docker compose run --rm app python -m pytest

# Start a command inside an existing service container
docker compose exec app sh

# Stop/remove project containers and default networks
docker compose down

docker compose config is one of the best debugging commands. It shows the model after file merging and variable interpolation.

up, start, run, and exec

After changing environment, ports, mounts, or image content, use up so Compose can recreate what changed. restart only restarts existing container configuration; it does not apply most Compose file changes.

docker compose up -d --build

Startup order is not readiness

depends_on short syntax establishes dependency order:

depends_on:
  - db

But a running database process may still be initializing. Use a health condition:

depends_on:
  db:
    condition: service_healthy

sequenceDiagram
    participant C as Compose
    participant D as db container
    participant H as health check
    participant A as app container

    C->>D: create and start
    loop until healthy or failed
        C->>H: observe health status
        H-->>C: starting / healthy
    end
    C->>A: create and start

Still make applications resilient. Dependencies can disappear after startup. Network clients need timeouts, bounded retries, reconnects, and useful error reporting.

Variables: interpolation vs container environment

These happen at different layers.

Compose interpolation:

services:
  app:
    image: "example/app:${APP_TAG:-dev}"

APP_TAG is read while Compose constructs its model.

Container environment:

services:
  app:
    environment:
      APP_MODE: "${APP_MODE:-development}"

The rendered value is passed into the container.

Inspect without starting:

docker compose config
docker compose config --environment

Do not assume a project .env file automatically becomes the container environment. It commonly supplies interpolation values; use environment or env_file to pass selected values into the container.

Multiple Compose files

Keep a portable base, then apply environment-specific changes:

docker compose \
  -f compose.yaml \
  -f compose.dev.yaml \
  config

Example development override:

services:
  app:
    build:
      target: development
    environment:
      APP_MODE: development
    volumes:
      - .:/app
    command: ["python", "app.py", "--reload"]

Always render the merged configuration; list and mapping merge behavior can surprise you.

Profiles make optional tools explicit:

services:
  adminer:
    image: adminer
    profiles: ["debug"]

docker compose --profile debug up -d

What Compose does not do

Compose does not automatically provide:

• multi-host scheduling;
• cross-host failover;
• cluster-wide desired-state reconciliation;
• zero-downtime rolling updates;
• production secret management by itself;
• backups;
• application observability;
• security merely because configuration is in YAML.

It is excellent for local development, integration tests, demos, CI jobs, and single-host applications. Use an orchestrator or platform when the availability and scale requirements demand one.

inherit.

Threat model first

Ask what you are protecting:

• the host from a compromised container;
• one container from another;
• credentials from image consumers;
• the registry from unauthorized writes;
• users from a malicious dependency;
• production from an unreviewed image;
• data from accidental deletion.

Controls only make sense against a threat.

flowchart TD
    SRC["source + dependencies"] --> BUILD["builder"]
    BUILD --> IMG["image"]
    IMG --> REG["registry"]
    REG --> HOST["Docker host"]
    HOST --> PROC["container process"]

    T1["dependency compromise"] -.-> SRC
    T2["secret leak"] -.-> BUILD
    T3["tag replacement"] -.-> REG
    T4["daemon/socket abuse"] -.-> HOST
    T5["application exploit"] -.-> PROC

Container security includes the whole chain, not just runtime flags.

High-impact runtime controls

1. Run as non-root

Dockerfile:

RUN groupadd --gid 10001 app \
    && useradd --uid 10001 --gid app --no-create-home app
USER 10001:10001

Verify:

docker run --rm myapp:1.0 id

Non-root does not neutralize every exploit, but it reduces what a compromised process can do inside its namespace and on writable mounts.

2. Drop capabilities

docker run \
  --cap-drop=ALL \
  --cap-add=NET_BIND_SERVICE \
  myapp:1.0

Add back only what the application demonstrably needs.

3. Prevent privilege escalation

docker run --security-opt=no-new-privileges:true myapp:1.0

4. Make the root filesystem read-only

docker run \
  --read-only \
  --tmpfs /tmp:rw,noexec,nosuid,size=64m \
  --mount type=volume,src=app-data,dst=/data \
  myapp:1.0

Every writable location becomes explicit.

5. Set resource and process limits

docker run \
  --memory=512m \
  --cpus=1 \
  --pids-limit=200 \
  myapp:1.0

6. Keep the default seccomp profile

Docker’s default seccomp policy blocks a selection of risky system calls while preserving broad compatibility. Do not use seccomp=unconfined casually.

7. Avoid --privileged

--privileged intentionally removes many isolation controls and grants broad device access. It is not a generic fix for permissions errors. Identify the exact required capability, device, mount, or policy instead.

The Docker socket is an administrative interface

This pattern is extremely powerful:

volumes:
  - /var/run/docker.sock:/var/run/docker.sock

It is also dangerous. A process with daemon control can often:

• start privileged containers;
• mount host filesystems;
• read other container configuration;
• alter networks;
• stop workloads.

Use narrowly scoped proxies or purpose-built APIs when automation needs limited Docker information. Do not expose an unauthenticated Docker TCP API.

Rootless mode can reduce daemon and container privilege on supported Linux setups, but it has prerequisites and feature constraints. It is one defense layer, not a reason to ignore application security.

Secrets: keep them out of images and logs

Never put production secrets in:

• Dockerfile ENV;
• ARG;
• copied .env files;
• image labels;
• image tags;
• build command lines or logs;
• source control.

Use:

• BuildKit secret mounts for build-time access;
• a runtime secret manager or mounted secret file;
• short-lived credentials where possible;
• separate credentials per environment and workload;
• rotation and revocation procedures.

Assume anyone who can pull an image can inspect all of its layers and metadata.

Supply-chain controls

Pin and trace

Connect:

source commit → build run → image digest → deployment

Use immutable digests for promotion. Record meaningful OCI labels:

LABEL org.opencontainers.image.source="https://example.com/acme/app" \
      org.opencontainers.image.revision="GIT_COMMIT_INJECTED_BY_CI"

Do not inject untrusted strings into build instructions without validation.

Scan

Scan:

• operating-system packages;
• language dependencies;
• secrets;
• misconfiguration;
• licenses where policy requires it.

A scanner reports evidence, not absolute safety. Results need severity, reachability, fix availability, exception ownership, and deadlines.

Generate an SBOM

A software bill of materials records components in the artifact. It helps answer:

“Are any deployed images affected by this newly disclosed library issue?”

Sign and verify

Artifact signatures and attestations can prove which identity produced an image and which process claims to have built it. Verification policy must run at deployment time; a signature no one checks is decoration.

Rebuild

Images do not patch themselves. Rebuild with updated base layers and dependencies, retest, then redeploy the new digest.

Base image policy

Choose bases with:

• an accountable maintainer;
• predictable update cadence;
• compatible libc and system libraries;
• a clear support lifetime;
• only the runtime content you need.

Pinning a base by digest prevents surprise changes:

FROM python:3.13-slim@sha256:...

But pinning also prevents automatic fixes. Use automated update proposals so the digest changes through review and testing.

Avoid installing “useful just in case” tools into the final runtime image.

Reliability on one Docker host

For a production single-host deployment, plan at least:

• daemon start at boot;
• explicit restart policies;
• resource limits;
• health monitoring outside the container;
• log rotation/collection;
• disk and inode monitoring;
• image and build-cache cleanup policy;
• tested data backups and restore;
• configuration/secret rotation;
• host security updates;
• an upgrade and rollback procedure;
• external availability checks.

Restart policy is not high availability. If the host fails, all local containers and local volumes are unavailable.

Observability

Three basic signals:

Docker adds infrastructure clues:

docker stats
docker events
docker inspect
docker system df

Application observability still needs explicit instrumentation and external retention. Container IDs are ephemeral; enrich telemetry with service, version, environment, image digest, and request identity.

Deployment patterns

Replace, do not patch

build new digest → test → start replacement → verify → shift traffic → remove old

Plain Compose on a single host does not automatically implement a zero-downtime rolling update. Design a reverse-proxy or orchestration strategy if this matters.

Database migrations

Treat schema change as a controlled operation:

• make migrations backward-compatible where possible;
• run them as an explicit release step or one-off task;
• do not let every replica race to migrate;
• plan rollback or forward-fix behavior;
• back up before destructive changes.

Configuration changes

Environment and mount changes generally require container recreation. Make application startup validate configuration and fail with a precise error.

Security review checklist

• Does the final image run as non-root?
• Are capabilities dropped?
• Is --privileged absent?
• Is the root filesystem read-only where practical?
• Are writable mounts narrow and intentional?
• Are published ports bound only where needed?
• Are CPU, memory, and PID limits set?
• Are build and runtime secrets outside layers and logs?
• Is Docker socket access absent or tightly controlled?
• Are images scanned, traceable, and promoted by digest?
• Are base images and dependencies updated on a schedule?
• Can data be restored from a tested backup?
• Are host, daemon, registry, and application logs monitored?

chain: build, distribute, create, start, connect, observe, replace.

Trace 1: source → image

Assume:

docker build -t registry.example.com/acme/api:1.4.2 .

sequenceDiagram
    autonumber
    participant U as Developer
    participant C as Docker client
    participant B as BuildKit
    participant R as Base registry
    participant S as Local content store

    U->>C: docker build -t api:1.4.2 .
    C->>B: Dockerfile + context + options
    B->>B: parse stages and dependency graph
    B->>R: resolve/pull missing base content
    loop each required build operation
        B->>B: compute inputs and cache key
        alt reusable result exists
            B->>S: load cached result
        else cache miss
            B->>B: execute operation in build sandbox
            B->>S: store output/content
        end
    end
    B->>S: write config + manifest + tag
    B-->>C: final image identifier
    C-->>U: build succeeded

What persists:

• content-addressed filesystem layers;
• image config and manifest;
• a local tag pointing to the result;
• cache metadata/results according to builder configuration.

What should not persist:

• temporary secret mounts;
• compiler toolchain left in an earlier build stage;
• files excluded from the build context;
• processes started during RUN.

Trace 2: image → registry

docker push registry.example.com/acme/api:1.4.2

sequenceDiagram
    participant C as Docker client/Engine
    participant A as Registry auth
    participant R as Registry

    C->>A: request authorization
    A-->>C: scoped token
    C->>R: check/upload missing blobs
    R-->>C: blobs accepted/already exist
    C->>R: upload config and manifest
    R-->>C: manifest digest sha256:...

Shared layers may already exist, so only missing blobs transfer. The final manifest digest is the immutable release identity.

A mature delivery pipeline then:

1. scans the image;
2. generates an SBOM/provenance;
3. signs or attests to the digest;
4. promotes that same digest;
5. enforces verification policy before deployment.

Trace 3: docker run → running process

docker run -d \
  --name api \
  --network app-net \
  --mount type=volume,src=api-data,dst=/data \
  --env-file ./api.env \
  --memory=512m \
  -p 127.0.0.1:8080:8000 \
  registry.example.com/acme/api@sha256:...

sequenceDiagram
    autonumber
    participant CLI as docker CLI
    participant D as dockerd
    participant REG as registry
    participant FS as image/storage driver
    participant NET as network driver
    participant RT as container runtime
    participant K as kernel

    CLI->>D: create request + exact image digest
    D->>REG: fetch missing manifest/config/layers
    D->>FS: assemble image rootfs + writable layer
    D->>FS: mount api-data at /data
    D->>NET: attach app-net + publish 8080:8000
    D->>RT: OCI bundle + env + limits + command
    RT->>K: create namespaces and cgroup
    RT->>K: set credentials/security policy
    RT->>K: exec image command as PID 1
    K-->>D: process running
    D-->>CLI: container ID

At this point:

• image layers are still read-only;
• one writable layer belongs to this container;
• /data belongs to the named volume;
• environment is recorded in container configuration;
• an isolated network interface has an address on app-net;
• host loopback port 8080 forwards to container port 8000;
• resource controls apply through the kernel.

Trace 4: one HTTP request

sequenceDiagram
    autonumber
    participant B as Browser
    participant H as Host 127.0.0.1:8080
    participant N as Docker networking
    participant A as API process :8000
    participant V as /data volume
    participant L as stdout log stream

    B->>H: GET /
    H->>N: match published port
    N->>A: deliver TCP stream
    A->>V: read/update durable state
    V-->>A: result
    A->>L: write request log
    A-->>N: HTTP 200 + body
    N-->>H: translated response
    H-->>B: HTTP response

Observe different layers:

curl http://127.0.0.1:8080/
docker logs api
docker stats api
docker inspect api
docker volume inspect api-data

No single command tells the entire story. Each command observes one boundary.

Trace 5: configuration change → replacement

Suppose api.env changes. Restarting the process with old container configuration does not apply a new docker run model. Replace the container:

flowchart LR
    OLD["old container
image A + config X"] --> STOP["graceful stop"]
    NEWCFG["new config Y"] --> CREATE["create replacement
same or new image"]
    CREATE --> START["health check"]
    START --> TRAFFIC["serve traffic"]
    STOP --> REMOVE["remove old writable layer"]
    VOL["named volume"] --> OLD
    VOL --> CREATE

The container identity changes. The named volume identity does not.

With Compose:

docker compose up -d

Compose compares the effective service model and recreates containers when relevant configuration or image inputs differ.

Trace 6: graceful shutdown

sequenceDiagram
    participant O as Operator/Compose
    participant D as Docker Engine
    participant P as PID 1
    participant DB as Dependency

    O->>D: stop container
    D->>P: SIGTERM
    P->>P: stop accepting new work
    P->>P: finish bounded in-flight work
    P->>DB: close/flush
    P-->>D: exit 0
    D-->>O: stopped

If the process does not exit before the grace period, Docker sends SIGKILL. No cleanup handler can run after SIGKILL.

The final mental model

flowchart TD
    subgraph DELIVERY["DELIVERY"]
        SRC["source + lockfiles
Dockerfile"] --> BUILD["BuildKit"]
        BUILD --> IMG["immutable image digest"]
        IMG --> REG["registry"]
    end

    subgraph RUNTIME["RUNTIME"]
        REG --> HOST["Docker Engine"]
        CFG["runtime config + secrets"] --> HOST
        NET["networks + published ports"] --> HOST
        STORE["volumes + binds + tmpfs"] --> HOST
        LIMIT["limits + security policy"] --> HOST
        HOST --> PROC["isolated process
PID 1"]
    end

    PROC --> LOG["stdout/stderr"]
    PROC --> STORE

If you remember only eight things

1. A container is an isolated process, not a tiny VM.
2. An image is immutable content; a container is one runtime instance.
3. Tags move; digests identify exact content.
4. Build context and Dockerfile order define both safety and cache efficiency.
5. localhost inside a container means that container.
6. Durable state needs a lifetime outside the container writable layer.
7. Health, restart, readiness, and high availability are different mechanisms.
8. Replace containers from reviewed inputs; do not patch them by hand.

every boundary, break it deliberately, and repair it from declared inputs.

Lab goal

You will build:

flowchart LR
    B["browser / curl
127.0.0.1:18080"] --> P["nginx proxy
container port 80"]
    P -->|"Docker DNS: app:8000"| A["Python app
non-root · read-only rootfs"]
    A --> V[("named volume
SQLite visit count")]

The application uses only Python’s standard library, so no language dependencies need downloading during the app build. Docker still needs network access to pull base images if they are not already local.

Estimated time: 45–60 minutes.

Prerequisites

docker version
docker compose version

Both should report reachable Docker Engine/Compose versions. If Client works but Server fails, start Docker Desktop or the Docker Engine service.

Create the project

Make a directory named docker-lab with:

docker-lab/
├── app.py
├── compose.yaml
├── Dockerfile
├── .dockerignore
└── nginx.conf

app.py

import json
import os
import signal
import socket
import sqlite3
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer


PORT = int(os.environ.get("PORT", "8000"))
MESSAGE = os.environ.get("APP_MESSAGE", "Hello from Docker")
DATABASE = os.environ.get("DATABASE", "/data/visits.db")
stopping = False


def connect():
    connection = sqlite3.connect(DATABASE, timeout=5)
    connection.execute("PRAGMA journal_mode=WAL")
    return connection


def initialize():
    os.makedirs(os.path.dirname(DATABASE), exist_ok=True)
    with connect() as connection:
        connection.execute(
            "CREATE TABLE IF NOT EXISTS counters "
            "(name TEXT PRIMARY KEY, value INTEGER NOT NULL)"
        )
        connection.execute(
            "INSERT OR IGNORE INTO counters(name, value) VALUES('visits', 0)"
        )


class Handler(BaseHTTPRequestHandler):
    def send_json(self, status, payload):
        body = json.dumps(payload, indent=2).encode()
        self.send_response(status)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    def do_GET(self):
        if self.path == "/health":
            try:
                with connect() as connection:
                    connection.execute("SELECT 1").fetchone()
                self.send_json(200, {"status": "healthy"})
            except sqlite3.Error as error:
                self.send_json(503, {"status": "unhealthy", "error": str(error)})
            return

        if self.path != "/":
            self.send_json(404, {"error": "not found"})
            return

        with connect() as connection:
            connection.execute(
                "UPDATE counters SET value = value + 1 WHERE name = 'visits'"
            )
            visits = connection.execute(
                "SELECT value FROM counters WHERE name = 'visits'"
            ).fetchone()[0]

        self.send_json(
            200,
            {
                "message": MESSAGE,
                "hostname": socket.gethostname(),
                "visits": visits,
            },
        )

    def log_message(self, format, *args):
        print(
            json.dumps(
                {
                    "client": self.client_address[0],
                    "request": format % args,
                }
            ),
            flush=True,
        )


def request_shutdown(signum, frame):
    global stopping
    print(json.dumps({"event": "shutdown", "signal": signum}), flush=True)
    stopping = True


initialize()
signal.signal(signal.SIGTERM, request_shutdown)
signal.signal(signal.SIGINT, request_shutdown)

server = ThreadingHTTPServer(("0.0.0.0", PORT), Handler)
server.timeout = 1
print(json.dumps({"event": "started", "port": PORT}), flush=True)

while not stopping:
    server.handle_request()

server.server_close()
print(json.dumps({"event": "stopped"}), flush=True)

Why the loop uses handle_request() with a timeout: the signal handler can set a flag, the loop wakes within one second, and PID 1 exits cleanly. Calling

server.shutdown() from the same thread that runs serve_forever() can deadlock.

Dockerfile

# syntax=docker/dockerfile:1

FROM python:3.13-slim

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

WORKDIR /app

RUN groupadd --gid 10001 app \
    && useradd --uid 10001 --gid app --no-create-home app \
    && mkdir -p /data \
    && chown app:app /data

COPY --chown=app:app app.py .

USER 10001:10001
EXPOSE 8000

HEALTHCHECK --interval=5s --timeout=2s --start-period=3s --retries=5 \
  CMD ["python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=1)"]

CMD ["python", "app.py"]

.dockerignore

.git
.env
__pycache__/
*.pyc
*.db
backups/
compose*.yaml
nginx.conf
README.md

Only app.py and Dockerfile are needed in this build context.

nginx.conf

events {}

http {
    access_log /dev/stdout;
    error_log /dev/stderr warn;

    server {
        listen 80;

        location / {
            proxy_pass http://app:8000;
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
    }
}

compose.yaml

name: docker-course-lab

services:
  proxy:
    image: nginx:alpine
    depends_on:
      app:
        condition: service_healthy
        restart: true
    ports:
      - "127.0.0.1:18080:80"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    networks:
      - front
    restart: unless-stopped

  app:
    build:
      context: .
    image: docker-course-app:dev
    init: true
    environment:
      APP_MESSAGE: "Hello from a declared Compose stack"
      DATABASE: /data/visits.db
    expose:
      - "8000"
    volumes:
      - app-data:/data
    networks:
      - front
    read_only: true
    tmpfs:
      - /tmp
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
    mem_limit: 256m
    pids_limit: 100
    restart: unless-stopped

networks:
  front:

volumes:
  app-data:

The application does not need any Linux capabilities. It writes only to the named volume at /data and tmpfs at /tmp.

Act 1 — Validate and build

From docker-lab:

docker compose config
docker compose --progress=plain build
docker image ls docker-course-app
docker image history docker-course-app:dev

Build again:

docker compose build

The second build should reuse cached steps.

Change only the APP_MESSAGE value in compose.yaml. Predict: does the image rebuild?

Answer: no. It is runtime configuration, outside the Dockerfile build.

Act 2 — Start and trace one request

docker compose up -d
docker compose ps
curl http://127.0.0.1:18080/
curl http://127.0.0.1:18080/
docker compose logs --tail=50 app proxy

Expected response shape:

{
  "message": "Hello from a declared Compose stack",
  "hostname": "a-container-hostname",
  "visits": 2
}

Trace:

curl
→ host loopback port 18080
→ proxy container port 80
→ Docker DNS resolves app
→ app container port 8000
→ SQLite file in app-data volume
→ JSON response

Act 3 — Inspect the boundaries

# Effective Compose model
docker compose config

# Runtime status and health
docker compose ps

# User identity: should be uid/gid 10001
docker compose exec app id

# Root filesystem: this write should fail
docker compose exec app python -c "open('/app/nope', 'w')"

# Declared volume: this write should succeed
docker compose exec app python -c "open('/data/ok', 'w').write('yes')"

# Mounts, limits, image, command, and network attachments
docker inspect docker-course-lab-app-1

# Live resource use
docker stats --no-stream docker-course-lab-app-1

# Project network and DNS membership
docker network inspect docker-course-lab_front

# Volume identity
docker volume inspect docker-course-lab_app-data

Generated names can vary if you changed the project name. Obtain exact names with

docker compose ps.

Act 4 — Prove container replacement is not data loss

Record the current visit number:

curl http://127.0.0.1:18080/

Recreate the application container:

docker compose up -d --force-recreate app proxy
docker compose ps
curl http://127.0.0.1:18080/

Observe:

• app container ID/hostname changed;
• visit count continued;
• the named volume kept its identity.

Now stop and remove the project without deleting volumes:

docker compose down
docker compose up -d
curl http://127.0.0.1:18080/

The count still continues because app-data survived.

Do not run docker compose down --volumes until you intentionally want to erase the lab database.

Act 5 — Prove configuration causes recreation

Change:

APP_MESSAGE: "Configuration changed without rebuilding"

Apply and inspect:

docker compose up -d
docker compose ps
curl http://127.0.0.1:18080/

Compose recreates the application because its effective container configuration changed. The image digest can stay the same. Build-time artifact and runtime configuration are separate.

Act 6 — Watch graceful shutdown

Follow logs in one terminal:

docker compose logs -f app

In another:

docker compose stop app

Look for JSON events resembling:

{"event": "shutdown", "signal": 15}
{"event": "stopped"}

This works because the Python process receives SIGTERM, the request loop wakes, and PID 1 exits. Compose’s init: true also supplies a minimal init process for reaping children and signal forwarding.

Restart:

docker compose up -d app proxy

Act 7 — Break networking on purpose

Stop the application while leaving the proxy:

docker compose stop app
curl -v http://127.0.0.1:18080/
docker compose logs --tail=30 proxy app

Expected boundary diagnosis:

• host port 18080 still accepts through nginx;
• nginx cannot reach app:8000;
• the problem is proxy → app, not browser → proxy.

Repair:

docker compose up -d app proxy
docker compose ps
curl http://127.0.0.1:18080/

Act 8 — Use an ephemeral diagnostic container

Find the network name:

docker network ls

Then:

docker run --rm \
  --network docker-course-lab_front \
  curlimages/curl \
  http://app:8000/health

This starts a disposable diagnostic container on the same network, resolves app through Docker DNS, calls the container port directly, then removes itself.

The app publishes no host port. It is reachable internally because network connectivity and port publishing are different.

Act 9 — Debug three common failures

Failure A: port already allocated

Symptom:

bind: address already in use

Diagnose:

docker ps --format 'table {{.Names}}\t{{.Ports}}'
lsof -nP -iTCP:18080 -sTCP:LISTEN

Fix the actual conflict or change only the host side:

ports:
  - "127.0.0.1:18081:80"

Failure B: application listens on container loopback

If app.py used:

ThreadingHTTPServer(("127.0.0.1", PORT), Handler)

the health check inside the same container might pass, while nginx on another container cannot connect. Listen on 0.0.0.0 for container-network traffic.

This is a valuable lesson: a health check can be healthy yet incomplete.

Failure C: volume permission denied

Diagnose:

docker compose exec app id
docker compose exec app ls -ldn /data
docker inspect docker-course-lab-app-1 \
  --format '{{json .Mounts}}'

Fix ownership at image initialization or through a controlled initialization step. Do not leap to root or chmod 777.

Act 10 — Cleanup

First inspect:

docker compose ps -a
docker volume ls
docker image ls docker-course-app

Remove containers and project network, preserving data:

docker compose down

When you intentionally want to erase the lab database:

docker compose down --volumes

Remove the lab image:

docker image rm docker-course-app:dev

Base images and reusable build cache may remain. Inspect with docker system df before deciding whether any broader cleanup is appropriate.

Docker CLI survival card

System and context

docker version
docker info
docker context ls
docker system df
docker events

Images and builds

docker build -t app:dev .
docker build --no-cache --pull -t app:clean .
docker image ls --digests
docker image inspect app:dev
docker image history app:dev
docker pull registry/repo:tag
docker push registry/repo:tag
docker image rm app:old
docker builder du

Containers

docker run --rm IMAGE COMMAND
docker run -d --name app IMAGE
docker ps
docker ps -a
docker logs -f --tail=100 app
docker exec -it app sh
docker inspect app
docker top app
docker stats app
docker diff app
docker stop app
docker start app
docker rm app

Networks

docker network create app-net
docker network ls
docker network inspect app-net
docker network connect app-net app
docker network disconnect app-net app
docker port app

Volumes

docker volume create app-data
docker volume ls
docker volume inspect app-data
docker volume rm app-data

Compose

docker compose config
docker compose build
docker compose up -d
docker compose ps
docker compose logs -f
docker compose exec SERVICE COMMAND
docker compose run --rm SERVICE COMMAND
docker compose stop
docker compose down

Debugging decision tree

flowchart TD
    S["Something is broken"] --> E{"Can client reach Engine?"}
    E -->|No| E1["docker version/context
start or select correct Engine"]
    E -->|Yes| C{"Container running?"}
    C -->|No| C1["ps -a → logs → inspect exit/OOM/error"]
    C -->|Yes| P{"Main process healthy?"}
    P -->|No| P1["logs → top → health → config → limits"]
    P -->|Yes| N{"Reachable from same network?"}
    N -->|No| N1["listen address → DNS → network membership → container port"]
    N -->|Yes| H{"Reachable from host/client?"}
    H -->|No| H1["published port → bind IP → firewall → proxy"]
    H -->|Yes| D{"Data correct and durable?"}
    D -->|No| D1["mount target → volume identity → permissions → app transactions"]
    D -->|Yes| O["inspect upstream dependencies
and application behavior"]

The order matters. Do not debug a firewall before establishing that the process is running and listening.

Glossary

Where next

After this course:

1. Containerize one application you already understand.
2. Add a database and proxy through Compose.
3. Make the image non-root and the root filesystem read-only.
4. Add CI that builds, tests, scans, and records the image digest.
5. Practice a backup restore and a failed-release rollback.
6. Learn an orchestrator only after these container boundaries feel ordinary.

Official references

• Docker overview
• Dockerfile reference
• Build cache
• Multi-stage builds
• Docker network drivers
• Port publishing
• Docker storage
• Compose file reference
• Compose startup order
• Docker Engine security
• Rootless mode