23.3g Docker Compose: multi-container service definitions in YAML¶

🌱 Context Introduction¶

When building AI applications, you rarely run just one container. A typical AI pipeline might include a model server, a database, a message queue, and a monitoring tool — all working together. Docker Compose lets you define and run these multi-container applications using a single YAML file. Instead of typing multiple docker run commands, you describe everything in one place, making your AI infrastructure reproducible and easy to share with your team.

⚙️ What is Docker Compose?¶

Docker Compose is a tool that allows you to define and manage multi-container Docker applications. You write a docker-compose.yml file that describes:

• Which containers (services) to run
• How they connect to each other
• What volumes and networks they use
• Environment variables and resource limits

For AI workloads, this means you can spin up an entire inference stack — GPU-accelerated model server, vector database, and API gateway — with a single command.

📄 The docker-compose.yml File Structure¶

A Compose file follows a simple structure with three main top-level keys:

• version — Specifies the Compose file format version (use version 3.8 or later for modern features)
• services — Defines each container as a named service
• volumes and networks — Optional sections for persistent storage and custom networking

Each service under services contains configuration similar to what you would pass to docker run, such as image, ports, environment, and volumes.

🛠️ Key Concepts for AI Engineers¶

Services¶

Each service represents one container. For an AI application, you might have: - A model-serving service running a TensorFlow or PyTorch model - A redis service for caching inference results - A postgres service for storing metadata

Dependencies¶

Services can depend on each other. For example, your model server should start only after the database is ready. Use the depends_on key to control startup order.

Environment Variables¶

AI models often need configuration like model paths, API keys, or GPU settings. Define these under the environment key for each service.

GPU Access¶

For NVIDIA GPU-accelerated containers, you specify runtime: nvidia and set environment variables like NVIDIA_VISIBLE_DEVICES to control which GPUs are available.

📊 Comparison: Docker Run vs Docker Compose¶

🕵️ Common AI Infrastructure Patterns with Compose¶

Pattern 1: Model Server + Database¶

A typical pattern is running a GPU-accelerated model server alongside a database for storing results. The model server exposes a REST API on port 8000, while the database stores prediction logs.

Pattern 2: Training Pipeline¶

For distributed training, you might have: - A training service that runs your training script - A tensorboard service for monitoring - A data-prep service that preprocesses datasets

Pattern 3: Inference Stack with Monitoring¶

Production AI stacks often include: - api-gateway — Routes requests to model servers - model-server — Runs inference on GPU - prometheus — Collects metrics - grafana — Visualizes performance

📊 Visual Representation: Docker Compose Multi-Container Orchestration¶

This diagram displays how Docker Compose defines and boots multi-container applications (e.g., frontend, API, database) over a shared network.

🧩 Service Dependencies and Startup Order¶

Use depends_on to define which services must start first. For example:

• model-server depends on redis and postgres
• api-gateway depends on model-server

This ensures your AI stack starts in the correct order. Note that depends_on only controls startup order, not readiness — your application code should handle waiting for services to be fully ready.

🌐 Networking in Compose¶

Docker Compose automatically creates a default network for all services, so they can communicate using service names as hostnames. For example, your model server can connect to the database using the hostname postgres instead of an IP address.

You can also define custom networks for: - Isolating sensitive services (e.g., database on a private network) - Connecting to external networks

💾 Volumes for Persistent Data¶

AI workloads generate large amounts of data — model weights, training logs, and inference results. Use volumes to persist this data across container restarts:

• Named volumes — Managed by Docker, stored in /var/lib/docker/volumes
• Bind mounts — Map a host directory into the container

For GPU-accelerated AI, you typically mount: - Model files from the host into the container - Output directories for inference results - Log directories for monitoring

🚀 Scaling Services¶

Docker Compose supports scaling services horizontally. For example, if your model server becomes a bottleneck, you can run multiple instances:

• Scale a service with the --scale flag
• Each instance gets a unique container name
• Use a load balancer (like Nginx) in front of scaled services

For GPU workloads, ensure each scaled instance has access to a different GPU by setting NVIDIA_VISIBLE_DEVICES appropriately.

🔍 Debugging Multi-Container AI Stacks¶

When something goes wrong, use these techniques:

• Check logs for a specific service with docker compose logs
• View running containers with docker compose ps
• Execute commands inside a running service with docker compose exec
• Inspect network connectivity between services

Common issues in AI stacks: - GPU not accessible inside the container (check runtime: nvidia) - Service cannot reach database (check network and hostnames) - Port conflicts (ensure unique port mappings)

✅ Best Practices for AI Engineers¶

• Version control your docker-compose.yml — Treat it like code
• Use .env files — Store sensitive values like API keys outside the Compose file
• Pin image versions — Use specific tags like tensorflow/tensorflow:2.12.0-gpu instead of latest
• Limit resources — Set deploy.resources limits for CPU and memory to prevent one service from starving others
• Test locally first — Validate your Compose file on a single machine before deploying to a cluster

📝 Summary¶

Docker Compose transforms how engineers manage multi-container AI applications. Instead of juggling multiple terminal windows and remembering complex docker run commands, you define your entire infrastructure in a single, readable YAML file. This makes your AI stack reproducible, shareable, and easier to debug — whether you're running a simple model server or a complex distributed training pipeline.