37.1d Part VIII–IX: Orchestration and Operations — 80 questions with rationales¶

📘 Context Introduction¶

Welcome to the practice assessment covering Orchestration and Operations for the NVIDIA-Certified Associate: AI Infrastructure and Operations exam. This section focuses on how engineers manage, automate, and monitor AI workloads across GPU-accelerated environments. You will explore concepts like container orchestration with Kubernetes, job scheduling, resource management, logging, and operational best practices for AI clusters.

This quiz bank contains 80 questions with detailed rationales to help you understand not just what the correct answer is, but why it is correct. Each question is designed to mirror real-world scenarios you might face when deploying and maintaining AI infrastructure.

⚙️ Part VIII: Orchestration Fundamentals (40 Questions)¶

🧩 Key Concepts Covered¶

• Container orchestration platforms (Kubernetes, Docker Swarm)
• GPU scheduling and resource allocation
• Pod lifecycle management for AI workloads
• Service discovery and load balancing
• Persistent storage for training data and models

📊 Sample Question Format¶

Question 1:
What is the primary benefit of using Kubernetes for AI workload orchestration?

• A) It provides a graphical user interface for monitoring
• B) It automates deployment, scaling, and management of containerized applications
• C) It replaces the need for GPU drivers
• D) It only works with NVIDIA GPUs

Rationale:
The correct answer is B. Kubernetes automates deployment, scaling, and management of containerized applications, which is essential for AI workloads that require dynamic resource allocation. Option A is incorrect because Kubernetes primarily uses a command-line interface (kubectl) and APIs. Option C is wrong because GPU drivers are still required. Option D is false because Kubernetes supports multiple hardware backends.

🛠️ Orchestration Comparison Table¶

🕵️ Common Orchestration Pitfalls¶

• Not setting resource limits for GPU memory and CPU — leads to resource starvation
• Forgetting to configure node affinity for GPU-enabled nodes — pods may land on CPU-only nodes
• Ignoring pod priority classes — critical training jobs may be preempted by less important workloads
• Using default storage classes without considering performance — slow I/O can bottleneck training

📊 Visual Representation: Orchestration & Operations concept mapping¶

This diagram details orchestration exam scopes, partitioning Kubernetes schedulers and Slurm job systems.

📊 Part IX: Operations & Monitoring (40 Questions)¶

🔍 Key Operational Topics¶

• Cluster health monitoring (GPU utilization, memory, temperature)
• Log aggregation and analysis (ELK stack, Prometheus, Grafana)
• Job scheduling and queue management
• Backup and disaster recovery for AI artifacts
• Security best practices (RBAC, network policies, secrets management)

📈 Sample Question Format¶

Question 42:
Which tool is commonly used to collect and visualize GPU metrics in a Kubernetes cluster?

• A) NVIDIA System Management Interface (nvidia-smi)
• B) Prometheus with NVIDIA DCGM Exporter
• C) Docker logs
• D) Kubectl top

Rationale:
The correct answer is B. Prometheus with the NVIDIA DCGM Exporter collects detailed GPU metrics (temperature, memory usage, power draw) and visualizes them in Grafana. Option A (nvidia-smi) is a command-line tool, not a continuous monitoring solution. Option C (Docker logs) only captures container logs, not GPU metrics. Option D (kubectl top) shows CPU/memory usage but not GPU-specific metrics.

🛡️ Operational Best Practices¶

• Implement pod resource quotas to prevent a single user from consuming all GPUs
• Set up alerts for GPU temperature exceeding 85°C or memory errors
• Use taints and tolerations to reserve GPU nodes for AI workloads only
• Enable audit logging for all kubectl commands to track changes
• Regularly backup model checkpoints and training data to persistent storage

🔄 Troubleshooting Workflow¶

1. Identify the symptom — Is the job stuck? Are GPUs idle? Is there an error log?
2. Check pod status — Use kubectl describe pod to view events and conditions
3. Inspect GPU health — Verify GPU availability and driver status
4. Review resource limits — Ensure pods have requested sufficient GPU memory
5. Examine network connectivity — Confirm pods can communicate with storage and other services
6. Analyze logs — Use kubectl logs or centralized logging to find error messages

✅ Final Exam Tips¶

• Focus on understanding why certain orchestration decisions are made (e.g., why use StatefulSets for databases vs. Deployments for stateless apps)
• Practice reading Kubernetes events — they often contain the root cause of failures
• Memorize key NVIDIA tools like DCGM, nvidia-smi, and the GPU Operator
• Understand resource allocation strategies (bin packing, spread, topology-aware scheduling)
• Review security concepts like service accounts, RBAC roles, and network policies

📚 Additional Resources¶

• NVIDIA GPU Operator documentation
• Kubernetes official documentation on device plugins
• Prometheus and Grafana monitoring guides
• NVIDIA DCGM user guide
• Kubectl cheat sheet

This practice assessment is part of the NVIDIA-Certified Associate: AI Infrastructure and Operations exam preparation. Each question is designed to reinforce core concepts and prepare you for real-world AI infrastructure management.