1.2e SMT / Hyper-Threading: logical vs physical cores¶

🧠 Context Introduction¶

When you look at a modern CPU, you might see "4 cores, 8 threads" or "16 cores, 32 threads" in its specifications. This can be confusing at first. Why are there two different numbers? The answer lies in a clever technology called Simultaneous Multithreading (SMT) — known as Hyper-Threading on Intel processors.

This topic explains the difference between physical cores (the actual hardware) and logical cores (virtual processors created by SMT). Understanding this distinction is essential for engineers working with AI workloads, as it affects how you allocate resources, size virtual machines, and optimize performance.

⚙️ What Are Physical Cores?¶

Physical cores are the real, tangible processing units inside a CPU chip. Each physical core has its own dedicated execution resources (ALU, FPU, cache, registers).

Key characteristics: - Each physical core can execute one instruction stream at a time (without SMT) - Physical cores are the "workers" — the actual hardware doing the computation - More physical cores generally mean better parallel processing capability - In AI workloads, physical cores handle heavy computation like matrix multiplications

🔄 What Are Logical Cores?¶

Logical cores (also called virtual cores or threads) are created by SMT/Hyper-Threading technology. Each physical core can appear as two logical cores to the operating system.

How it works: - The physical core has two sets of architectural state (registers, program counter) - While one thread is waiting for data (memory access), the other thread can use the execution units - The OS "sees" twice as many cores as physically exist - Logical cores share the physical core's execution resources

📊 Physical vs Logical Cores — Comparison Table¶

🛠️ Why Does This Matter for AI Infrastructure?¶

For engineers managing AI infrastructure, understanding SMT is critical for several reasons:

• Virtual machine sizing: When you provision a VM with 8 vCPUs, you might be getting 4 physical cores with SMT enabled. This affects performance isolation.

• AI training workloads: Heavy matrix operations benefit from physical cores. SMT can sometimes cause resource contention for compute-bound AI tasks.

• Inference serving: SMT can help handle multiple concurrent inference requests efficiently.

• Licensing: Some software licenses are based on physical cores, not logical cores.

🕵️ How to Check Physical vs Logical Cores¶

On a Linux system, engineers can check core topology using these methods:

Check total logical cores: The command nproc shows the number of logical cores available to the OS.

Check physical core count: The file /proc/cpuinfo contains detailed CPU information. Look for the cpu cores field.

Check SMT status: The file /sys/devices/system/cpu/smt/active shows whether SMT is enabled (1 = enabled, 0 = disabled).

View core-to-thread mapping: The command lscpu displays a clear summary including: - CPU(s): Number of logical cores - Core(s) per socket: Number of physical cores - Thread(s) per core: SMT factor (usually 2)

📈 Performance Considerations for AI Workloads¶

🧪 Simple Example: 4 Physical Cores with SMT¶

Imagine a server with 4 physical cores and SMT enabled:

• Physical cores: 4 (actual hardware)
• Logical cores: 8 (each physical core appears as 2)
• OS sees: 8 CPUs available for scheduling

When you run an AI training script: - Without SMT: 4 threads running simultaneously - With SMT: 8 threads running, but sharing resources in pairs - Performance gain: Typically 20-30% more throughput for mixed workloads

📊 Visual Representation: Physical vs. Logical Core Mapping¶

This diagram shows how SMT/Hyper-Threading maps 8 logical cores visible to the operating system onto 4 physical hardware CPU cores, illustrating execution resource sharing.

✅ Key Takeaways for New Engineers¶

• Physical cores = actual hardware; logical cores = virtual processors via SMT
• SMT improves throughput by using idle execution resources
• For compute-heavy AI workloads, SMT may not always help (sometimes hurts)
• Always check whether your cloud instance or server has SMT enabled
• When sizing VMs for AI, understand if you're getting physical or logical cores
• Use lscpu or /proc/cpuinfo to verify core topology on any Linux system

📚 Quick Reference¶

💡 Pro Tip: For AI training workloads, consider disabling SMT in BIOS or using CPU pinning to dedicate physical cores to critical processes. For general-purpose AI infrastructure, SMT usually improves overall resource utilization.