GPU Computing

Originally designed to render 3D graphics, GPUs proved transformative for AI because their architecture — thousands of small, parallel cores — maps perfectly onto the tensor operations at the heart of deep learning. Training a large language model requires performing the same mathematical operation across billions of weight values simultaneously, a workload at which GPUs excel and CPUs fundamentally cannot compete.

The GPU landscape most relevant to enterprise AI centers on NVIDIA's data center lineup: the H100 (current flagship, up to 80GB HBM3 memory), A100 (widely available, strong price-performance on most workloads), and the L40S (optimized for inference at lower cost). AMD's MI300X has gained meaningful enterprise traction with its 192GB HBM3 memory capacity, making it attractive for large model deployments. Each GPU generation doubles roughly every two years on standard benchmarks, meaning multi-year hardware procurement cycles carry real obsolescence risk.

GPU memory — measured in gigabytes of HBM (High Bandwidth Memory) — is the most common bottleneck in enterprise AI. Model size in GPU memory follows: parameters × bytes per parameter. A 70B parameter model at 16-bit precision requires ~140GB — too large for a single H100 (80GB) and requiring tensor parallelism across multiple GPUs. Understanding the memory arithmetic is prerequisite to any serious infrastructure planning conversation.