Signal Server ratios move from eight GPUs per coordinating CPU toward two GPUs per CPU

Summary

Deloitte describes how server hardware is being reshaped for AI workloads. Chips are diversifying into GPUs, neural processing units for efficient inference and tensor processing units for particular machine learning tasks. Server refreshes now include mixed configurations of central and graphics processors. Whereas racks once held four to eight GPUs on a tray coordinated by one CPU, the pattern is increasingly two GPUs per CPU. The report sees these shifts converging into a custom-built AI data centre with more GPUs relative to CPUs, new server models and orchestration for hybrid workloads, rapidly deployable form factors and optical links between processors. Not everyone agrees on how far this should go, since David Linthicum argues most valuable enterprise AI workloads will run fine on CPUs and only heavy training needs specialised processors.