Reiner Pope – Chip design from the bottom up

In this deep dive into AI chip design, Reiner Pope, CEO of Maddox, dismantles the inner workings of AI accelerators from the ground up, starting with the most fundamental building blocks: logic gates and wires. He demonstrates how a 4-bit multiply-accumulate operation—central to neural network computations—is implemented using AND gates and full adders, revealing that the real cost isn't in the computation itself, but in the data movement required to feed it. This insight leads to the revolutionary concept of systolic arrays, which bake matrix multiplication directly into hardware, drastically reducing communication overhead by storing weights locally and streaming inputs in slowly. Pope explains how this shift—from individual CUDA cores to massive, fixed-function systolic arrays—has driven the efficiency gains in modern AI chips like NVIDIA’s Tensor Cores and Google’s TPUs. He contrasts GPU and TPU architectures, showing how GPUs’ tiled, fine-grained design limits scalability, while TPUs’ coarser, unified structure enables larger, more efficient arrays. The discussion then turns to clock cycles, pipeline registers, and the trade-offs between speed, area, and throughput, culminating in a powerful insight: the most efficient chips aren’t just faster—they’re designed to minimize the cost of moving data, not just computing it.