No country left behind with sovereign AI

In this episode of the Stack Overflow Podcast, host Ryan Donovan speaks with Steve Watt, VP of the Office of the CTO at Red Hat, about the growing movement toward sovereign AI and data sovereignty. They explore how nations like Saudi Arabia, the UAE, and regional initiatives such as Massachusetts Innovation Hub are building infrastructure to ensure their citizens aren't left behind in the AI revolution. The discussion dives into the technical and geopolitical challenges of creating sovereign AI ecosystems—ranging from power and cooling demands for AI hardware to the need for open, transparent pipelines and data sets. Steve emphasizes that true open-source AI must include not just model weights, but also the full training pipeline and data provenance. He also highlights innovations like VLLM, LLMD, and the VLLM semantic router that enable scalable, reliable inference on Kubernetes, while discussing the shift toward disaggregated, modular AI systems and inference-focused accelerators like those based on RISC-V and CPU inference. The episode underscores the tension between rapid AI advancement and the lagging regulatory and infrastructural frameworks, especially in regions unable to build new data centers due to land, water, or energy constraints. Key takeaways include: 1) Sovereign AI is not just about geographic data control but also about ensuring equitable access to AI infrastructure; 2) True open-source AI requires transparency in data, pipeline, and model training—not just weights; 3) Disaggregation and modularization of AI systems (via semantic routers, LLMD, and CPU inference) are critical for scalability and cost control; 4) The rise of inference-focused accelerators and RISC-V offers a path to energy-efficient, regionally deployable AI; 5) The 'sovereign paradox'—where countries must operate AI infrastructure abroad due to infrastructure gaps—threatens the core promise of sovereignty. The episode concludes with a call for deeper collaboration between governments, technologists, and open-source communities to build inclusive, resilient AI ecosystems.