Colin Whitlatch: Advanced In-Platform AI at Kahua

In this episode of Constructed Futures, host Hugh Seaton sits down with Colin Whitlatch, CTO of Kahua, to explore how the company has rapidly integrated advanced in-platform AI into its enterprise software, not as a gimmick but as a natural evolution of its core mission. Whitlatch explains that Kahua’s AI strategy is rooted in decades of platform development focused on security, performance, and usability—values that now align perfectly with AI’s strengths in natural language processing and dynamic workflows. Rather than relying on third-party APIs, Kahua built a compliant, government-ready AI infrastructure using only two core capabilities: generative AI and vector embeddings, enabling secure, high-speed operations across both commercial and government networks. A key innovation is their 'ephemeral RAG' system, which creates temporary, on-the-fly vector databases from user-specific data, drastically reducing latency by eliminating external round trips. This, combined with fast JSON parsing and a 'mixture of assistants' architecture inspired by models like Gemini’s mixture of experts, allows for near-instant responses and context-aware task execution. The team also prioritized token efficiency from day one, avoiding wasteful practices like sending entire PDFs to LLMs, and introduced 'pseudo tool calls' to maintain security and performance by intercepting and optimizing AI-driven actions. Finally, they’ve implemented a natural language intent layer between user queries and AI processing, allowing the system to understand business context and deliver precise, concise results without overloading users with irrelevant information.