Making AI operational in constrained public sector environments

Public sector organizations face significant constraints in security, governance, and operations that complicate AI adoption. A Capgemini study found 79 percent of public sector executives globally are wary about AI’s data security due to sensitive government data and legal obligations. Purpose-built small language models (SLMs) offer a promising path to operationalize AI by being housed locally, providing greater security and control than large language models (LLMs). SLMs typically use billions of parameters, making them less computationally demanding and able to perform as well or better than LLMs in empirical studies. Gartner predicts that by 2027, small, specialized AI models will be used three times more than LLMs, suggesting a shift towards bringing AI tools to the data rather than sending data to the cloud.