5 Takeaways from the InnovateUS Series on What Public AI Actually Means for Government
This month, I joined Beth Noveck and Mihir Kshirsagar for the final session of the Innovate US series on Public AI. In our session, we built on themes from earlier installments to tackle the core question: what does it actually mean to use AI in the public interest, and what should leaders be doing today especially around governing and funding AI in government?
It seems that many of today's AI conversations continue to get stuck on a definitional problem. What even is public AI? Part of the challenge we discussed during the workshop is that we naturally hit a wall when we treat public AI as a singular thing with a binary approach: build your own sovereign model or buy commercial. But anyone who has been around the GovTech market long enough knows that technology decisions are rarely that simple.
Here are the five takeaways I keep coming back to from the series, and from every conversation I have with government leaders.
The most useful way to think about public AI is as a stack. Not a single approach, not a single product, not a single model, and definitely not a single vendor. Like any good infrastructure stack, it's hybrid by design.
Some layers are well-served by commercial frontier models. For summarization, drafting, research augmentation, and translation, commercial models deliver enormous value today. Agencies should be using them.
Other layers may require something different. Systems processing CJIS data, benefits determinations, child welfare cases, or infrastructure controls demand local inference, air-gapped environments, and models that agencies can inspect, audit, and control end-to-end. If you think about AI running critical infrastructure components, this becomes a near-term architectural requirement I'm already seeing agencies grapple with.
And then there's the middle layer, the most strategically important and the least discussed: the governance and orchestration layer that sits across everything. Regardless of how AI is delivered to your agency (commercial, open-source, locally hosted, or embedded inside an application you bought five years ago that suddenly has AI features), you need a unified way to enforce policy, manage risk, and maintain visibility. That layer has to be model-agnostic, vendor-agnostic, and built for a world where the number of AI systems in your environment grows every week.
We've been here before. I watched this exact evolution play out with cloud, from my time in government through my years at e.Republic. Every agency that wanted to go "all in" on cloud ended up running some level of hybrid. The ones that succeeded built for that reality from day one rather than pretending it would be one or the other.
AI will follow the same arc, but faster. Which makes open standards critical. We need model interoperability, data portability, and governance enforcement that let agencies swap models without rebuilding the stack. We need procurement frameworks that don't lock agencies into a single vendor's ecosystem for a decade. And we need cooperative vehicles, through MSAs, ISACs, and state-level shared services, that give smaller agencies access to the same capabilities large state enterprises can build on their own. We need a strategy that is multi-model by default.
The agencies that will lead aren't the ones picking a side in the build-vs-buy debate. They're the ones architecting for a hybrid stack where every layer is governed consistently.
I hear this constantly. An agency tells me they've standardized on a single frontier model, or an aggregation of models packaged into a single experience, and they're covered. That only makes sense if you believe AI is a single product you deploy once.
Every application in your environment is in an arms race to embed more AI. Your employees are layering their own tools on top of that. Each system may be running different underlying models with different data handling practices. I have to update my slide decks every three days just to keep up with model name changes and major feature updates.
The point I made during the workshop is that if your strategy is deploying a single tool, that's not a strategy. That's a convenience. Agencies need to be model-agnostic, with both the portability and the contractual ability to switch models when necessary. Different models are getting better at different things, and you'll likely want different ones optimized for different workloads. That's exactly what we see in the enterprise. Multi-model, AI-orchestrated across multiple applications, is the current reality that governance needs to streamline. That doesn't mean enterprise AI won't evolve in how different AI silos are accessed in the future, but that's not today's reality.
In our own research and implementations, we've seen that roughly 62% of AI usage in government is shadow AI, meaning agencies don't have full visibility into the majority of AI their employees are using. What's important to remember is that those employees aren't being malicious. They're being resourceful. They're finding ways to reply to emails faster and augment how they do their jobs. They're consumers first, and consumers have access to incredibly powerful tools right now.
The mistake I see agencies make is treating this like a security problem you can solve with a block list or network-level controls. What we need to remember is that if you simply ban shadow AI, you don't stop the behavior. You drive it to personal devices and lose visibility entirely. Instead of playing whack-a-mole at the network level, meet employees where they are, help them use AI responsibly, and educate them in real time rather than routing them through another round of compliance training nobody retains.
And shadow AI isn't just ChatGPT in a browser. It's embedded AI surfacing inside tools you already own. Your CRM added an AI assistant last quarter. Your EDM started summarizing files. Zoom is transcribing your meetings. These look like legitimate traffic tied to existing applications, and network-level monitoring doesn't catch them. You need visibility at the endpoint, across every interaction, regardless of how the AI enters the workflow.
One of the things I feel most strongly about is the need to operationalize governance in an employee-centric way. Nearly every agency has some form of AI governance in place. A policy document, a committee, a set of AI guidelines. These are great starting points, but they aren't enough. The gap isn't policy. The gap is operationalizing policy. And even if you don't have an AI policy yet, you have a mix of other policies that already apply to your agency's use of AI. So where do you start?
You can't control what you can't see, so start with visibility. You need to know what AI tools are active in your environment, and I promise it's far more than what you think is there.
Then compliance. Acceptable use policies, data security policies, state law, open records law all apply to AI today. The question is how you make that real for an employee using ChatGPT or Gemini who may not know if a use case is banned or bordering on high-risk. They're just trying to get through their day.
Then real-time risk mitigation. If someone is replying to an email with a frontier model and accidentally includes PII, you need to catch it, prevent the data loss, and educate the user in the moment, not sign them up for a year of training. That's where governance becomes an infrastructure layer, not a feature in an application. Just like cybersecurity.
I've watched agencies use the shifting federal landscape as a reason to pause. "We're going to let this play out and then update our governance." That's a mistake.
Federal executive orders have not undone state-level action or preempted local AI governance. They've explicitly carved out responsible internal AI use at the state and local level. The federal focus has been on private-sector model disclosure, not on how a state agency governs its own deployments.
As Mihir Kshirsagar pointed out during our session, the preemption argument is largely posturing. States have the power of the purse. The federal government has also made it clear that its approach to deregulation of AI at the state and local level is not focused on government use, but rather on how agencies may try to overregulate frontier models for disclosure and similar requirements. If anything, the current environment makes it more urgent to get your governance house in order, not less.
We closed the workshop on action items that agencies and participants should look toward in order to move the needle, and the market, forward. Below are a few of my thoughts.
For the individual: be a champion for the use of AI. Find ways to connect it to real needs in your agency. Do it safely, share what you learn, and advocate. If you're getting resistance, show what's possible. Be a disruptive innovator. And know this: just as we got our handle on generative AI, agentic AI is arriving, and it's going to start eating the transactional layers of government. We need to be prepared.
For the organization: don't feel like you need an AI policy before you can use AI. Policies matter, and we offer a free Policy Wizard at Darwin for exactly that reason. But you already have existing rules, regulations, and compliance obligations that apply to AI today. Start with what you're trying to solve, align it to a real operational need, understand the risk profile, and set KPIs so you know whether you're moving the needle.
And band together. Join associations. Plug into networks like the GovAI Coalition, InnovateUS, and Princeton's Center for Information Technology Policy. Look at shared risk frameworks like the NIST AI RMF. We didn't ask banks to each build their own risk management infrastructure from scratch. We created shared frameworks and held institutions accountable. Government AI needs the same approach.
The agencies that will lead are the ones building visibility across their entire AI footprint, operationalizing their policies in real time, designing for a hybrid future, and keeping humans in the loop to make decisions grounded in good data. The building blocks are there. Now we need to connect the dots.
Thanks to Beth Noveck, Mihir Kshirsagar, and the entire InnovateUS team for an outstanding series. Honored to help bring it home.
.png)
.svg)
