The AI agent governance gap: How CIOs can gain control

AI agents are popping up across the enterprise faster than most CIOs can track them.

Developers build side-project agents, departments connect agents to external tools and SaaS vendors increasingly embed agentic features directly into enterprise software. Many organizations still lack clear inventories and governance processes for those systems, even as the agents gain access to sensitive data. Additionally, CIOs face growing pressure to understand how these systems operate and how to audit them if something goes wrong.

TechTarget interviewed David Baum, founder and CEO of AI governance platform Roval, about the emerging challenges surrounding agentic AI governance. Baum argued that many enterprises still cannot fully see or classify the agents operating across their environments, that traditional IT monitoring tools struggle to track agent behavior and that organizations need new approaches to inventory management and auditability.

In the following interview, Baum explains why AI agents create new governance problems and what CIOs can do to build oversight and accountability into agentic systems.

Editor's note: The following transcript was edited for length and clarity.

Why is visibility into AI agents so difficult for enterprises right now?

David Baum: First, these are new, and they're a different breed of system. Agents are not servers, they don't have IP addresses and they don't appear as rows in your infrastructure inventory. An agent running as a scheduled function inside a SaaS platform, or something similar, doesn't leave the same type of footprint that system operators and administrators are used to seeing -- you can't find it in your configuration management database.

Second, agents aren't applications. So, a single SaaS application might contain three, five or 10 embedded AI features, each behaving as an independent agent with its own data access patterns. Existing systems don't really see the agents operating inside these applications.

Third, agents are not users. They do act on behalf of users, often have credentials and may use service accounts, API keys or OAuth tokens, but your identity and access management system doesn't show that user with the usage patterns you'd expect. You can't monitor it in the same way with the same type of tools. It doesn't show that the user's permissions are being exercised in an autonomous way 200 times an hour -- that would completely blow up every monitoring view and dashboard. So, it's a problem of legacy tools not keeping up with this rapidly evolving way of building capabilities.

When we think about AI agent security incidents or compliance incidents, what kind of things typically go wrong?

Baum: Incidents can be difficult to classify because existing tools often lack the insight needed to do so. Systems like Sentry, that monitor everything that happens on the network layer and offer you observability and insight into all traffic through a proxy, router or a firewall, capture quite a lot. However, agents operate not only on the LLM API call layer, where they can include or not include various types of sensitive or non-sensitive information as they call an API, but they also use several tools and internal data sources. Therefore, the complexity and heterogeneity of incidents are among the main problems. Building systems that can monitor and observe that level of complexity is really where most enterprises fail.

You capture some of the aspects of an incident, but not all of it -- and that's where things go wrong. It can be as simple as the unaudited execution of a decision, where a model wasn't supposed to make a decision based on a set of parameters, but model drift pushed the model's confidence just enough to reach a limit the old model didn't. To detect something like that, you need insight into the call and the response, but you must also understand what happened during the last model upgrade. This is very difficult to debug and understand. Additionally, an incident could just be access to a tool where the underlying user credentials the agent used had changed for some other reason.

What does "not knowing what agents you have" look like in real organizations?

They eventually move on to a different organization or role and forget to turn off the agent.

Baum: The first category is developer side projects. Early adopters often build agents that do actual business work, but they're developers in their business. Maybe they're using Claude Code, Codex or Cursor, but they have a lot of authority because they build systems. They eventually move on to a different organization or role and forget to turn off the agent. Then, suddenly, you get drift in tool access or the underlying model.

The second is department-level deployments, where a department operates as a silo, as they often do. These deployments are one of the fastest-growing and hardest-to-govern categories, especially when departments work with multiple partners or integrations and connect agents to external tools. They start working on internal matters and may also have access to systems governed by other departments or business areas. It often turns into a source of conflict within the organization.

Another quickly growing category is vendor-provided agents, where your agents aren't things that you've deployed internally. They're embedded in SaaS tools that you or your users subscribe to. Many of them are enabled by default when you start using these tools and platforms, and the observability and access to that is even more limited. Some platforms offer observability with the tool, whereas others you have to upgrade to enable it. Mostly, these things go undetected, even though they often have access to internal data and systems.

How are enterprises currently trying to govern their agents?

Baum: We have a few best-in-class cases. Take Brex, for example. They've just published an open source package on GitHub that actually helped them monitor some of their agents. Their internal developers built that themselves because they saw the need, and they're a massive business within many different business areas within payments, HR and logistics. They built this system to be able to govern their own internal systems, but also to be able to comply with their larger enterprise customers' needs for insight and observability.

The worst case is when the CIO is first confronted with the issue because someone on the board asks, 'How many AI agents do we currently have deployed across the business?' In many cases, the answer is, 'I don't know.'

That's obviously a terrible position for a CIO to be in, but it's probably still the most common scenario today. Many organizations are not yet prepared. They don't have the systems, organizational structures or culture in place to properly monitor and govern these agents.

What does auditability mean in the context of autonomous agents?

Baum: First, you must be able to inventory your agents. Second, you want a classification taxonomy or rubric that classifies agents by capabilities, such as whether the risk is low, medium, high, or maybe a five-degree rubric. Even more important now than in the traditional view of IT systems is blast radius – the effect if something goes wrong. One is how risky the agent is, depending on the behavior it's designed to have, and the other thing is the total blast radius, which is more a function of what that agent has access to.

An audit must contain at least that classification of risk and an analysis of the blast radius.

This is where you find the real friction, because the more useful or helpful an agent is, the more authority you've given it. That also increases the blast radius. An audit must contain at least that classification of risk and an analysis of the blast radius.

Then, the governance aspect is the mitigation strategy. How do you handle this? Who is responsible, and what is the response protocol if something goes wrong? That's especially important if you're within a regulated industry.

If an AI agent does something it wasn't supposed to do, what information would a CIO need to trace what happened?

Baum: Ideally, you need a proper system in place to monitor and provide observability. Some of the larger enterprise platforms have emergent capabilities that help with that. One broadly adopted, both open source and commercially available framework called LangChain has an extension called LangSmith, which is an agentic observability platform that you can freely use as an open source, self-hosted version, or you can adopt it.

You can use systems like Rebuild or Roval, which monitor the network traffic generated by agents, monitor the tool use in a sidecar to the agent and leave an audit trail of everything that the agent does, including the action it takes in scripts on the file, what systems it accesses, what network egress it has, which systems it contacts, what it receives, what it sends and what that contains.

That's especially important in regulated industries, where you want to monitor whether an agent is sending information to the LLM API, which contains personal information. Agents need to be monitored in a multimodal way at several levels of their operational infrastructure.

How is regulation, including efforts like the EU AI Act, shaping how organizations think about AI agents?

Baum: It hasn't affected many organizations yet for two reasons. One is the EU AI Act requirements for implementation have been pushed out until the end of 2027, which is probably a good thing, even though it does increase the time span in which things can go wrong.

Part of the problem is that the AI Act wasn't written with agents in mind, and it actually doesn't use the term at all. It does briefly mention LLMs, but it was written based on research done years ago on prediction engines and the use of deep neural nets. It's really hard to interpret what organizations should do to comply with the EU AI Act.

There's an army of advisors out there working on helping businesses interpret and understand how they should implement it, but there hasn't been any developments yet in standards for various sections and subsections of industry. It's largely left to industry itself to determine how we interpret and develop best practices for agent inventory, audit, monitoring and management.

What are some quick wins that CIOs can implement over the next six to 12 months when it comes to AI agent governance?

Baum: Do this in a way that encourages teams across the enterprise to continue working with agents, because if done well, the rewards are very tangible. You don't want to kill this off with fear-mongering.

Make it easy for employees to register agents in a centralized inventory.

First, make it easy for employees to register agents in a centralized inventory. Ideally, build software that lets the agent do it itself and updates its own configuration. Some of the enterprise platforms support this, but mostly they don't.

Second, make sure that a single individual is responsible for each agent, or a team of agents. Organize them topically or by division or organization, and connect with those people. Build a community and knowledge among those people about the risks and opportunities. That could be your primary channel for spreading the culture around the policy.

But for the agents, people having a culture around policy doesn't mean anything. Agents need policy as code. So, my third piece of advice would be to build in policy as code. Make sure that you manage at least two aspects of agentic behavior. One would be a sidecar approach where you deploy code that the agent acts upon, reads and is also controlled by at the process level. Also, make sure you deploy policy as code within the network layer so you can manage network access for the agent. This will give you access to audit and monitor what happens over time and stop any agent that has gone rogue.

You say a single individual should be responsible for each agent or team of agents. So, who's responsible for agents in most organizations right now?

Baum: A wonderful thing about this technology is that you can use natural language to describe what you want, and the software builds itself. This leads to an explosion in citizen developers. That's the positive term, whereas rogue agents are the negative side of that.

I love to think of this as employees becoming citizen developers, where they can use natural language to build a software tool that solves a problem -- which is exactly where you want to go. However, the challenge with that is it's not always clear who is accountable for that agent. A citizen developer may have enough knowledge of the business problem to describe it to a software system that builds the software, but they certainly don't have the experience to manage it. And if you look up at the command chain, their team leader doesn't either -- maybe they work in HR, finance or marketing.

You need embedded technologists with agentic knowledge in every part of the business, but not as an intelligent service. They should be helpful people who can offer guidance, but also be accountable and in control.

Never say that no one is allowed to develop or deploy agents, or that agents are only for the IT department.

One final note: never say that no one is allowed to develop or deploy agents, or that agents are only for the IT department. IT department ownership of this is a surefire way to kill a lot of the value creation.

Is there any misconception that enterprises have about governing AI agents that you see?

Baum: AI often requires an upfront investment, and it can take time to generate a return on that investment. So, many organizations are afraid to add tooling just to get this stuff done. In a very uncertain environment, it's often hard to take an investment like this seriously enough, because this is experimental technology. The misconception is that we can manage agents with existing tools, management systems and auditing frameworks. These are completely different animals.

Tim Murphy is a site editor and writer for the IT Strategy team at TechTarget.