How Much Authority Are We Ready to Give AI?

As frontier AI leaders debate the risks of increasingly powerful systems, enterprises face a more immediate question: how much decision-making and action should they delegate to AI?

Warnings over the weekend at the frontier of AI development have put the question of AI safety and control back in the spotlight. Anthropic CEO Dario Amodei has called for slowing the pace of frontier AI development so safety measures have time to catch up. OpenAI CEO Sam Altman and Elon Musk appear to agree. Much of this debate focuses on what could happen as frontier AI models become dramatically more powerful.

Inside companies, and parallel to this, the worry is losing control of agentic workflows before governance catches up. TDWI research indicates that fewer than one-quarter of organizations have agentic AI in production. We see many organizations now taking a more cautious approach to agentic AI. The research also indicates that AI governance remains relatively early stage in many organizations. Organizations have an opportunity to put responsible AI practices, governance mechanisms, and appropriate guardrails in place before increasingly autonomous AI becomes widespread. They should not wait to do so.

From assistance to authority

Generative AI largely introduced enterprises to AI that helps people do work. Agentic AI increasingly introduces AI that can do work. That’s an important distinction and there is a meaningful difference between an AI system that recommends an action and one authorized to take it. Think of it as a progression:

AI recommends → AI generates → AI decides → AI acts → AI acts across systems → AI coordinates with other agents

At every stage, the organization is potentially delegating more authority to the technology. A copilot that drafts an email creates one level of risk. An agent authorized to send the email creates another. An agent that can modify a customer’s account, initiate a transaction, change enterprise data, invoke another system, or direct another agent creates something different again. So one of the most important governance questions for leaders isn’t simply: are we using agentic AI? It’s how much authority have we delegated to AI?

Responsible AI becomes more important as autonomy increases

Of course, controlling what an AI system can do is only part of the issue. Organizations have spent years developing responsible AI principles around fairness, bias, transparency, explainability, privacy, safety, and accountability. Those issues don’t disappear as AI becomes more autonomous. In some ways, they become more consequential.

Consider an AI system that recommends which customers should receive an offer, which job candidates should move forward, or which transactions should be flagged. Organizations already need to ask whether those recommendations are biased or unfair. Is the underlying data appropriate and trustworthy? Can an important decision be explained? Can someone challenge the outcome? Is sensitive information being appropriately protected?

Now give that AI system the authority to act. A biased recommendation is a problem. An autonomous system capable of operationalizing that recommendation at scale can be a much bigger one. That’s why responsible AI and agentic governance can’t be treated as separate conversations. Organizations need guardrails around both the decisions AI makes and the authority it has to act on those decisions.

Governance should scale with risk, authority, and consequence

One lesson that has resurfaced to me over the years is that technology exposes weaknesses in the foundations underneath it. With AI, those weaknesses might include poor-quality data, inadequate governance, unclear policies, poorly defined processes, ambiguous accountability, insufficient skills, or an organizational culture that isn’t prepared for the technology being deployed.

As AI becomes more autonomous, those weaknesses can become more consequential. A generative AI system operating on poor data might produce an incorrect answer. An autonomous system operating on poor data might act on that answer. That means that governance should scale with the authority delegated to AI, the risk inherent in the decisions it makes, and the potential consequences of its actions. That means organizations need to understand not only what an AI system is capable of doing, but what it should be permitted to do.

What data can it access? What systems can it interact with? What decisions can it make? Are those decisions explainable and fair? What actions can it take independently? When is human approval required? How will its behavior be monitored? Who is accountable when something goes wrong? And under what circumstances should its authority be restricted or revoked?

These aren’t questions to answer after deployment. They should help determine whether and how much autonomy should be granted in the first place.

“Human in the loop” isn’t enough

This also raises a question I’ve been exploring in my research on human oversight. It is easy to say that a human will remain in the loop. But what does that actually mean?

If someone is asked to approve an AI recommendation, do they have enough information to challenge it? If an agent takes hundreds or thousands of actions, can a person meaningfully review them? If people become accustomed to approving AI recommendations, does oversight eventually become a rubber stamp?

And explainability becomes particularly important here. We can’t say that a human is meaningfully exercising judgment if that person doesn’t have enough information to understand why the AI reached an important decision.

Meaningful human control isn’t simply about inserting a person somewhere in an automated process. It requires deliberately deciding where human judgment is necessary, what information people need to exercise it, what they are accountable for, and whether they can realistically intervene.

We have a window to get this right

Enterprises don’t have to wait until autonomous agents are embedded throughout their operations to figure out how to govern them. They can start now. That means understanding what AI systems and agents are already being used. Assessing the decisions they make and the risks associated with those decisions. Establishing boundaries around data and system access. Testing for bias and fairness where appropriate. Determining what needs to be explainable and to whom. Defining which decisions require human judgment. Monitoring what AI systems actually do—not simply what they were designed to do. Establishing accountability. And making sure authority can be restricted when circumstances change.

The question is how much authority are we prepared to give AI inside our organization, and do we have the responsible AI practices, governance, guardrails, and meaningful human oversight to manage that authority? With agentic AI still relatively early in production and AI governance still developing, the time to answer these questions is now.

The goal shouldn’t be to govern autonomy after we’ve deployed it. It should be to decide how much autonomy we’re prepared to govern before we grant it.