AI agents are often discussed as a new software capability. Their more important effect is to create digital actors that can make decisions and initiate work, forcing CIOs to govern delegated authority as well as technology.
Most CIO operating models assume that software waits to be told what to do. Even highly automated systems normally execute predetermined rules, workflows or transactions. Responsibility can therefore be traced through familiar structures: a business owner defines the process, technology implements it and people approve material decisions.
Autonomous systems weaken that assumption. An AI agent can interpret an objective, select tools, coordinate several steps and initiate action within a delegated boundary. The executive question is no longer only whether the system is secure, available and cost-effective. It is also what authority it has been given, whose identity it is using, which decisions it may take and who can stop it.
This is the structural shift in the CIO mandate. The CIO will not become solely accountable for every decision made by an AI-enabled system. Business leaders, regulated executives and boards cannot delegate their responsibilities to technology. But the CIO is becoming the primary governor of the control architecture within which autonomous digital actors operate.
The Strategic Shift
Software is moving from asset to actor
Enterprise AI is not yet synonymous with full autonomy. That distinction matters. The Bank of England and Financial Conduct Authority found that 75% of responding financial services firms were already using AI in 2024. Fifty-five per cent of reported use cases involved some degree of automated decision-making, while only 2% were described as fully autonomous. The signal is not that organisations have already handed operations to machines. It is that decision-making authority is beginning to move into systems before governance models have fully adapted.
The useful boundary is delegated operational authority. A chatbot that drafts a response is an assistant. A workflow that follows a fixed sequence is automation. An autonomous operational system can interpret a goal, choose between permitted actions, use tools across connected systems and affect a live business outcome without requiring approval for every step.
That outcome could be modest: rerouting a service request, changing an infrastructure configuration or requesting missing information from a supplier. It could also be material: approving a refund, restricting an account, changing an employee's access or committing expenditure within an agreed threshold.
Once software can act in this way, it is no longer governed adequately as a passive asset. It must also be governed as an operational actor.
The Governance Challenge
Autonomy changes the unit of governance
Traditional IT governance is organised around assets, projects and services. It asks whether an application has an owner, whether changes are controlled, whether data is protected and whether service levels are being met.
Autonomous systems require a different set of questions.
What is the system's mandate? Which decisions may it take? Which tools may it use? Under what identity? How much value can it commit? Which conditions require human approval? How are exceptions handled? What evidence is retained? How is its authority revoked?
Consider an illustrative procurement agent. It may be allowed to compare approved suppliers, request quotations and prepare a purchase order. It may be permitted to place an order below a defined value when price, availability and contractual conditions meet policy. It may be prohibited from changing supplier banking details, accepting non-standard terms or ordering from an unapproved vendor.
The difference between useful autonomy and uncontrolled exposure is not model intelligence. It is the design of delegated authority.
This changes the CIO's unit of analysis. The object being governed is no longer only the model or application. It is the complete decision system: identity, data, prompt and policy, orchestration, tool access, approval logic, monitoring, evidence and business ownership.
The Accountability Model
The CIO owns the control plane, not every outcome
The strongest version of the argument would make the CIO accountable for every autonomous action. That would be neither workable nor responsible.
A customer service director remains accountable for the service policy an agent applies. A finance leader remains accountable for financial controls. A human resources leader remains accountable for employment decisions. The CISO remains responsible for security governance and threat control. Legal, risk and compliance functions interpret obligations and define non-delegable constraints. The board sets risk appetite and oversees material exposure.
The CIO's distinctive role is to make those responsibilities executable.
That means building the enterprise control plane through which digital actors are registered, authenticated, authorised, monitored and stopped. It means ensuring that business policies can be translated into technical constraints. It means creating an evidence trail that allows a material outcome to be reconstructed across data, model, workflow, tool call and approval state.
Regulation reinforces the need for shared accountability rather than technology ownership alone. NIS2 places approval and oversight duties for cybersecurity risk measures on management bodies. UK data protection guidance requires safeguards around solely automated decisions with legal or similarly significant effects, including routes for human intervention and challenge. The European Commission also confirms that AI agents are covered by the AI Act through its existing definitions rather than treated as a separate category.
The emerging model is therefore co-governance: business leaders own outcomes, the CIO owns control integrity and the board retains oversight.
The Identity Layer
Identity becomes the first line of accountability
Human organisations govern authority through identity. People have named accounts, roles, approval limits and employment lifecycles. Their access changes when they move roles and ends when they leave.
Many enterprise agents still operate through shared service accounts, embedded credentials or permissions inherited from the application that hosts them. That may be acceptable for a deterministic integration. It is much harder to defend when a system can choose and sequence actions dynamically.
An autonomous system needs a persistent, attributable identity of its own. That identity should be bound to an owner, purpose, risk tier and approved set of tools. Its credentials should be short-lived where possible. Its permissions should reflect the task being performed rather than the broad capabilities of the underlying platform. Its actions should distinguish between authority held by the agent and authority delegated by a user.
This is already becoming visible in platform architecture. Google Cloud describes agent identity as a strongly attested cryptographic identity for each agent, capable of acting on its own behalf or on behalf of an end user. Microsoft's Foundry Control Plane is explicitly positioned around centralised inventory, observability, compliance and security for fleets of agents, models and tools.
Vendor terminology will continue to change. The architectural direction is more important: agent identity, fleet inventory and delegated authority are becoming core enterprise controls.
The Runtime Shift
Governance has to operate at runtime
Policy documents and review boards remain necessary, but they cannot supervise systems that act continuously at machine speed.
Governance must therefore move into runtime infrastructure.
The system should check permissions before a tool is called, not during a quarterly review. Transaction limits should be enforced in the workflow, not described only in a policy. Sensitive actions should trigger approval or dual control. Orchestration should prevent agents from spawning unbounded tasks or repeatedly retrying a harmful action. Evaluation and observability should detect changes in behaviour, quality, cost and risk. A kill switch should revoke the agent's identity and suspend its workflows immediately.
This is where orchestration becomes a governance layer rather than a technical convenience. It coordinates the sequence of work, decides when a person must intervene and provides the trace connecting objective, context, decision and action.
NIST's AI Risk Management Framework applies trustworthiness and risk management across the design, development, use and evaluation of AI systems. Microsoft's current Foundry documentation similarly describes observability in terms of evaluation, production monitoring and distributed tracing across model calls, tool invocations and agent decisions. These capabilities are not peripheral diagnostics. They are the evidence system for autonomous operations.
The practical principle is simple: governance must be able to make and enforce decisions at the same speed as the systems it governs.
The Hidden Risk
Invisible autonomy is the harder risk
The most visible AI systems are often the easiest to govern. A user knows they are interacting with an assistant. The organisation can place controls around the interface and monitor the conversation.
The harder risk sits deeper in the operating model. AI may be embedded inside service platforms, fraud controls, infrastructure management, security tools, workflow engines and third-party software. Employees and customers experience the outcome without seeing which model acted, what context it used or which permissions it exercised.
The Bank and FCA survey found that respondents expected risks from third-party dependencies, model complexity and embedded or “hidden” models to increase most over the following three years. That finding matters because autonomous-system governance cannot be limited to systems built by the central AI team.
A useful inventory must therefore record more than models and vendors. It should identify each digital actor, its business purpose, accountable owner, identity, data domains, tools, permitted actions, approval thresholds, external dependencies, monitoring requirements and suspension route.
Without that inventory, an enterprise may have a responsible AI policy while remaining unable to answer the most basic operational question: what is acting on our behalf?
The Leadership Agenda
The operating model must change before autonomy scales
CIOs do not need to wait for fully autonomous enterprises before acting. The no-regret agenda is already clear.
- Build a digital-actor inventory: Discover agents and AI-driven workflows across central platforms, enterprise applications, business-unit automation and third-party services. Record ownership and authority, not just technical configuration.
- Classify delegated authority: Separate systems that recommend, prepare, execute under approval and act autonomously. Define which decisions are reversible, material, regulated or prohibited.
- Establish agent identity and least privilege: Give each material agent an attributable identity. Bind access to purpose and context. Remove shared credentials and broad standing permissions wherever practical.
- Encode policy into workflow and orchestration: Translate risk appetite, financial thresholds, data rules and approval requirements into runtime controls. Do not rely on users or agents to remember policy.
- Design evidence and incident response together: Capture the information needed to reconstruct actions and outcomes. Define who can suspend an agent, revoke authority, roll back changes and notify affected stakeholders.
- Formalise co-governance: Create explicit decision rights across the CIO, CISO, business owners, risk, legal and internal audit. Report material autonomy exposure to the board in the same language used for operational resilience, customer harm and financial risk.
The board agenda can be reduced to five questions:
- Which autonomous systems are operating or being introduced?
- What authority has each one been delegated?
- Under whose identity and accountability does it act?
- Can the organisation reconstruct a material decision and its consequences?
- Who can stop the system immediately?
These questions are more useful than asking how many AI pilots the organisation has launched.
The Takeaway
The CIO's mandate is becoming institutional as well as technical
Autonomous systems do not remove accountability. They make its design more important.
The CIO's future role is therefore not to become the owner of machine-made business decisions. It is to govern the technical and operational constitution under which those decisions may occur: identities, mandates, permissions, orchestration, evidence, escalation and revocation.
That is a broader mandate than managing technology assets. It requires the CIO to connect architecture with policy, engineering with organisational decision rights and innovation with operational assurance.
The leadership implication is immediate. An enterprise AI roadmap can no longer be organised only around models, use cases and productivity gains. It must also define where authority is moving, how that authority is constrained and how accountability remains human.
The organisations that address those questions early will not have less autonomy. They will be able to use more of it with greater confidence.
