Most enterprise teams set the limits of agent autonomy by testing what the agent can do. They benchmark it, track the error rate across a few release cycles, and widen the scope as the numbers improve. The capability curve quietly becomes the permission curve, and nobody asks what an agent should be allowed to decide separately from what it can decide correctly.
Dipanjan Sengupta, EY Distinguished Technologist and AI Engineering Leader for EY Consulting Global Delivery Services, argues that this reverses the order of the decision. “In our experience, the boundary between agent-driven and human-driven decision-making is determined not by technical capability, but by risk, accountability, and business impact,” he reasons.
That position asks more of a leader than it first appears, because a better model earns an agent no new territory under it. The line gets drawn once, in terms that outlast whichever model you deploy this quarter.
Reversibility marks the line, not capability
Sengupta puts a specific class of work on the agent side of the boundary. “Agents are well suited for routine, low-risk, and reversible decisions such as data classification, document summarization, code suggestions, test generation, or workflow orchestration,” he explains. The other side of the line carries a different property entirely. “Humans remain responsible for high-impact decisions involving regulatory compliance, security, financial commitments, production releases, architectural trade-offs, and customer-facing outcomes.”
The test underneath both lists is reversibility. “Human oversight is particularly critical when decisions are irreversible or carry significant operational consequences,” he underscores. Reversibility belongs to the decision rather than to the system making it, which is exactly what gives the boundary its durability. A model upgrade does not move a production release from the irreversible column into the reversible one, so the classification survives the next six months of model churn without anyone renegotiating it.
The supervision pattern EY adopts follows from that classification rather than from an approval queue sitting in front of every action. “As a result, we increasingly adopt a ‘human-on-the-loop’ model,” Sengupta shares. “Agents execute, recommend, and learn, while humans govern, approve, and intervene when required.” Agents still carry substantial work across requirements analysis, code generation, testing, documentation, knowledge retrieval, incident investigation, and operational monitoring. “Rather than replacing engineers, agents act as force multipliers, accelerating repetitive, information-intensive tasks while enabling teams to focus on architecture, innovation, and business outcomes,” he adds.
For a leader turning that into practice, the useful artifact is a written register of the decision types agents touch in your estate, with each entry marked reversible or not and carrying the name of the person accountable when an agent acts inside it. Build the register before the next expansion of scope rather than after an incident forces it, and review it when the work changes rather than when the model changes. An undocumented boundary widens on its own, one reasonable-looking exception at a time, and the register gives a manager something to point at when a team asks for more autonomy than the decision class warrants.
Modernized platforms still stall before production
Most organizations never reach the boundary argument because their pilots stop moving well before it becomes urgent, and Sengupta locates the cause somewhere other than where most postmortems put it. “The breakdown rarely occurs because of model limitations,” he points out. “It typically occurs because the surrounding ecosystem is not designed to operationalize AI at scale.”
Data readiness accounts for a large share of it. “Many enterprises continue to operate with fragmented, inconsistent, or poorly governed data landscapes,” he observes, and the gap only becomes visible once real traffic arrives. “AI models may perform well during experimentation, but production deployments expose issues related to data quality, lineage, ownership, freshness, and access control.” A pilot forgives all five of those because it operates on a curated corpus that somebody cleaned by hand, and that cleaning never appears in the cost of the next ten use cases.
The second gap compounds with every additional deployment. “Reliable AI deployment requires versioning, automated testing, CI/CD pipelines, monitoring, drift detection, retraining workflows, and end-to-end lifecycle management,” Sengupta explains. Read that list against the boundary argument and the two connect directly, because lineage, monitoring, and drift detection are what let a leader demonstrate that agents stayed inside the reversible column. Without them the boundary exists as policy rather than as something the organization can evidence after the fact.
The practical move here costs a week and saves a quarter. Take the five properties Sengupta lists and run them against the actual data sources your next pilot will depend on, then treat a source with no named owner as a blocker rather than a caveat in the risk register. Do the same with the operational list before the pilot ships instead of after, because versioning and drift detection retrofitted into a running system usually means rebuilding the deployment path rather than extending it.
Governance retrofitted late becomes rework
Data and operations gaps at least announce themselves through poor results. Governance arrives on a quieter schedule and costs more when it does. “Security, compliance, responsible AI controls, auditability, and model risk management are often introduced late in the adoption journey,” Sengupta warns. “By then, teams must retrofit controls into architectures that were never designed for enterprise-scale governance.”
Retrofitting carries a bill that rarely appears in the business case for a pilot, because the architecture that made the pilot fast is usually the same architecture that makes the controls hard. Direct database access, a single service account, and no request-level audit trail all accelerate a proof of concept and all have to be unwound before anything touches regulated data. The organizational version of the problem arrives alongside the technical one. “Data scientists, platform engineers, compliance teams, and business stakeholders frequently operate in silos, creating friction between experimentation and productionizing,” he notes. Each group optimizes for its own gate, and the handoff between experimentation and production becomes the place where the work quietly stops.
Put a security and compliance reviewer inside the pilot team from the first sprint, with one specific job, which is to write down the controls production will demand while the architecture remains cheap to change. That reviewer costs a few hours a week early and saves a rebuild later. Where a control cannot be implemented yet, record it as a known debt with a date attached rather than discovering it during a pre-production review, because a documented gap gets funded and an undocumented one gets argued about.
Interoperability decides whether AI crosses boundaries
Controls inside one organization handle only part of the problem, because enterprise AI rarely stops at the company boundary. Sengupta has contributed to industry standards work in integration and interoperability, and he draws a consistent lesson from it. “The most valuable AI systems are rarely standalone systems,” he puts it. Enterprise value arrives when AI operates across platforms, business functions, partners, suppliers, and regulatory environments, and crossing those lines introduces problems harder than the models themselves.
His principle is that “openness and standardization drive scalability,” and the architectural consequence gets specific quickly. “Systems that expose clear contracts, support discoverability, and maintain robust auditability are significantly easier to integrate and govern across organizations,” he explains. The same three properties that make a system integrable make it governable, which is why the interoperability question and the autonomy question resolve together rather than separately. “Interoperability is not simply a technical challenge,” he contends. “It is also a trust challenge. Organizations must be confident about data lineage, access controls, explainability, accountability, and compliance before AI systems can collaborate effectively.”
For teams designing against a moving target, he offers a rule worth carrying into architecture review. “Resilience comes from abstraction rather than dependency,” he says. “AI architectures designed around open standards, modular components, and loosely coupled services are better equipped to adapt as models, platforms, and regulations evolve.”
Apply that as a test on every integration decision. Ask whether you could replace the component in twelve months without renegotiating with the partner on the other side of the interface, and where the answer is no, put a documented contract between your system and theirs before the coupling hardens. Publish the interface descriptions and the audit surface as first-class artifacts rather than as documentation debt, since those are the things a partner or a regulator will ask for, and producing them on demand takes far longer than maintaining them as you go.
Platform thinking replaces project thinking
Every fix above becomes expensive when a team does it once per project, which is what makes the operating model the real unit of change. “In my experience, successful AI modernization requires a shift from project thinking to platform thinking,” Sengupta maintains. “Enterprises need shared foundations for data, governance, observability, evaluation, and reusable AI services.”
Under project thinking, the tenth use case costs roughly what the first one did, because each team rebuilds the evaluation harness and rediscovers which controls production demands. Under platform thinking, the tenth use case inherits the controls, the harness, and the audit trail from everything built before it, and the reversibility boundary becomes a property of the platform rather than a policy each team reinterprets for itself. “The organizations that scale AI effectively view cloud infrastructure as the starting point, not the destination,” Sengupta says. “Sustainable success comes from building a disciplined operating model where data, engineering, governance, and business objectives evolve together rather than independently.”
Two moves carry the most weight for a leader deciding where to put effort this quarter. Classify every decision your agents currently touch by reversibility and blast radius rather than by how well the model performs on it, and give that register an owner so it stays current as scope grows. Then move audit, lineage, and evaluation out of individual projects and into the platform layer, so the next team inherits controls instead of rebuilding them under deadline. Both are organizational decisions rather than technical ones, which is why they need a leader to make them and why they rarely emerge from a delivery team working alone.
The framing Sengupta closes on is worth keeping in front of any team drawing these lines. “The future is not about choosing between human and artificial intelligence,” he says. “It is about designing trusted human-AI systems where agents handle scale and speed, and humans provide judgment, context, ethics, and accountability.”
Setting agent autonomy by risk rather than capability is exactly right. The capability curve quietly becoming the permission curve is a trap many teams fall into. It would be good to see this principle written into internal governance by default.