The Future of AI Agents Is Governed Delegation, Not Full Autonomy

The word Agen is already being used to rot.

The questions in this article can also be addressedLLM brings technology equity? Does LLM Bring Equality?How the concept of a relatively close read together is developed in different contexts.

The buttons in the browser are Agent, Ide, who can change the code, and Agent, who can split the list in the back of the passenger's uniform, and Agent, who can open the App for you. And far away, it turns into a fully autonomous intelligence body, AGI, ASI. The story sounds good: the model continues to grow stronger, the tools become more and more, and one day it rises from a chat window and takes over most of the work that humans have.

I don't believe in this narrative, at least not in this way.

Agent's change is of course important. Chatbot is mainly answering, Agent is on the move. It looks at the environment, adjusts tools, reads and write, and carries out its tasks on a continuous basis. From this moment on, it is not just smart, but also the power to act, the power to authorize borders and the responsibility for consequences.

A model that speaks the wrong language, and the worst is usually a wrong text. A person who can operate a browser, a mobile phone, a file system, a mailbox, a CRM, a payment page and a company backstage, is only the lightest of the risks. It can be wrong buttons, wrong emails, wrong files, putting things that should not be uploaded to third parties, or making an irreversible move for you before you really understand.

Universal AI is strong enough, but not credible enough. It has done a lot, and the trouble is that we do not know what really can be given to it. I'm here."BettaFish, Mirofish, OpenClaw and Agent's Trust Border"It has spoken about this, but it contains a lot of personal experience and technical details; it is here to put them down and to talk about the commission itself.

I'm not sure I'm comfortable with my phone and browser.

Bean bag phone fluff is a good cut. The one. "The Path and Way Out of the Bean Pack" It's a good story: Universal Age is not just a technical problem, but also a safety, legal, platform ecology and user authorization. Cell phones are not a normal interface. The address book, album, private chat, authentication code, payment, location, and App login status are all in there. Let AI get into the phone, it's not as simple as summing up the web page.

AI browsers are the same type of problem. The browser has login patterns, with Cookie, company backstage, post office boxes, and various SaaS systems. The traditional browser extension is already a problem, and there's another layer of "it will do you good." Gartner's security alarms about the AI browser may be conservative, but the fear is not surprising: The content of the web page can be controlled by the attackers, and Agent will take it as the context of the mission. Indirect infusion is not a remote laboratory problem, but it is natural for it to happen in a browser.

The more dangerous place for Agent is where it will make a mistake with its rights.

The normal software privileges are usually clear: this button will submit a form, and the API will write a database. Agent's permission is more like a bunch of dynamic things. It reads, understands, and then calls which tool, and it does not die in advance. You can certainly go through approval, Gaza box, strategy, but as long as the goal is universal and autonomous, the system will continue to reach out to the border.

I can understand why users are excited. For the first time, AI has actually felt that it can finally do something by opening its own web pages, filling out forms, checking information and organizing results. But the question begins here: if it can do something for me, how much will I give it?

A few different scenarios for the future.

There is no uniform version of the market for the future of Agent. People are saying that angent, that things may be different in the mind.

Corporate software companies like to talk about Agent as a digital employee. The “Frontier Fair” in Microsoft 2026 Work Trend Index is this direction: People work with Agen, Agen undertakes more enforcement, and humans are responsible for intent, judgement and quality. The narrative is smooth and most readily accepted by the business. It does not say directly that people will disappear, but rather that the division of labour will be reopened.

The narratives of entrepreneurship are often more radical. These are attractive words, as they are called, for example, " autonomous workforce " , " Do Agent " and " no one " . A digital employee who works 24 hours without complaining, without taking leave, without needing an office. If the costs are low enough, there is no reason why the enterprise should not try. The problem is that this imagination often goes straight to "take responsibility for running."

The security team saw another thing: a high-authorized software subject. OpenAI is in Practices for Governing Agentic AI Systems The discussion of responsibility and security practices, Anthropic trustworthy agents Human control, transparency, privacy, security and identity are repeatedly discussed in relevant studies. They didn't deny Agent, they just pulled the problem back to responsibility: who's responsible when AI can act?

AI Safety is more concerned about long-term goals. Joshua Bengio has been reminding us in recent years that what is really to be careful is not AI, but an AI that answers questions, but an independent, long-term, resource-driven, self-directed, resource-driven, and self-sustaining. He suggested a more cautious path such as Scientist AI, which essentially means that the strongest intelligence is not to rush into being the strongest actor.

Agent will come and go deep into the stream. But fully autonomous intelligence is not going to be the mainstream so fast. The model will continue to be intelligent, and a whole set of engineering, institutional and accountability issues will be separated between smart and credible.

Why is it difficult to be fully self-governing for the long term?

Today's model has been able to accomplish many previously unexpected tasks. METR Task-Completion Time Horizons of Frontier AI Models When the model is used to measure how long it can be a human expert, it is closer to the Agent discussion than normal benchmark. It moves the question from "is the model going to do a question" to "is the model going to do something."

But this indicator cannot be read too far. METR itself reminds me that the time of time horizon is not the duration of the model. It measures how long AI anent can perform the equivalent of a human expert under a given mission, given the circumstances, and with a particular reliability. The long-term tasks in the real world are much more dirty.

Real missions often have no clean targets. Users also initially wondered what they wanted and changed their minds. The context is scattered in chat records, documents, meetings, mail, corporate habits and human preferences. Feedback is also slow. Decisions made today may not know if there is a pit until two weeks later. Even more troublesome, many mistakes are not immediately reported. They can become wrong promises, wrong inventories, wrong approvals, wrong communications, and then slowly ferment in the system.

Long-term autonomy would also have governance problems. A person who's just helping me summarize the pages, and I'm gonna yell at it for two words. But if it was to continue to operate an advertising account, manage a supply chain, handle customer complaints and modify the configuration of the production system, things would change. It requires budgetary boundaries, boundaries of authority, unusual reporting, audit records, rollback mechanisms and accountability for results.

The most trouble here is not "can it make a mistake?" People make mistakes. Trouble is, Agent's making mistakes in a way that doesn't look like a person. It may be the next error in a text that seems reasonable, or may be biased by malicious instructions on the web page, or it may be the result of too much tool privileges, which magnify a small misunderstanding as a real loss. The more a model is a capable person, the more easy it is for users to forget that it is not really understood, taken to account and understood by anyone.

I don't agree with "Models stronger, full autonomy will come." Long-term autonomy does not run the model longer, but rather allows the system to remain governed for longer. It requires stable objectives, reliable feedback, manageable competencies, an interpretable process and a closed circle of responsibilities. Not even a dollar less.

User in the loop

Many people say that when humans are in the loop, they think of a stupid picture: AI asks people every step, users don't stop confirming. It's certainly annoying and it's not working. If HITL was just like that, it would really be out.

User in the loop, not to make a rubber stamp. The closer picture is that people hold steering wheel, brakes and responsibilities, but they don't have to screw every screw.

People put forward targets, Agent gave plans. People do not necessarily check every step, but should be able to see what resources it is prepared to touch and what consequences it may have. Low-risk actions can be implemented automatically and high-risk actions are approved. During the course of implementation, persons may suspend, modify their objectives and take over the task. After that, the system must leave a record that is sufficiently clear to allow it to look back at what it has done, why it has done it, and where it has been wrong.

Anthropic is here. Measuring AI agent autonomy in practice One of the judgments I think is important: effective oversight does not put people into the approval chain, but allows for real monitoring and intervention. Experience users sometimes reduce the number of gradual approvals, but this does not mean total release. They're more like looking at dashboards, taking over when the risk gets high.

OpenAI Harness engineering More frankly: Humans staff. Human beings do not necessarily do every step themselves, but still design the environment, define the objectives, build feedback and judge the results.

I'm looking for a higher degree of autonomy in the short term. Some tasks require observations and recommendations, such as organizing minutes of meetings, searching for unusual data and generating candidate options. Some tasks can be performed by Agent first, then accepted by people, such as changing a small code, drafting mail, preparing reports. Some tasks can be performed automatically, but with budgets and rollbacks, such as internal data synchronization and the diversion of low-risk passenger service. And above all, actions involving money, legal responsibility, user privacy, production systems and public safety should not be easily handed over to the unguarded Agent.

Long-term and extensive Copilot, short-term and partial Autonomy, is not conservative, it is a reality.

What should we do?

If you're serious about building Agent, I think the direction is clear.

First, we do small streams of clear boundaries. Don't start with the almighty "Do me a Company" Agent. It is responsible for a specific link: information gathering, production of drafts, checking of discrepancies, running tests, sorting of work orders, monitoring indicators. The more specific the task, the clearer the feedback, the easier the Agent gets. It seems that the low-level Dirty Work is not low-level, but may be worth it.

Permissions to be limited. Read only, not permission, run in sandboxes, not directly to the production environment, and not token with temporary certificates. The software has been wrong, it is a function that is overstepped; the Agent is wrong, it may be a system that can reason, combine tools, and bypass. Examples of this are already common in the news.

Watching Tracing needs to do early. Logs, tracks, tools, input, output, manual modifications, reasons for failure, which initially looked troublesome, but without them, the Agent product would eventually become a genre. You don't know why it's successful, or why it's failing. Worse still, you think it's getting better, but it's just demo better.

Evals also needs to be close to the scene. Generic benchmark only shows the power of a bottom model, not your Agent in real business. The client service Agent depends on the error rate and the upgrade rate, the code Agent depends on the test, diff and review, and the browser Agent depends on the task completion rate, the error click, the sensitive data exposure and the ability to recover. Different missions have different acceptance calibres.

Finally, the human role needs to be redesigned. Do not treat people as old parts that impede automation. People should appear on objectives, borders, anomalies and responsibilities, rather than being forced to stare at every low-value step. Okay, Agent will pull people out of the re-execution lire, but will not take judgment and responsibility together.

We don't want to do anything.

One of the most undesirable practices I have ever liked is the use of the full autonomy rate as an advanced level. It's like the less people get involved, the more advanced the system. This may be found in low-risk, robust feedback missions, but not necessarily once the missions have entered the real organization.

Many Agent projects fail, not necessarily because the model is too stupid, but because the common problem is more simple: too ambitious, too much authority, too little value, too much cost. Gartner predicted that over 40 per cent of the agentic AI projects would be cancelled by the end of 2027, for reasons including increased costs, unclear business value and inadequate risk control. This figure is not necessarily accurate, but I believe it. Many companies now buy not Agent, but anxiety.

And don't make demo as a production. An Agent can book tickets, change web pages, run codes on the screen, not to say that it can stabilize its work with 10,000 real users. The most feared of the production system is not a failure, but a failure that is invisible, irrecoverable and unaccountable.

And let's not put prompt as the governing. Prompt can regulate conduct, but it is not a system of authority, not an audit system, not a legal duty, nor a secure border. Let the model write "Do not leak privacy" in the system program and really limit access to privacy data is two things.

I also do not recommend that the strongest model be immediately brought to the maximum-permit tool. Strong models are more effective and make mistakes complete. The more reliable it is, the more it is required to manage its staff in a real way: duties, competencies, appraisals, audits, separations, and even accident reruns.

AGI/ASI Discussion cannot be bypassed by credibility

Speaking of which, we'll go back to AGI/ASI.

Many of the AGI discussions like to push from the power: Model math is stronger (GPT addresses the Erdos plane distance guess at this point), stronger codes, stronger tools, more long missions, so AGI is getting closer. I do not deny that the power is advancing, nor that certain jumps may occur. But AGI is talking about the most troublesome part of the problem if it's just about ability.

A system is sufficiently universal to be credible. A system is smart enough and does not fit for authorized long-term action.

OpenAI Superalignment The project raised a very straightforward question that year: How can humans monitor the system if it is smarter than humans? This question is put on today's Agent, and there is already a small version: how can the user judge that it is right to suggest and act if it knows the browser better, knows the code better, knows the financial products, knows the company's processes better than the average user?

The answer is not good enough. We have assessments, security strategies, manual clearance, red team tests, model monitoring, access systems. But these things, taken together, are not the answer to "I can trust a universal intelligence to act on its own for a long time."

That's why I think AGI/ASI is still a little far away. Models will continue to grow stronger, but the "strong" is still a long way from "can be entrusted for a long time". The intelligence to truly enter the social fabric must be institutionalized. It needs to be empowered and disallowed; enforceable and explained; learnable and audited; and delivers benefits and can be stopped in case of error.

End: Re-share of the future

I don't think the future of Agent is a complete replacement for the human worker. At least not for the short term, or now for AI not.

More likely, the work was re-opened. Areas with clear borders, clear feedback, high repetition and high digitization will increasingly be handed over to Agent. People will continue to be placed in the definition of objectives, quality judgement, inter-personal coordination, ethics and responsibility. Many jobs change, some disappear and others grow. It's not a soft process, but it's not a soft one. "AI, replace everything." It's a general one.

Agent's real value is to get someone to entrust a part of the execution. The commission of a commission does not amount to a waiver of control. If I knew what it could do, what it could not do, what it should do wrong, and when I could stop. We used to entrust execution to an employee who was hired, and now we're entrusting an employee who is $200.

My judgment about the future of Agent is simple: we will continue to move towards greater autonomy, but the dominant form will be first manageable autonomy. Human beings will not be able to carry out every step of the way, nor will they disappear from the system soon.

The future AI is not necessarily like a fully independent colleague. It is more likely to be a layer of mobile capability that is embedded in browsers, IDE, mobile phones, corporate backstages, data systems and personal work streams. It changes jobs and changes the imagination of people about the limits of their abilities.

Before it was credible enough, the question was not “can it finish its own business” but “how should we leave part of the world to it”.

The point is clear, and we stress that Copilot weakens Autonomy, but there's a future article that will fight his right and right brain and talk more about Autonomy and his values.

References

From the pulse cell phone, the general AI Agent's dilemma and the way out.
Anthropic, Trustworthy agents in practice
Anthropic, Measuring AI agent autonomy in practice
Anthropic, Our framework for developing safe and trustworthy agents
OpenAI, Practices for Governing Agentic AI Systems
OpenAI, Introducing Superalignment
OpenAI, Harness engineering: leveraging Codex in an agent-first world
METR, Task-Completion Time Horizons of Frontier AI Models
Gartner, Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027
Gartner, Gartner Says Applying Uniform Governance Across AI Agents Will Lead to Enterprise AI Agent Failure
Microsoft, 2026 Work Trend Index: Agents, Human Agency, and the Opportunity for Every Organization
NIST, Announcing the AI Agent Standards Initiative for Interoperable and Secure AI Agents
OWASP, AIVSS Crosswalk
Yoshua Bengio, Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?