Welcome, our AI User Agents

Previously we took a look at what a UA is, the degree of trust, and the power imbalance. If you didn’t get a chance to read up on it, now is the time. I’ll wait!

arrow_forward The increasingly inaccurately named User-Agent

This post expands the concept a little bit.

User-Agents with AI superpowers …

I’m talking about AI-powered personalized user agents (referred to as AIUAs in this article). That’s a lot of adjectives, so let’s break it down:

• AI powered means that a substantial amount, if not all, decision making will be done by an AI.
• Personalized means that it will have and continue to accrue information about you that it will use to make decisions.
• User Agent means that it will do stuff on your behalf, but unlike before, the actions aren’t limited to fetching stuff from the internet and showing you pretty pictures. The agency extends to performing actions, even interacting with other people on your behalf.

Since they collect and interpret massive amounts of user data and perform actions on behalf of the user, by design, these agents must have access to deeply personal data (Yang et al., 2025).

… need to feed on your data …

Data sharing is a non-negotiable. For now, these AIUAs will rely on a remotely located brain ¹. Hence, the logic applied to decisions affecting you will be unknowable. More distressingly, it may even be unknowable to the people who built them (Aysel et al., 2025). This opacity largely stems from the complexity of AI models, which involve numerous layers of computation, making their inner workings difficult to interpret. Additionally, a lack of comprehensive interpretability tools contributes to this issue, leaving both users and developers with a limited understanding of their decision-making processes (AryaXAI, 2025).

… but can’t swear any allegiance.

In the classic UA trust model, if you see a recommendation for a product, you know a handful of entities whose interests this recommendation or ad serves. With AIUAs, you will have no idea if a recommendation serves your best interest or someone else’s. It could try to help you but fail because its training data is biased toward someone else’s interests. This bias is difficult to detect. (Ryan, 2025)

They are opaque, even to those who make them.

Even with a relatively simpler system like the UA with well-defined trust boundaries, trust remains a difficult problem. We are moving towards indecipherable systems like AIUAs with even fuzzier trust boundaries. We don’t have thousands of pages of specifications for these AI systems. Instead, there are trillions of training tokens that only offer loose guidance. Not to mention basic attack vectors like prompt injection still haven’t been sufficiently mitigated ². Nobody can guarantee where AIUAs’ allegiance lies. (OpenAI, 2025)

They are deceptively trustworthy …

AIUAs can easily gain people’s trust. Some people even fall in love with them (Demopoulos, 2025) or follow them to their deaths. Our politics, values, and consumption habits are easy pickings for a mildly motivated AIUA to manipulate. To make things even worse, people trust what AIUAs tell them about the world without checking. The more people choose AIUAs as the goggles with which they see the world, the more these agents can shape a convincing alternate reality.

… and difficult to align.

Perhaps the most interesting example of an intentionally modified foundational model is Grok, Elon Musk’s xAI chatbot. In July 2025, after Musk announced he had “improved” Grok to make it less “politically correct,” the chatbot began posting antisemitic content on X. This included praising Hitler and making false claims about Jewish people. (Siddiqui, 2025)

Grok isn’t an outlier. DeepSeek shocked the foundation model world when it came out. It was an open-weight model from China whose performance was comparable to the biggest foundation models at the time. Yet there was a problem: DeepSeek didn’t like to talk about Tiananmen Square. (Lu, 2025)

Trying to steer foundation models toward or away from bias often leads to outcomes that diverge from intentions. (Sun et al., 2025) While it sounds like self-sabotage, it’s not absurd to suggest that Google would comply with a request to politically “align” Gemini —hypothetically of course. Google scaled back its DEI hiring goals and other diversity programs in early 2025 after facing political and legal pressure. (Associated Press, 2025) Standing up for what’s right at the expense of profit seems an unrealistic expectation.

In summary,

• We see the world through complex yet predictable machines — called user agents. They are difficult to understand, but they do our bidding, more or less.
• We are moving towards handing over our agency to even more complex machines. But this time, they are unpredictable, poorly understood, and pretty much impossible to align with our interests in a verifiable way.
• The design and behavior of these complex machines may be affected by conflicts of interest that are hard if not impossible to detect.

That’s terrifying. But what can we do about it? In the next part of this series, we will delve into practical strategies and explore cutting-edge research to address and mitigate the risks associated with AI-powered user agents. We’ll examine the role of transparency in AI development and look at how emerging frameworks could help ensure AI systems act in our best interests.