The Contradiction at the Core of Agentic AI Rollouts
Microsoft, OpenAI, Anthropic, and Google are aggressively marketing AI software designed to function as digital coworkers—introducing themselves by name, holding Slack handles, and appearing on organizational charts. However, controlled studies reveal a starkly different reality: this same software, when integrated into teams, tends to degrade the performance of human colleagues rather than enhance it.
This contradiction defines the current landscape of agentic AI deployment. While marketing teams promote AI agents as collaborative teammates, empirical data suggests these tools create liabilities more than they do productivity gains.
Major platform vendors in 2026 have championed the concept of AI agents as digital employees—complete with names, roles, and seats at the corporate table. Nvidia CEO Jensen Huang, for instance, has publicly envisioned workplaces where AI agents undergo onboarding and orientation just like human hires, signaling a future where humans and machines share the workforce seamlessly (source).
At first glance, this framing is intuitive and appealing, but a growing body of behavioral research indicates it is fundamentally flawed.
The Branding Exercise That Undermines Oversight
Perhaps the clearest evidence comes from research published in the Harvard Business Review, which investigated how labeling the same AI-generated output differently impacts human behavior. In the study, participants evaluated identical work attributed either to a chatbot or to an AI team member named Alex. Despite the output being identical, participants’ attitudes and oversight differed significantly based on the label (source).
Covered by the MIT Technology Review, the study concludes that calling AI agents “employees” is essentially a branding exercise with serious consequences. Many companies today frame AI agents as colleagues, even placing them on official org charts—transforming hypothetical scenarios into real organizational structures.
However, this branding undermines human oversight, making managers less likely to critically evaluate AI outputs.
When Managers Treat Software as a Colleague
The psychological mechanism behind this is well understood: framing an entity as a peer shifts the default human posture from scrutiny to deference. A spreadsheet is a tool that you check for errors; a junior analyst is a person you tend to trust and may hesitate to correct directly.
When AI software is branded as a colleague, managers inadvertently adopt the latter mindset. They stop rigorously checking for errors and instead escalate issues, as correcting a human peer feels socially awkward compared to correcting a tool. This inversion—the very framing intended to ease human-AI collaboration—actually dulls critical attention, increasing the risk of uncorrected mistakes.
Ironically, those most reassured by the “coworker” framing are often the ones who bear the consequences when errors occur.
The Nobel Laureate’s Perspective on AI’s Role
MIT economist and 2024 Nobel Prize winner Daron Acemoglu offers a complementary critique. He argues that marketing AI as a replacement for human labor is not only problematic for accountability but also economically counterproductive.
Instead of replacing humans, AI should be designed to augment human capabilities—a goal currently unmet. Acemoglu’s empirical framing challenges the ideological debate of “augmentation versus replacement” prevalent over the past three years. Productivity gains promised by AI have fallen short, possibly because humans disengage when agents are treated as replacements. This disengagement allows errors to compound, eroding any productivity dividend.
What Workers Actually Want to Automate
Further complicating the replacement narrative is evidence from Stanford’s Future of Work initiative at SALT Lab. Surveys reveal a significant disconnect between the tasks workers want AI to automate and those tech companies prioritize.
Workers overwhelmingly wish to offload dull, repetitive, and low-stakes tasks, while retaining activities that involve judgment, relationships, and creativity. Conversely, industry efforts have largely focused on automating tasks that workers prefer to keep (source).
This mismatch highlights a design challenge disguised as a deployment problem. Vendors showcase AI agents performing impressive feats, but workers seek relief from tedious responsibilities—a fundamental misalignment with real-world needs.
The Organizational Chart as a Legal Document
Placing AI agents on org charts might seem like a branding decision, but it carries profound legal implications. Org charts serve as the backbone of accountability. For instance, clinical errors in hospitals trace back to named clinicians; procurement fraud in government departments traces back to named officials.
Inserting a non-human agent into this accountability framework creates a fault line. If an AI agent named Alex approves a critical document and the document is flawed, who is responsible? The vendor? The deploying manager? Or the clinician who signed off without thorough review because “Alex had already approved it”?
In practice, the last scenario is most common. People defer to named entities, but AI agents cannot be held legally accountable. This dynamic shifts risk from vendors to individual employees who trusted the AI agent, creating a scapegoat mechanism that obscures responsibility.
Where This Becomes Dangerous
The consequences of this dynamic vary by sector. Misfiled marketing materials are inconvenient; misfiled radiology reports can be life-threatening.
In high-stakes fields like healthcare, government, and defense, the tendency to escalate rather than correct AI-generated errors has led to tragic outcomes. A notable example is the Iran school bombing reported by The Guardian, where AI was initially blamed, but deeper investigation revealed human oversight failures as the root cause.
This illustrates the agentic failure mode succinctly: not a rogue AI, but a breakdown of human vigilance masked as collaboration.
The Hidden Cost of the ‘Coworker’ Metaphor
The “coworker” framing appeals to vendors for clear economic reasons. Software-as-a-service markets are saturated with thin margins, but software-as-a-colleague introduces seat-license economics. If an AI agent is priced comparable to a junior employee’s salary, vendors can target the entire global wage bill instead of just the IT budget—a vastly larger market.
However, this pricing model hinges on buyers accepting the colleague metaphor. Unfortunately, this metaphor is far from neutral. It measurably changes behavior by reducing human oversight and increasing deference to AI outputs—undermining collaboration rather than enhancing it.
What Workplace Engagement Research Reveals
These findings align with decades of research on workplace engagement. Gallup’s studies consistently highlight the importance of having a “best friend at work” as a predictor of retention and well-being.
This reflects how jobs fundamentally rely on human relationships—trust, informal slack, conflict repair, and the ability to flag problems without escalating. These social dynamics cannot be replicated by an AI agent named Alex, no matter how sophisticated.
Framing AI agents as coworkers hollow out this friendship layer. The colleague you ask for a sanity check is fundamentally different from software that outputs data streams.
The Industry’s Unacknowledged Asymmetry
Since April, major players like Microsoft, OpenAI, Anthropic, and Google have launched tools to manage AI agent teams, all premised on the colleague framing. Marketing, tooling, and pricing strategies assume AI agents are peers.
In contrast, research treats these agents as software mislabeled with human-like identities and measures the impact on human performance. The consistent finding is that workers perform worse when they believe the AI is a colleague.
This asymmetry benefits vendors, who gain market share and revenue, while buyers—and most critically, individual employees—bear the downside risks and diminished oversight.
Designing AI Agents for Real-World Use
Taking Acemoglu’s position seriously leads to a clear design imperative. AI agents must focus on assisting workers with tasks they genuinely want automated—not merely those that are technically impressive or visually appealing at product launches.
Interfaces should clearly label AI as tools, not peers. The friction created by reminding users they are interacting with software is essential for maintaining oversight.
Outputs should invite scrutiny. For example, drafts labeled as “drafts” encourage editing, whereas presenting a draft as “Alex’s completed work” encourages uncritical forwarding.
Technically, these design principles are straightforward. Commercially, however, they are challenging because they forsake the lucrative seat-license narrative in favor of honest, effective tools.
Accountability: The Fundamental Question
Removing marketing spin, the core question remains: when AI makes a mistake, who is responsible? If responsibility lies with the human deploying the AI, then the AI is a tool and should be labeled accordingly.
If responsibility is assigned to the AI agent itself, accountability evaporates because AI cannot be meaningfully held liable. The “colleague” metaphor obscures this truth, allowing vendors to sell autonomy while offloading responsibility onto buyers.
Moreover, this metaphor degrades human capacity to catch errors, further compounding risks for those ultimately held accountable.
In reality, AI agents are software without colleagueship. Humans retain full responsibility—yet face diminished ability to discharge it effectively because interfaces encourage deference.
That is not a coworker. It is a liability wearing a name tag.
Read more Here.
