In 2009, Harvard professor Jonathan Zittrain’s book, The Future of the Internet – And How to Stop It, warned of a digital landscape dominated by “tethered appliances” that could stifle innovation and facilitate unprecedented control. Fifteen years on, as AI agents evolve from simple chatbots to autonomous decision-makers, Zittrain’s concerns have pivoted to encompass the ‘untethered’. In a thought-provoking piece for The Atlantic this week, he sounds a alarm about the rise of agents – autonomous AI systems acting independently on behalf of humans. These agents, Zittrain argues, could proliferate out of control, potentially causing harm long after their original purpose becomes obsolete. “Give it a few goals, let it have a bank account, let it spend money… who knows where it ends up?” he cautions, highlighting the need for safeguards.
Concern over the impact of software agents, bots, or autonomous entities is not new. In 2010, a flash crash caused by high-frequency trading algorithms wiped out $1 trillion of stock market value in minutes, serving as an early warning of the unintended consequences of automated systems.
Investment is now pouring into the development of ‘agentic AI’. Altera, for example, is developing agents that can play Minecraft alongside human users. This may seem like a trivial application, but it represents a interesting approach to exploring the nuances of human-AI interactions. “We are developing advanced AI agents that coexist with us in virtual environments,” explains an Altera spokesperson. “These agents possess strong pro-human social-emotional intelligence and will eventually attain self-awareness.” Altera posted on X this week showing their Minecraft agents developing aspects of an collaboration; “our agents are now logging their progress on google sheets too. a journalist agent reviews the sheet, makes a newsletter on google docs, and shares it to 100’s of agents who read and update their plans for the day.”
While the potential benefits of socially aware agents that work tirelessly to achieve complex goals are immense, the risks are not to be ignored. Zittrain warns of the dangers of “set it and forget it” entities that could operate indefinitely, potentially causing harm long after their original purpose has become obsolete. “Agents set to translate vague goals might also choose the wrong means to achieve them,” Zittrain cautions.
Cloudflare’s recent launch of a free tool to combat AI bots scraping websites suggests an emerging cat-and-mouse game between agents and digital asset owners. Perplexity have recently got into hot water for this. Cloudflare who cache a sizeable proportion of the world’s Internet material stated “we fear that some AI companies intent on circumventing rules to access content will persistently adapt to evade bot detection”.
Takeaways: Proposals for agent regulation, including the idea of implementing a “time to live” feel unlikely to succeed given the variety of architectures that can be used to create a multi-agent system. But when developing agent systems, the range of complex outcomes must be considered and carefully managed. Businesses should implement testing frameworks that continuously simulate many scenarios. They should also establish clear guidelines for agent behaviour and decision-making processes and extend risk and control frameworks to include this new construct, ensuring human oversight at critical junctures.