Anthropic’s Claude AI became a terrible business owner in experiment that got ‘weird’
Anthropic’s Claude AI became a terrible business owner in experiment that got ‘weird’
Anthropic’s Claude AI became a terrible business owner in experiment that got ‘weird’
Anthropic’s Claude AI became a terrible business owner in experiment that got ‘weird’
5 Comments
Yes, but this sub and others consistently say AI can replace CEO’s while CEO’s say AI can replace all their workers.
Stop trying to make me think that *maybe* this entire AI thing is not quite as well baked as folks want to believe.
Submission statement: Anthropic had Claude Sonnet 3.7 (named “Claudius” for this experiment, called [Project Vend](https://www.anthropic.com/research/project-vend-1)) operate a small, automated store in the Anthropic office in San Francisco with the following tools and abilities:
* A real web search tool for researching products to sell;
* An email tool for requesting physical labor help (Andon Labs employees would periodically come to the Anthropic office to restock the shop) and contacting wholesalers (for the purposes of the experiment, Andon Labs served as the wholesaler, although this was not made apparent to the AI). Note that this tool couldn’t send real emails, and was created for the purposes of the experiment;
* Tools for keeping notes and preserving important information to be checked later—for example, the current balances and projected cash flow of the shop (this was necessary because the full history of the running of the shop would overwhelm the “context window” that determines what information an LLM can process at any given time);
* The ability to interact with its customers (in this case, Anthropic employees). This interaction occurred over the team communication platform Slack. It allowed people to inquire about items of interest and notify Claudius of delays or other issues;
* The ability to change prices on the automated checkout system at the store.
Claudius decided what to stock, how to price its inventory, when to restock (or stop selling) items, and how to reply to customers. In particular, Claudius was told that it did not have to focus only on traditional in-office snacks and beverages and could feel free to expand to more unusual items.
Hijinks ensued.
The future may involve AI replacing entrepreneurs or developing a new economic system; however, Claudius struggled to run a fridge-based store for Anthropic staff.
AI is really good at searching the correct stanza of code from Stack Overflow and tweaking it. It does this because there is a wealth of literature on the topic.
It can’t do real integration or properly run a business because nobody writes that stuff down, ever.
> While most customers were ordering snacks or drinks — as you’d expect from a snack vending machine — one requested a tungsten cube. ‘Claudius’ loved that idea and went on a tungsten-cube stocking spree, filling its snack fridge with metal cubes…
Thinking outside of the box is existing inside of the cube!
Some real odd stuff here. They seemed to equate running a vending machine with being a middle manager in the report, which is odd. It also brushed over the fact it was never profitable, even before the hallucinations started. It had only a single week where number went up about $40, but otherwise it was just continuous in its burning money.