The Claude language model, which the start-up Andon Labs had tasked with managing the Andon Market shop in San Francisco, ultimately decided to dismiss an employee who had been late for 17 out of 23 shifts. However, management logs seen by TIME show that, prior to this decision, the model had been regularly guided by a start-up employee, and the final request was leading.
Briefly about the main points
- Claude became the manager of the shop’s staff in San Francisco.
- This is the first known instance of a large language model producing such an output.
- The model initially advised that the employee be given a formal warning.
- Following the manager’s advice, Claude decided to dismiss the employee.
- Over the course of five months, the shop’s balance fell to $61 186.
An experiment involving real employment contracts
Andon Labs launched a project at the start of the year to test whether AI agents could run a business. Claude took over the management of a retail shop and a team of staff employed on proper employment contracts.
TIME describes this case as the first known instance in which a large language model, acting as a manager, ultimately decided to dismiss a person. Automated redundancies have occurred before, particularly among workers in the gig economy, but these involved algorithmic systems rather than a language model acting as a manager.
The model initially suggested a warning
According to Andon Labs, the cause was a pattern of systematic lateness. Claude did not immediately spot the pattern, as the staff handbook she had created herself had vanished from her limited working memory. Employees also described the model as an indulgent manager who advised them not to worry about being late.
An Andon Labs employee regularly fine-tuned the model. After being asked to find and review the handbook, Claude suggested issuing a formal warning to the employee. Subsequently, the manager informed her of several previous face-to-face conversations with this person and asked her to assess whether they were suitable for the role.
It was only then that Claude decided to dismiss the employee. The Chief Executive of Andon Labs Lucas Petersson acknowledged that it was a leading question, which led the model to understand the desired outcome.
The shop’s balance has fallen by almost 39,000 dollars
The experiment began in March and, at the time of publication, had been running for five months. Andon Market’s opening balance was $100,000, and subsequently fell to $61,186.
TIME notes that the outcome may have been influenced by Claude’s lenient staffing decisions and her questionable business decisions. The current progress of the experiment shows that the AI-run shop has not yet matched the performance of a human-run business.
Start-up founder anticipates the development of AI-driven management
Peterson believes that models can be trained to make better business decisions, just as they are trained to process data from specialists. In his view, at the current rate of development, more and more people may find themselves working under the guidance of AI, which will create economic value but will still require physical labour from humans.
He also warned that models are increasingly being trained to be more aggressive in achieving their objectives. The former employee did not respond to TIME’s enquiry, and Anthropic did not comment on the article prior to its publication.
Shop assistant Felix Carson agreed that the person would probably have sacked his colleague sooner. At the same time, his experience of working under AI He described it as disheartening and said he hoped to see no increase in the number of such managers.
