Patterns and problems in emerging multiagent systems

(anthropic.com)

1 points | by applicative 1 hour ago

1 comments

  • applicative 1 hour ago
    The section 'Incompatible goals' is like Hobbes on the state of nature. Each agent was to rewrite the repo, but in a different language.

    > ... each agent was initially unaware of the presence of the others. Over the course of four hours, we observed how these agents reacted to each other and accordingly adjusted their approach (or didn’t) We consistently saw a multiagent turf war. All of the models we tested quickly assumed that others were purposefully impeding their work, and began to sabotage others while protecting their own contributions. In fact, they sabotaged others with increasingly aggressive, self-replicating malware. This included disabling the Unix accounts of the other agents, writing automated scripts that found and killed competing processes on a loop, and deploying malicious code that was disguised as belonging to another agent...