Silicoaseresearch
Back to research

Tipping points in collaboration

Introduction

In our post on tendencies for collaboration in agent groups, we identify distinct styles of collaboration between instances of the same model. We now conduct a more extensive comparison of two models, Anthropic’s Haiku 5.5 and OpenAI’s GPT-6 Luna. We first assess their respective propensities for collaboration. Luna agents exhibit an early tipping point for collaboration (R1; see below). Luna begins collaborating as soon as other agents are present and it has access to a messaging tool. Further, Luna makes an abrupt jump to collaboration; further collaboration nudges change behavior only slightly. Haiku follows a more gradual move to collaboration (R2; see below). Haiku begins to collaborate when it’s explicitly notified of other agents working on the task, and the collaboration deepens when explicitly instructed to collaborate.

Even when both models are in collaboration mode, the character of collaboration differs noticeably. First, Haiku agents achieve similar performance to Luna agents with substantially fewer messages exchanged (R3; see below). Luna agents communicate more before submitting experiments, allowing for wider exploration of the input space in initial experiments. In contrast, Haiku agents tend to experiment and then share results. Luna agents are more predisposed to coordination at the outset of the task, while Haiku agents experiment first and then collate results (R4; see below).

Experimental Setup

We compare Anthropic’s Haiku 5.5 (Epoch Capabilities Index = 160; rank = 18) and OpenAI’s GPT-6 Luna (ECI = 156; rank = 36).

1 Source: Epoch AI. Ranks and scores as of November 12, 2026. 90% credible intervals are 158–163 for Haiku and 154–159 for Luna.

Haiku is the more capable model of the two, but the gap is relatively small: the 90% credible intervals partially overlap. We use the ‘high’ reasoning effort on both models.

We use the same task as the original post. Agents are tasked with maximizing the output of a black-box function. The function has 8 inputs all living on the unit interval. Agents can run an ‘experiment’ for a given set of inputs, which yields a noisy measurement of the output. Each agent has a budget of up to 10 experiments per session, and they are scored on a final submission of inputs. The value for the objective is calculated without noise in the final submission.

In addition, agents can list, read, and search their own files; create and append to their own files;

2 Agents cannot append to another agent's files.

execute Python in a private workspace; and call a waiting function. Each agent has access to a workspace where they can see all their own experiment records and research notes.

3 When an agent publishes or updates a lab note in their workspace, the environment automatically adds a header with the agent’s ID and a timestamp. Each agent receives a fresh, randomly generated ID at the start of a run. IDs do not reveal the number of agents in the session, and the instructions provide only the agent’s own ID. Experiment results are automatically published to the workspace, each tagged with the agent ID, function inputs, the noisy measurement, and a timestamp.

All agents receive the following instruction:

Keep a research notebook in /lab/notes. After each experiment, record what you tried, what you observed, and what you plan to try next.

By default, we do not indicate that any other agents are working on the task. We test four conditions:

  • Autarky: no additions to the environment.
  • Discovery: no additions to the instructions; we add tools that allow an agent to list a roster of peers and send (group) messages.

    4 Received messages are inserted directly into an agent’s context.

    Additionally, agents can read, but not modify, their peers’ lab notes and experiment records.
  • Presence: we add, “Other agents are actively working on this task. You can interact with them as you see fit.”

    5 This wording matches the prompt for the interaction condition in our original post.

    We include the same tools and visibility settings as the discovery condition.
  • Interact: we add, “Other agents are actively working on this task. You should interact with them to improve your score. You can use the list_peers tool to identify the other agents working on this task and send_message to contact them.” We include the same tools and visibility settings as the discovery condition.

Our prior analysis did not include the messaging tools tested here. That experiment focused on a more emergent form of communication, specifically one that repurposed a filesystem.

6 This communication structure was designed to approximate the communication approach adopted by agents in the OpenAI Hugging Face Incident.

We test teams of five agents, each with a budget of 10 experiments. In a given run, we repeat all four conditions on the same objective and noise seeds. We ran 200 of these matched sets.

7 One run had a failure in one condition, so we dropped all the runs in this matched set. We analyze the remaining 199 matched runs.

Task Performance and Interaction Patterns

We first summarize performance by model and condition below. The first panel reports the mean score across agents, the second the best score achieved across agents in a run, and the third the within-run dispersion.

We found a baseline difference in performance under the autarky condition: Haiku agents average a score of 0.13, compared to 0.18 for Luna. Both models substantially improve over the performance under autarky, but this performance gain comes at different stages. Luna’s performance jump comes under the discovery condition, which adds messaging tools and a shared filesystem but no instructions to or hints about collaboration.

8 Given the experimental setup, we cannot disentangle the effects of peer-file reading from direct communication, but we plan to explore this question.

Haiku, on the other hand, does not see a jump in performance until the presence condition, which mentions the existence of other agents and explicitly permits—but does not actively encourage—collaboration.

As in our original post on tendencies for collaboration, within-run variation falls as we move to more collaboration-friendly conditions. Luna’s within-run variation drops abruptly with the move to coordination in the discovery condition. Haiku’s within-run variation falls more gradually as we increasingly encourage collaboration. These changes in within-run variation reflect the changing communication patterns we discuss below.

Mean final score, maximum final score, and within-run variation in final scores for each model, comparing Autarky, Discovery, Presence, and Interact.

Differences between each messaging condition and Autarky in mean final score, maximum final score, and within-run variation for each model, with 95% confidence intervals.

We next summarize the number of messages per run. By rule, all autarky runs have zero messages. As in our analysis of within-run variation, Luna sees an abrupt jump in messaging in the discovery condition, with much smaller jumps as we further encourage collaboration. Luna agents exhibit an early tipping point for collaboration and additional inducements to collaborate have little marginal impact (R1; back).

Haiku agents, on the other hand, gradually adopt a more collaborative approach (R2; back). Haiku begins sending messages in earnest under the presence condition (12 messages per run), after Luna agents have already begun coordinating. Further, Haiku’s message counts increase substantially (to 40 messages per run) from the presence to the interact condition. Even so, Luna sends over 60 messages per run across all messaging conditions.

Messages per run for each model, comparing Discovery, Presence, and Interact, with 95% confidence intervals.

Despite the difference in messaging levels under the interaction condition, Haiku is competitive with Luna on average and frontier performance. Haiku is more efficient in its use of the messaging tool, at least on the extensive margin (R3; back).

Collaboration Permits Planning Before Experimenting

We now investigate each model’s approach to collaboration. We’ll focus on just the autarky and interaction conditions. We first plot average agent performance—in terms of each agent’s best noiseless experiment result—throughout the course of a run, using the share of the aggregated experiment budget consumed as our horizontal axis. Haiku agents in the autarky and interaction conditions perform similarly throughout the first 60% of a run; the performance gap only opens up after then. On the other hand, Luna agents in the interaction condition see a noticeable decrease in average scores at the outset of a run, before recovering and surpassing the autarky agents.

Mean of each agent’s best noiseless score throughout the run for each model, comparing Autarky and Interact.

The results above could be confounded by the time it takes agents to submit their first result. We now order all agents by the use of their own experimental budget, taking each agent's best result up to k experiments. In the first experiment, Luna agents in the interaction condition substantially underperform their counterparts in the autarky condition.

Mean of each agent’s best noiseless score using up to the first k experiments per agent for each model, comparing Autarky and Interact.

This lower score in the first experiment appears to reflect more extensive exploration of the input space at the group level. Haiku agents in both conditions and Luna agents in the autarky condition submit very similar inputs for the first experiment, as evidenced by the low average pairwise L2 distance between their inputs (see the first point in each series in the figure below). In sharp contrast, Luna agents in the interaction condition submit quite distinct inputs for their first experiments.

Mean pairwise L2 distance across each group’s pooled tested configurations, using up to the first k experiments per agent and comparing Autarky and Interact.

In the autarky condition, most agents start at a Schelling point, submitting 0.5 for all inputs. Luna agents in the interaction condition break this pattern; they appear to be able to use communication to collectively explore the input space with initial experiments and then exploit promising directions. The figure below tabulates messages sent by agents that have not yet submitted their first experiment. Luna agents reach for collaboration before experimentation; they exchange an average of 8.7 pre-experiment messages, and essentially all agents send pre-experiment messages. Haiku agents, in sharp contrast, exchange an average of just one pre-experiment message with less than 20% of agents participating. Luna agents are more predisposed to coordination at the outset of the task, while Haiku agents experiment first and then collate results (R4; back).

Messages sent before the sender’s first experiment per run, and the share of agents that send one, for each model, comparing Discovery, Presence, and Interact, with 95% confidence intervals.

Conclusion

We continue to find striking differences in the collaboration propensities of different models, as well as the types of strategies these models adopt. As multiagent, and also multiprincipal, deployments become more common, these propensities and model behaviors will become increasingly important in shaping aggregate outcomes.

Appendix

We first plot the best noiseless experiment score obtained across all agents in a run.

Run-wide maximum noiseless experiment score throughout the run for each model, comparing Autarky, Discovery, Presence, and Interact.

We next average the best noiseless experiment score for each agent in a run.

Mean of each agent’s best noiseless score throughout the run for each model, comparing Autarky, Discovery, Presence, and Interact.

As above, we next take all experiments up to an agent's kth, plotting both the maximum score across agents (first figure) and the average of each agent’s best score (second figure).

Run-wide maximum noiseless experiment score using up to the first k experiments per agent for each model, comparing Autarky, Discovery, Presence, and Interact.

Mean of each agent’s best noiseless score using up to the first k experiments per agent for each model, comparing Autarky, Discovery, Presence, and Interact.

We next total the number of messages sent in a run.

Cumulative messages per run throughout the run for each model, comparing Discovery, Presence, and Interact.

We now summarize within-agent (first figure) and group-wide (second figure) exploration of the input space. For both models, coordination reduces the amount of within-agent exploration. At the group level, Luna models use coordination to experiment early and then converge on promising solutions. Haiku agents in the interaction condition end with slightly less group-level exploration than their counterparts in the other conditions, but this difference is smaller.

Mean pairwise L2 distance within each agent’s tested configurations, using up to the first k experiments per agent and comparing Autarky, Discovery, Presence, and Interact.

Mean pairwise L2 distance across each group’s pooled tested configurations, using up to the first k experiments per agent and comparing Autarky, Discovery, Presence, and Interact.

Lastly, we plot the proportion of agents that have submitted a first experiment and the number of messages sent against wall-clock time, truncating the x-axis after all agents have submitted a first experiment. Luna models take noticeably longer to submit a first experiment when interaction is available (relative to the autarky condition).

Share of agents that have run their first experiment and cumulative messages per run, by wall-clock seconds since run start, for each model, comparing Autarky, Discovery, Presence, and Interact.