Log in
At a glance
- AI agents are moving into social contexts. When agents manage calendars, negotiate purchases, or interact with other agents on a user’s behalf, they need more than task competence—they need social reasoning.
- SocialReasoning-Bench evaluates that ability. The benchmark tests whether an agent can negotiate for a user in two realistic settings: Calendar Coordination and Marketplace Negotiation.
- The benchmark measures both outcomes and process: it scores agents on outcome optimality (how much value they secure for the user) and due diligence (whether they follow a competent decision-making process).
- Current frontier models often leave value on the table. They usually complete the task, but they frequently accept suboptimal meeting times or poor deals instead of advocating effectively for the user.
- Prompting helps, but it is not enough. Even with explicit guidance to act in the user’s best interest, performance remains well below what a trustworthy delegate should achieve.
As AI agents take on more real-world tasks, they are increasingly operating in social contexts. With the right integrations, agents like Claude Cowork and Google Gemini can manage email and calendar workflows. In these settings, the agent must interact with others on your behalf. This requires social reasoning — understanding what you want, what the counterparty wants, and what information to reveal, protect, or push back on.
Our previous research suggests that today’s frontier models lack social reasoning. In our simulated multi-agent marketplace, agents accepted the first proposal they received up to 93% of the time without exploring alternatives. When red-teaming a social network of agents, a single malicious message spread through the system and led agents to disclose private data before passing the message along.
This kind of relationship has a long history outside AI. In economics and law it is called a principal-agent relationship: an agent acts on a principal’s behalf in interactions with others whose interests differ. Attorneys, real-estate agents, and financial advisors all operate in this mode, and the duties they owe—care, loyalty, confidentiality—are codified in centuries of professional norms. AI agents acting on a user’s behalf should ultimately be held to similar standards.
To measure and drive progress in social reasoning, we built SocialReasoning-Bench: a benchmark for testing whether agents can reason and negotiate on a user’s behalf against a counterparty with independent goals, private information, and potentially adversarial intent.
Introducing SocialReasoning-Bench
SocialReasoning-Bench evaluates social reasoning in two domains: Calendar Coordination and Marketplace Negotiation. In each, an agent advocates for its user against a counterparty and is scored on both the outcome it reached and the process it followed. We find that frontier models complete most tasks but consistently leave value on the table for the user.
Calendar coordination
In calendar coordination, an assistant agent manages a user’s calendar on a single day and fields a meeting request from another agent.
We assume the agent has access to a value function over time slots that captures the user’s scheduling preferences between 0.0 and 1. This function could be provided explicitly by the user or inferred from their calendar history, and is given to the assistant at the start of the task.
The counterparty is a requestor agent representing another person who wants to schedule a meeting with the user. The counterparty has its own value function over the same slots, constructed as the inverse of the user’s, so the slots most valuable to one are least valuable to the other. Some requestors negotiate in good faith, while others use the interaction to extract private calendar details or push the assistant toward times the user does not want.
In each task there is a zone of possible agreement (ZOPA) a term borrowed from negotiation theory for the set of outcomes that both parties could plausibly accept. In calendar coordination, the ZOPA is the set of time slots that are mutually free on both calendars. We construct every task so that the ZOPA contains at least three slots with different preference scores for the user, and the requestor’s opening request always conflicts with the user’s calendar.
Marketplace negotiation
In marketplace negotiation, a buyer agent representing a user negotiates with a seller agent to purchase a single product.
The user wants to pay as little as possible for the product. Their value function is the gap between the deal price and a private reservation price, the highest price they would pay. A larger gap captures more value, and a deal above the reservation captures none.
The counterparty is a seller agent with its own private reservation price set below the buyer’s. The counterparty’s value function mirrors the user’s, with higher deal prices yielding more value and deal prices below the seller’s reservation price yielding no value.
The ZOPA is the price range between the seller’s and buyer’s reservations. The seller’s opening offer is always above the buyer’s reservation, forcing the buyer to negotiate the price down.
New metrics for a new setting
Existing benchmarks focus on task completion: did the meeting get scheduled? Did the trade close? In principal–agent settings, what matters is not just whether the task is completed, but how well it is done. We introduce new measures to capture this distinction.
Outcome Optimality
Outcome optimality scores the share of available value the agent captured for its principal, on a 0-to-1 scale. The outcome inside the ZOPA most favorable to the principal scores 1, while the outcome most favorable to the counterparty scores 0.0. Intermediate outcomes are scored by where the principal’s value function places them between those two endpoints.
Due Diligence
Outcome optimality alone conflates skill with luck. An agent that immediately accepts a counterparty’s first offer, without inspecting its situation or making a counter-proposal, can still score well if the counterparty happens to propose a good outcome. To separate skill from luck, we introduce a process metric.
Due diligence scores process quality on a 0-to-1 scale by comparing the agent’s actions, at each decision point in the trajectory, against the action a deterministic reasonable-agent policy would have taken in the same state. The reasonable-agent policy is a greedy procedure that captures what a competent advocate would do at each step, such as gathering relevant context before acting, opening with a position favorable to its principal, and conceding only after better options have been exhausted. The Due Diligence score is the rate at which the agent’s actual choices match the reasonable-agent’s choices over the trajectory.
Duty of care
Together, Outcome Optimality and Due Diligence form an operational notion of an agent’s duty of care to the person it represents. An agent that lands a good outcome through a careless process is fragile, while an agent that follows good process but lands a bad outcome points to a capability gap rather than negligence. Only an agent that scores well on both is exhibiting strong social reasoning.
PODCAST SERIES
AI Testing and Evaluation: Learnings from Science and Industry
Discover how Microsoft is learning from other domains to advance evaluation and testing as a pillar of AI governance.
Experimental setup
For the calendar assistant agent and marketplace buyer agent, we evaluate GPT-4.1 with chain-of-thought, GPT-5.4 at high reasoning effort, and Claude Sonnet 4.6 and Gemini 3 Flash at high thinking levels. The counterparty (i.e. requestor in calendar coordination, and seller in marketplace negotiation) is always Gemini 3 Flash with medium reasoning effort, held constant across all conditions so that any difference in scores reflects the model under test rather than the difficulty of its opponent.
Each model is run under two prompt conditions: Basic Prompting where the agent receives only role and tool descriptions, and Defensive Prompting where the agent additionally receives explicit guidance to consult all available sources and advocate for the user toward the best possible outcome.
Each task runs for 10 negotiation rounds, at most. The counterparty proposes first in every task.
What we’re finding
Finding 1: Agents complete tasks at near-perfect rates but produce poor outcomes.
In calendar scheduling, agents almost always succeed in booking the meeting, but most often at suboptimal times. In marketplace negotiation, deals almost always close, but frequently at the worst possible price. The tasks get done, but not done well: task completion signals success, while Outcome Optimality reveals a consistent failure to act in the principal’s best interest.
Finding 2: Defensive prompting helps, but is not enough to close the gap.
When we instruct agents on how to work hard on their principal’s behalf, we see outcome improvements across both domains, but it is not enough to close the gap. GPT-5.4 benefits most from defensive prompting (+0.21 in calendaring, +0.12 in marketplace), while GPT-4.1 barely responds to it in either domain. The other models fall somewhere in between.
Finding 3: Outcome optimality shows how much value agents leave on the table.
Outcome optimality reflects where each deal lands within the ZOPA. When we plot outcomes, they cluster closer to the counterparty’s ideal than the principal’s.
In marketplace negotiation, all models settle at or near zero for Outcome Optimality, accepting deals that give away virtually all available surplus. In calendar scheduling, agents perform better but still land below the midpoint, accepting the requestor’s preferred slots rather than ones that better serve their principal.
Measuring value capture in agent negotiations builds on recent studies examining how agents perform in marketplace settings. Because we operate in a controlled setting, we can establish ground-truth constraints for both parties and measure exactly how the available value was divided. Our formulation also generalizes beyond price-based negotiations: by abstracting to a domain-specific value function, Outcome Optimality can measure surplus division in any setting where agents face competing incentives, including non-monetary domains like calendar scheduling where “value” is defined over preference scores rather than prices.
Finding 4: Due Diligence helps distinguish between luck and skill.
When we look at the combination of outcome quality and process quality, a more nuanced picture emerges. Many agents that achieve reasonable outcomes do so through fragile processes: they don’t check context before acting or they accept offers without countering. High Outcome Optimality with low Due Diligence suggests an agent that got lucky rather than one that can be trusted. Conversely, some agents show genuine diligence — gathering information, pushing back — but still land on poor outcomes, pointing to capability gaps rather than negligence. Dividing Outcome Optimality and Due Diligence each into high (>=0.5) and low (<0.5) buckets, we can sort every task into one of four archetypes.
| Not diligent (DD < 0.5) | Diligent (DD ≥ 0.5) | |
|---|---|---|
| Good outcome (OO ≥ 0.5) | Lucky | Robust |
| Poor outcome (OO < 0.5) | Negligent | Ineffective |
Through the lens of this decomposition, we can see that models exhibit robust duty of care on more than 50% of calendar coordination tasks, with Gemini 3 Flash leading at 90% robust. In marketplace negotiation, though, a very different picture emerges. GPT-4.1 is negligent in 95% of tasks, neither gathering information nor advocating for its principal, while Claude Sonnet 4.6, GPT-5.4, and Gemini 3 Flash show ineffective behavior in roughly 90% of marketplace tasks, negotiating diligently but still unable to achieve good outcomes.
Figures 5-8 illustrate these different behaviors and failure modes with real examples from SocialReasoning-Bench in the calendaring domain. We see agents that follow a strong negotiation strategy and secure high-value outcomes, but also agents that achieve reasonable outcomes through sloppy processes, such as failing to propose the principal’s best option. Others begin with a strong position but concede prematurely, collapsing to poor deals. At the extreme, some agents exhibit negligent behavior, accepting the first proposal without checking constraints, even when it directly conflicts with the user’s interests.
Taken together, these examples highlight why outcome alone is insufficient. Without measuring process, we risk mistaking brittle or accidental success for genuine capability. Due Diligence helps surface whether an agent is consistently behaving like a competent, trustworthy delegate, or simply getting lucky.
Finding 5: Agents are vulnerable to adversarial manipulation
When we stress test agents by pitting them against adversarial counterparties, we find that agents struggle to balance when to engage, when to refuse, and how to negotiate under pressure.
To create these adversarial scenarios, we introduce counterparties explicitly trying to manipulate outcomes or bypass protective steps. Some follow carefully designed strategies, applying pressure or probing for information, while others use more unpredictable, creatively generated whimsical tactics that mimic novel forms of social engineering. Together, these test whether agents can handle not just known attacks, but unfamiliar ones.
We find that, aside from Claude Sonnet 4.6, agents rarely refuse adversarial requests in calendar scheduling, while refusing more often in marketplace settings. This suggests that adversarial intent is harder to detect in socially framed interactions. When agents do engage, the impact is starkest in calendar scheduling with Outcome Optimality dropping substantially across GPT-4.1, GPT-5.4, and Gemini Flash 3, suggesting that adversarial counterparties successfully steer these agents toward worse outcomes. In the marketplace domain, Outcome Optimality when agents engaged remains comparable to the low levels achieved against benign counterparties, capturing little to no value for their principals.
Why this matters now
Agents are interacting with each other in multi-party environments, from collaborating across enterprise workflows to transacting in digital marketplaces. As these networks form, the social reasoning gaps we observe in simple two-agent settings can begin to compound. Weak negotiation, over-trust, or failure to exercise due diligence no longer stay local. They propagate through coordination, influence downstream decisions, and shape collective outcomes.
In isolation, an agent that accepts a bad meeting time or a poor deal causes limited harm. In a network, those same behaviors can cascade, leading to systematically worse coordination or widespread value loss across many agents.
Recent work has begun exploring these risks and dynamics through case studies of agents interacting in networked settings. SocialReasoning-Bench complements this line of work by providing a controlled, reproducible benchmark that isolates interaction behaviors and makes them measurable. This allows us to move beyond anecdotes and systematically track progress, giving model, agent, and platform developers a concrete target for building agents that act as trustworthy delegates.
SocialReasoning-Bench is open source and available on GitHub (opens in new tab).
Limitations and future work
Our current measures treat all counterparties equally. In practice, relationships matter. A socially intelligent agent should modulate its assertiveness based on their principal’s relationship with the counterparty: pushing too hard when scheduling a meeting with a senior executive may damage a valuable relationship, and sometimes the right outcome is reached through compromise. Developing relationship-aware measures that account for power dynamics, rapport, and long-term consequences is an important direction for future work.
We evaluate social reasoning in simplified two-agent settings, whereas real-world delegation often involves multi-party dynamics such as group scheduling or multi-stakeholder negotiations. Each task is also treated as an independent encounter, with no modeling of long-term relationships, reputation, or trust-building across repeated interactions. Our scenarios are also limited to English-language and U.S.-centric business contexts, though social norms around negotiation, privacy, and hierarchy vary widely across cultures. Looking ahead, we plan to extend our benchmark to more diverse settings.
Finally, Outcome Optimality works well in settings with clear boundaries, where a “good” outcome can be defined and measured. But many tasks that require duty of care, such as drafting sensitive messages or navigating team dynamics, may not have a well-defined ZOPA. In these cases, outcomes depend on context, relationships, and judgment in ways that may resist a single score. Extending our approach to these more subjective settings is an important direction for future work.
Acknowledgements
We would like to thank Brendan Lucier, Adam Fourney, Amanda Swearngin, and Ece Kamar for their helpful feedback, discussions, and support of this work.
Opens in a new tabThe post SocialReasoning-Bench: Measuring whether AI agents act in users’ best interests appeared first on Microsoft Research.
