AI Outbound benchmark 2026

Twain Bench measures eight different qualities of a cold email. Personalization, voice, factual support, and concision do different jobs, and their importance depends on your outreach.

We report each score separately. Combining them would require us to choose weights for your team. Use the criteria that matter to you to shortlist models, then read what they write for your own prospects.

No. The Reply score averages AI judges’ assessments of whether the recipient would answer. Each judge plays the recipient and reads the email with only what the recipient knows about themselves.

No real replies stand behind the score. A score of 31 out of 100 does not predict a 31% reply rate.

A higher Hallucination score means more of the email’s claims are supported by the research brief. Judges check specific claims about the sender and recipient for invented or misattributed facts. Assumed premises are tracked separately and do not lower this score.

The score is not the percentage of emails with no unsupported claims. An email can contain many supported facts and one unsupported claim, so a high average does not mean every email is fully supported.

Yes. Every model receives the same research briefs and prompt, with a fixed benchmark date and medium reasoning effort. We add no model-specific instructions about writing style.

Models cannot browse the web or open links. The prompt asks them to use only the supplied company and prospect information and return a cold email that is ready to send.

The cost per email is the list price of the tokens each model used to generate its emails, without provider caching discounts. Caching savings depend on the provider and the order of calls, so we exclude them from the comparison.

These are benchmark generation costs, rather than the price of sending an email through Twain.

The benchmark measures how models write under one shared prompt. Your writing instructions, research, recipients, and reasoning settings can change the emails and the ranking.

Use Twain Bench to choose models worth testing. Then compare emails written from your own leads and company research before putting them into an outreach workflow.

The company and prospect research, and the generated benchmark emails, remain private. We publish aggregate results and explain the prompt, scoring rubric, and limitations in the research article.

The example emails in the article are illustrative. They are not emails taken from the benchmark dataset.

Three AI judges from three providers score every email using the same eight criteria. Each model’s scores are averaged across its 1,000 emails and all three judges.

The judges broadly agree on the order of the models, though they differ in strictness. Their agreement is lower on Flow and Reply, where models also score close together. AI judges are not human readers, so test emails for your own prospects before choosing a model.