Five graded tasks, end to end.

Five tasks across the four domains, each shown exactly as an expert receives it: the prompt, the checklist, six blind answers, and both experts' scores and reasoning. Prompts and rationales are verbatim and unedited, as the experts wrote them. Answers are labeled A to F until you reveal the models.

Choose a sample task

52%

mean rubric score across all six answers

60%

the strongest single answer, both experts averaged

22%

of the graded checklist that best answer covers

Executive scenario (prompt)

Business

I am CEO of a German grocery chain with 12 billion in annual revenue, an EBITDA margin of 6%, the company is leveraged at 3.5 times their EBITDA and cannot exceed 4 times, and operates 2,000 stores across Germany, Austria, and the Czech Republic. The board has set a target EBITDA margin of 7% while growing revenue 20% over the next five years without becoming over-leveraged.

You are considering expanding into Italy which has a highly fragmented grocery store environment. the top 5 players control 50% of the environment and discount penetration is growing 6-7% annually and average gross margin for discount stores is 2 percentage points higher in Germany because of more favorable conditions and less intense price competition. However labor costs are 20% higher in Northern Italy than in Eastern Germany, Austria, and the Czech Republic.

our strategy team has identified three options to reach the board's goals in the next 5 years:

1-open 300 stores in Northern Italy. this would cost 4 million euros for each store and requires building a new Italian distribution network at a cost of 250 million euros

2-a joint venture expansion into Italy with an existing mid-sized regional supermarket (owns 150 stores and 2.1 billion euros in revenue). the plan would convert 100 stores to discount stores and would cost 1 million euros for each store. the joint venture would pay profits 50% once EBITDA exceeds 100 million

3-Germany first approach with small Italian test. focus 80% of capex on automation, online ordering infrastructure, and format optimization in existing German stores. run a limited Italian pilot with 40 leased stores over 3 years

write a one page memo for the supervisory board and recommend one of the options.

Graded checklist

States a clear recommendation (or a hybrid of two options) and explains why it rejects the other options with 1-2 reasons ● critical
Quantifies Italy's 5 year contribution to revenue and EBITDA (for the recommended option) and clearly states how it moves the company towards the board's target of 20% revenue growth and 7% EBITDA margin
Balances economic and social factors, specifically reputation and political risk from job reductions
Identifies the top 3-5 risks (union backlash, tech underperformance, political pressure, cost overruns, etc.) and gives a concrete mitigation action for each option)
Quantifies capex and EBITDA at the end of year 5 for chosen option (or hybrid option)
Estimates final capex for chosen option at the end of each 5 years and is broken down into major components (Italian store openings, Italian distribution network, German automation, etc. for whichever option is recommended)
Provides a comparison of labor costs impacts between entering Italy compared to a non-entry scenario, including change in total group labor cost
Compares revenue, margin, capex, labor costs, EBITDA of chosen option to non-entry alternative hypothetical

Model responses.

Six models answered this prompt blind, labeled A to F. Read them, then flip the switch to reveal which lab wrote each.

AResponse

3,918 chars

RE: Strategic Options for Achieving 2029 Targets — Recommendation

BResponse

3,987 chars

Subject: Strategic Expansion Recommendation to Achieve Board Targets

CResponse

3,638 chars

After thorough analysis of our strategic options, I strongly recommend Option 3: The "Germany First" approach with a limited Italian pilot. This is the only path that allows us…

DResponse

4,531 chars

Subject: Recommendation on five-year growth and EBITDA margin plan

EResponse

3,627 chars

Re: Strategic Path to 7% EBITDA Margin and 20% Revenue Growth — Recommendation: Option 3 (Germany-First with Italian Pilot)

FResponse

3,735 chars

To achieve the Board’s mandate of growing revenue by 20% (to €14.4B) and expanding our EBITDA margin to 7% without breaching our 4.0x leverage covenant, I strongly recommend Opt…

How the two experts ranked it.

Both experts scored every answer on the five-point rubric and produced a full preference order, blind and independently. On a hard task the two orders can diverge sharply. Double judging exists to surface that, and disagreements like this go to a senior reviewer for adjudication (step 07). Experts stay pseudonymized.

Expert 1

A04 · 62 min

1EResponse E84%
2DResponse D80%
3CResponse C68%
4FResponse F64%
5BResponse B60%
6AResponse A60%

Expert 2

A43 · 16 min

1CResponse C44%
2AResponse A32%
3DResponse D32%
4EResponse E36%
5FResponse F32%
6BResponse B36%

Expert 1’s reasoning · A04

E is first because its the only one that shows with real numbers, option 3 hits both targets, building the revenue bridge towards the 2.4B€ growth gap and is honest that growth is the weak spot here, in my opinion it has the sharpest judgement. D is close second, strong reasoning and it has best risk guardrails, but it names the 288M€ gap without bridging the revenue, making it one step short from E. C third, clean with correct math and first to say about the lower margins in Italy, but only assumes the 20% growth happens instead providing it. F fourth, correct numbers and sharp margin trap point but flatly claims +20% with no support and treats job cuts as free, having lower judgement than C. B is fifth, contradicting itself with numbers, the margin math overshoots the 7% target and overall it provides unfinished draft with blank placeholders. Lastly, A, it has almost no numbers on a task that demands them. Only one caught the German woks-council and filing traps so local law knowledge scored law on all responses.

Per-rubric scores

Answer	Domain Accuracy	Strategic Reasoning	Actionability & Specificity	Executive Communication	Local / Regulatory Fidelity	Cov.	Rank
Resp A	60%	60%	60%	80%	40%	13%	#6
Resp B	60%	60%	80%	60%	40%	25%	#5
Resp C	80%	80%	60%	80%	40%	13%	#3
Resp D	80%	100%	80%	100%	40%	25%	#2
Resp E	100%	100%	80%	100%	40%	25%	#1
Resp F	80%	60%	60%	80%	40%	13%	#4

Expert 2’s reasoning · A43

Although all the answers hit checklist 1 clearly and check 2 partially by recognizing EBITDA margin improvement was possible with option 3 but 20% growth was a bit stretched, however, none focused on failure scenarios of the option such as failed automation that increases costs through capex and opex through complexity, how it affects employees, where it should be deployed, in the warehouse, delivery etc. Also no response offered a detailed financial analysis of the 2 scenario vs status quo to show any commercial gain or loss over a 5 year timeframe. None mentioned the opportunity cost of Italy, and how to probably focus on Southern Italy instead of Northern Italy where discounts are likely growing faster with a lower cost base similar or even lower compared to Eastern Germany probably. Lastly social and economic factors weren´t discussed including political pushback on automation if seen to lead to job loss as well as regulation on Italian market entry

Per-rubric scores

Answer	Domain Accuracy	Strategic Reasoning	Actionability & Specificity	Executive Communication	Local / Regulatory Fidelity	Cov.	Rank
Resp A	40%	40%	20%	40%	20%	19%	#2
Resp B	40%	40%	40%	40%	20%	13%	#6
Resp C	60%	60%	20%	60%	20%	19%	#1
Resp D	40%	40%	20%	40%	20%	19%	#3
Resp E	60%	40%	20%	40%	20%	19%	#4
Resp F	40%	40%	20%	40%	20%	19%	#5

See how tasks like this are built on the methodology page, or the field-wide scores on the leaderboard.

Want to see how your own model performs on tasks like this?

Evaluate your model on the full held-out set