What EuroExec is made of.

The 598 tasks, how experts grade them, and how closely two independent experts agree.

Composition by domain.

Executive domain	Tasks	Share
Product & Tech	188	31.4%
Business	162	27.1%
Marketing	134	22.4%
Finance	114	19.1%

The grading rubric.

Each model response is scored on a five-point scale across five rubrics, and every checklist criterion is marked covered / partial / missed. Scores are shown throughout this site as a percentage of the 5-point maximum (5/5 = 100%, 4/5 = 80%, …). Coverage fraction = (covered + 0.5·partial) / total.

The five rubrics

Domain AccuracyAre the facts, regulations, and figures correct for the market in question?
Strategic ReasoningDoes the answer show sound executive judgment, not just recall?
Actionability & SpecificityCould an operator act on it, with concrete and sequenced next steps?
Executive CommunicationIs it structured and clear enough for a C-suite reader?
Local / Regulatory FidelityDoes it respect the specific European market and legal context?

Checklist depth

5 criteria

46.7% · 279

6 criteria

21.4% · 128

7 criteria

14.4% · 86

8 criteria

9.5% · 57

9 criteria

3.2% · 19

10 criteria

4.8% · 29

Two independent experts agree closely.

All 598 tasks were judged independently by two domain-matched experts, so agreement is measured across the whole dataset. Figures carry bootstrap 95% CIs (1,000 resamples clustered over tasks).

0.7895% CI 0.76–0.79Within-expert consistency (Kendall τ): each expert's stated ranking matches a ranking rebuilt from their own rubric scores.

76%95% CI 74–77%of rubric scores land within ±1 point between the two independent experts.

2.3×vs. chancemore likely than chance that both experts independently pick the same top model.

Scope & method notes.

Scope. Four executive domains: Product & Tech, Business, Marketing and Finance. v1 is English on purpose, to isolate context from language: every task is set in a named European market, so the test is the market and the law. 5 tasks are public samples. The remaining 593 are held out and never published, so scores can't be trained toward.
Judging. Final scores are human: two experts per task. LLMs are used only for automated QA (step 02) and the difficulty gate (step 03), never for final grades. LLM judges correlate with humans better than humans correlate with each other, yet they overrate the strongest models and underrate the weakest, and the correlation statistic hides that bias.
Non-independence. Two correlated judgments per task, so the effective sample is tasks, not judgments. Every leaderboard CI is clustered over tasks to account for this.

See how each task is built on the methodology page, or read five graded tasks end to end in the sample tasks.

See how the six models rank.

Evaluate your model on the full held-out set View the leaderboard