How EuroExec is built.
Quality comes from separating who writes a task from who judges it, across seven steps.
-
Expert authoring
44 vetted domain experts (CV screen plus a one-on-one interview; median 7 years of experience; based in 10 European countries) each write a hard, self-contained executive scenario and a 5 to 10 point grading checklist, drawn from problems they have worked first-hand. Copy-paste is disabled in the authoring tool, so every prompt is typed by the expert who wrote it.
-
Automated QA
An LLM screens each task on six criteria. Blocking: hint leakage, checklist relevance, prohibited patterns. Warning: professional language, a clear role, concrete specificity. Anything that fails goes back to the author with structured notes.
-
Difficulty gate
Claude Haiku 4.5 attempts each prompt over 5 passes, and Gemini Flash, from a different model family, measures how much of the checklist it covers. A task passes only when first-pass coverage stays at or below 60%. Anything easier goes back to the author for a rewrite (about 30 rewrites per item in the first release). Because tasks were filtered against Claude Haiku 4.5, its score here is biased downward. We report it anyway.
-
Lead review
With the automated gates passed, a domain lead reviews the prompt and its checklist for realism, clarity, and a fair grading key, then approves the task to proceed. Authoring is deliberately kept separate from judging.
-
Blind generation
Six frontier models each answer the prompt. Responses are stripped of model identity and shuffled into blind slots, so evaluators never know which lab produced which answer.
-
Dual expert evaluation
Two independent, domain-matched experts each grade every response: five rubrics scored 1 to 5 (Domain Accuracy, Strategic Reasoning, Actionability & Specificity, Executive Communication, Local / Regulatory Fidelity), checklist hit / partial / miss, a forced ranking of all answers, and a written rationale. Every task is judged twice, independently.
-
Adjudication & gold
A senior data reviewer calibrates scoring, adjudicates disagreements between the two experts, and signs off the final, versioned gold set that ships.