Legal Domain Expert — AI Training & Evaluation

About this role

You will work directly with a leading AI lab to improve how frontier AI systems reason about real-world legal work. | The role focuses on reviewing legal tasks and model outputs, writing instruction specifications, developing golden solutions, and creating benchmark datasets. | You will translate senior-level legal judgment into clear, measurable evaluation criteria for AI training and research. | This is a full-time, hybrid W-2 role in the Bay Area, California.

Legal AI Evaluation Expert

Responsibilities

• Review legal knowledge-work tasks and AI outputs for weak reasoning, missing behaviors, factual issues, and professional-quality gaps.
• Write instruction specifications, golden solutions, and realistic legal tasks based on actual legal practice.
• Design challenging legal benchmarks and evaluation datasets in collaboration with AI researchers.
• Calibrate evaluation standards and translate complex legal judgment into precise, teachable scoring criteria.

Qualifications

• Juris Doctor (JD) from an accredited law school, active admission to at least one U.S. state bar, and good standing required.
• Minimum 5+ years of substantive post-qualification legal practice at a reputable law firm, corporate legal department, regulator, or court.
• Senior-level specialization in areas such as corporate law, litigation, regulatory/compliance, intellectual property, employment, or tax.
• Hands-on experience using LLMs, excellent written communication, and ability to distinguish rigorous legal reasoning from plausible but incorrect outputs.