AI Evaluation Analyst

About this role

In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters. You will play a crucial role in producing evaluation and training data that directly influence how powerful AI models understand and interact. This is an exciting opportunity to operate at the forefront of AI, with a focus on written and verbal clarity, multi-turn conversation design, and deep analysis of model behavior.

AI Evaluation & Data Annotation Specialist

Responsibilities

• Author detailed, task-based multi-turn conversations and rubrics aligned with project specifications.
• Test and refine conversation drafts against frontier large language models, iterating to meet quality and difficulty requirements.
• Deliver comprehensive evaluation assets including transcripts, target behaviors, binary rubrics, and supporting evidence

Qualifications

• Native-level written English with exceptional clarity, structure, and attention to detail.
• Prior experience in data annotation, RLHF, SFT, evaluation, or prompt engineering for AI systems.
• Working knowledge of frontier LLM behaviors and common model failure patterns.