Research Evaluation Specialist (PhD / Researcher / Professor)

Skills

About this role

In this role, you'll apply your advanced research expertise to help train next-generation AI systems by developing rigorous, high-difficulty evaluation material. You'll create original questions and defensible answers that test advanced reasoning, methodological nuance, and cross-source synthesis, while sourcing and verifying information through primary and authoritative references. This opportunity is suited for PhD-level researchers, professors, and subject-matter experts with strong scholarly research experience, exceptional analytical thinking, and the ability to independently produce precise, challenging, and well-supported evaluation content.

Research Evaluation Specialist (PhD / Researcher / Professor)

Responsibilities

• Create original, high-difficulty question-and-answer pairs that rigorously assess advanced AI model capabilities.
• Source, verify, and document answers using primary sources and authoritative references with clear citations and reasoning.
• Design questions requiring advanced reasoning, methodological nuance, or synthesis across multiple sources.
• Test questions against AI systems, identify insufficiently challenging items, and iteratively increase complexity while maintaining accuracy.
• Write questions and answers with high precision and clarity while eliminating ambiguity and ensuring defensible responses.
• Integrate reviewer feedback and consistently uphold project guidelines and quality standards.

Qualifications

• PhD completed, active candidacy, or equivalent research experience as a specialist, researcher, or professor in the relevant domain.
• Demonstrated track record of scholarly research, deep subject-matter expertise, and familiarity with primary literature.
• Exceptional analytical thinking, attention to detail, and written English communication skills.
• Experience sourcing and triangulating information from authoritative references.
• Proven ability to work independently and reliably in remote engagements.
• Strong ability to compose challenging, original, and methodologically sound questions.
• Experience in AI training or evaluation is advantageous but not required.