Red-Teaming Quality Assurance Lead (QAL)

Skills

About this role

In this hourly, remote contractor role, you will work as a Red-Teaming Quality Assurance Lead (QAL) to oversee quality, consistency, and trainer performance across AI red-teaming and safety-evaluation projects. You will review AI-generated safety evaluations, adversarial prompts, risk analyses, and trainer/QA work; evaluate output quality against project guidelines; provide precise written feedback; and ensure that all contributors follow the expected quality standards.

AI Safety, AI Red-Teaming, Risk Assessment

Responsibilities

• Quality monitoring: Spot-check red-teaming items, identify quality issues, provide ongoing feedback through DMs, and escalate recurring or critical issues.
• Safety and red-team review: Evaluate adversarial prompts, model responses, risk classifications, safety analyses, policy explanations, and vulnerability reports for accuracy, realism, and usefulness.
• Trainer and QA communication: Update trainers and QAs on Discord about new item guidelines, project changes, workflow updates, quality expectations, and red-teaming-specific review standards.
• Question handling: Respond to trainer/QA questions clearly and promptly, especially around risk categories, adversarial strategy, policy boundaries, edge cases, and severity.
• Trainer/QA activation management: DM contributors who are inactive or not working, encourage activation, track follow-ups, and flag availability issues when needed.

Qualifications

• Bachelor's, Master's, or professional experience in Computer Science, Cybersecurity, AI Safety, Trust & Safety, Public Policy, Psychology, Linguistics, Law, Security Studies, Risk Analysis, or a related field.
• Strong grasp of the English language to follow project guidelines, communicate with teams, and provide clear written feedback.
• 3+ years of experience in AI safety, red-teaming, cybersecurity, trust and safety, content policy, risk analysis, adversarial testing, model evaluation, content moderation, or related workflows.
• Strong understanding of AI risk categories, adversarial prompting, jailbreak patterns, harmful-content taxonomies, misuse scenarios, policy interpretation, model behavior, and safety evaluation principles.
• Ability to evaluate red-teaming content against detailed rubrics and identify issues such as weak adversarial design, unrealistic scenarios, poor risk categorization, or policy misinterpretation.