QA Engineer

Skills

About this role

In this role, you'll apply your QA expertise to help train next-generation AI systems through high-quality software quality evaluation and human-in-the-loop feedback. You'll evaluate AI-generated technical outputs against defined rubrics, identify inaccuracies and gaps, and contribute structured feedback that improves model performance. This opportunity is suited for experienced QA professionals with strong manual and automated testing expertise and prior paid experience in human data annotation, RLHF, AI response evaluation, model evaluation, or rubric-based grading

QA Engineer

Responsibilities

• Evaluate and rate AI-generated technical outputs against defined quality criteria and rubrics, identifying inaccurate or incomplete responses.
• Design and assess comprehensive functional, regression, edge-case, negative, and boundary test cases.
• Review bug reports and test documentation for reproducibility, completeness, and appropriate severity assessment.
• Identify, isolate, and document defects with precise reproduction steps using structured tracking mechanisms.
• Provide detailed, actionable written feedback and annotations to support efficient issue resolution.
• Collaborate with project teams to refine evaluation guidelines and continuously improve testing standards and methodologies.

Qualifications

• Proven professional experience as a QA Engineer, SDET, Test Engineer, QA Analyst, or similar software quality assurance role.
• Strong command of QA fundamentals, including test case design, bug tracking, regression testing, and manual and automated testing.
• Hands-on experience with automation frameworks and test management tools such as Selenium, Playwright, Cypress, Appium, Postman, Jira, TestRail, Zephyr, or BrowserStack.
• Prior paid human data experience supporting AI training through annotation, labeling, RLHF, AI response evaluation, model evaluation, or rubric-based grading is required.
• Excellent analytical and problem-solving skills with meticulous attention to detail.
• Strong written English communication skills, B2 level or above.
• No formal degree required; practical testing experience is prioritized.