About the role
Mercor connects elite creative and technical talent with leading AI research labs. The Data Scientist - AI Evaluation role involves designing grading criteria for data science work, evaluating AI-generated and human-created deliverables, documenting assessment decisions, and incorporating reviewer feedback to improve evaluations. ResponsibilitiesDesign precise, task-specific grading criteria for data science deliverables. This includes exploratory data analyses, statistical modeling work, machine learning pipelines, and technical reportsEvaluate AI-generated or human-created work against established criteria to ensure quality and accuracyProvide detailed written justifications for evaluations and scores to maintain transparencyApply consistent, evidence-based judgment for reproducible and defensible assessmentsIncorporate structured feedback from senior reviewers and iterate on submitted work for continuous improvement Qualifications1+ years of professional data science experienceExper
Skills
- Machine Learning