Verification & Validation Data Scientist

Summary This role exists to answer one question independently: do the synthetic-audience and automated systems actually perform as claimed? It is a verification-and-validation function drawn from data science, closer to model validation, ML evaluation, and the independent V&V discipline used in regulated and safety-critical software than to market research. The V&V Data Scientist will design and run the evaluation that benchmarks system output against ground-truth data, owns the accuracy metrics and error bounds, and certifies fitness for use, working independently of the engineer who builds the models. The separation of duties is the point: the builder does not grade their own work. Responsibilities Design and run the independent validation methodology: benchmark synthetic-audience and automated output against real panel holdout and other ground-truth data. Own the evaluation metrics, test harness design, and reported accuracy bounds; quantify where each system is and is not reliable. Certify fitness for use, translate results into clear “validated for this use / not validated for that use” determinations. Operate independently of the AI/Synthetic Engineer: verify the models and pipelines AI/Synthetic Engineer builds without having built them. Monitor for model and data drift over time; re-validate as models, prompts, and source data change. Maintain the audit trail and evidence base that supports the function’s credibility with stakeholders and any external auditor. Education & Experience Requirements Education ·         Bachelor’s degree in computer science, mathematics, or engineering or relevant equivalent experience in lieu of degree.  Master’s degree preferred. Experience Seven (7) or more years in data science, model validation, ML evaluation, or an independent V&V / QA function. Demonstrated experience independently validating models or systems against ground-truth data, evaluation design, benchmarking, and error analysis.   Certifications - NA Knowledge, Skills & Abilities Strong applied statistics and metrics design; fluency in accuracy, bias, calibration, and uncertainty quantification. Working understanding of LLMs and synthetic-respondent systems and their failure modes — enough to test them rigorously. Independence and rigor: the temperament to challenge results and withhold sign-off, including from colleagues’ work. Clear communication: able to tell a non-technical executive, with evidence, why a result can or cannot be trusted. Ability to effectively leverage artificial intelligence (AI) tools and technologies to streamline workflows, enhance productivity, and improve overall work quality. Familiarity with survey/behavioral data as a validation target preferred.   Physical Requirements This position operates in a typical office environment (which includes a home office setting) and requires the ability to perform essential job functions with or without reasonable accommodation. Physical requirements may include: Prolonged periods of sitting at a desk and working on a computer. Frequent use of hands and fingers for typing, handling documents, and using office equipment. Occasional standing, walking, bending, and reaching. Ability to lift and carry up to 30 pounds as needed. Clear verbal and written communication skills for effective interaction with colleagues and stakeholders. Work Environment Hybrid Schedule (3 Days In-Office/2 Days Remote) This position follows a hybrid work schedule, with Tuesday through Thursday in office and Monday and Friday remote.  Employees must be available during standard business hours, with core hours beginning between 8:00–9:00 a.m. and concluding between 5:00–6:00 p.m. local time. Travel:  Occasional 0 – 10% #LI The hiring range for this position is $110,000 to $135,000 per year. This range is an estimate, and the actual salary may vary based on the candidate's experience, skills, and qualifications. SHRM offers a competitive and comprehensive total rewards package. The benefits for this position include professional growth and development, health, dental, vision, well-being, health savings, flexible spending, retirement, open leave, and annual discretionary bonus and incentives.