Annotation Data Scientist, Evaluation Integrity (Siri)
External
Apple · Cambridge, UKPrepare for this interview
EliteAI-generated questions, company research, and talking points tailored to this role
About the role
Play a part in the ongoing revolution in human-computer interaction. Siri is evolving - and the way we evaluate it has to evolve with it. Join the Evaluation Integrity team to help build the trusted quality signal behind every Siri release. Within the Siri evaluation organization, the Human Evaluation sub-team is responsible for answering the question: can we trust our evals? We do that by designing human-in-the-loop (HITL) annotation tasks that scrutinize every moving part of an agentic evaluation - the simulated user agent, the conversation it has with Siri, and the automated evaluators that grade the exchange. This role sits at the intersection of data science, human annotation engineering, and evaluation methodology, and is instrumental in turning human judgment into a rigorous, reproducible signal that directly informs pre-ship model and product decisions.
Your Match
How well this role fits your profile.
Company Intel
What employees say
Worked at Apple? Share your experience