Proficiency in Python and SQL for data manipulation, pipeline monitoring, and quality analysis - you should be comfortable writing light scripts to parse formats, run statistical checks, and build lightweight tooling
Working knowledge of LLM internals: RLHF/SFT training loops, how prompt structure affects output distribution, RL environment setup qualities (tool use) for agentic data collection / eval projects.
Hands-on experience with at least one agentic or LLM workflow framework (LangChain, DSPy, AutoGen, direct tool-use via API, or equivalent)
Demonstrated ownership of a data or ML pipeline from scoping through delivery - including quality design, not just throughput tracking
Strong written communication: you'l
Additional Information
About Pareto
Humanity is in a virtuous cycle: human insight improves AI, and better AI expands what people can do. Sustaining it depends on the one input that can't be automated: expert human judgment .
At Pareto, we build the platform that turns that judgment into the data , evals , and RL environments frontier models learn from. We work with leading frontier labs like Anthropic and GDM, and we give skilled people everywhere a way to shape the future of AI and share in what it creates.
This RL environment and human-data infrastructure is already in production. Our job now is to scale it.
About this role
Pareto builds human training data pipelines for frontier AI labs. As a Strategic Projects Lead, you sit at the center of that work - owning the architecture, execution, and continuous improvement of complex data collection and evaluation workflows from first scoping call to final delivery.
This is a technical operations role, not a project management role. You'll be expected to read code, deep dive data, reason about LLM internals, design evaluation frameworks, and - increasingly - deploy and iterate on AI agents to automate the work your pipelines do today. We're actively building toward a model where agentic systems handle quality gates, expert routing, and output review, and SPLs are the people designing and operating those systems.
You'll work directly with AI researchers and technical program managers at our client organizations, own delivery against model performance benchmarks, and lead a team of project managers who handle day-to-day execution tracking.
Pareto-ai · Remote