Staff Software Engineer-AI

About the role

DigitalOcean's Agentic AI organization provides a powerful inference cloud, Managed Agents, and robust Feedback systems that enable customers to run AI inference confidently at scale. We are looking for a Staff Software Engineer to serve as a technical leader within our Feedback Systems team, driving the architecture for the massive-scale infrastructure that simulates, tests, and evaluates AI agents. As an IC5 Staff Engineer, you will define the architectural vision for systems that simulate multi-agent, multi-turn deployments complete with tool integration. This enables customers to run and analyze "what-if" scenarios to evaluate alternative configurations for their AI agents. This is a high-impact leadership role where you will solve advanced problems at the intersection of LLM orchestration, synthetic data generation, and behavioral simulation-including defining realistic user personas from historical telemetry and structuring automated evaluation objectives and constraints. You will set the technical standard for the team and guide the engineering strategy across the Agentic AI organization.

Responsibilities

• Persona & Scenario Generation: Designing ML pipelines that analyze historical user conversations to automatically extract, define, and synthesize realistic user personas and multi-turn simulation goals, mirroring real-world customer behavior.
• "What-If" Evaluation Frameworks: Building the core methodology and scoring infrastructure that allows customers to run alternative configuration scenarios, benchmark agent behavior, and safely evaluate non-deterministic agent outputs against defined success criteria.
• Architectural Leadership: Leading the end-to-end design and architecture of high-throughput, stateful workflow orchestration systems capable of managing complex, multi-turn AI agent simulations at massive scale.
• System Design & Integration: Defining robust, scalable API contracts and system boundaries bridging upstream telemetry data, asynchronous simulation engines, and secure remote execution environments.
• Technical Strategy: Driving the technical roadmap for the Feedback Systems team, balancing long-term scalability and resilience with iterative product delivery.
• Complex Problem Solving: Designing elegant solutions for hard distributed systems challenges, including rate limiting, backpressure, state management, and reliable execution of non-deterministic workflows.
• Mentorship & Elevation: Mentoring senior engineers, leading cross-organizational architectural reviews, and establishing engineering best practices for code quality, testing, and system observability.
• AI/ML Infrastructure Integration: Applying your practical experience with AI/ML platforms to design and implement the backend infrastructure that powers our evaluation engines, actively managing the complexities of integrating with LLMs, prompt routing, and non-deterministic agentic workflows.
• What You'll Add to DigitalOcean:
• Agentic Expertise: 5+ years of software engineering experience with deep proficiency in modern AI/ML frameworks, LLM orchestration (e.g., LangChain, AutoGen, CrewAI, or custom multi-agent frameworks), and production-grade Python and Go.
• Behavioral Modeling & Persona Synthesis: Background in processing natural language data (e.g., historical user chat logs, support tickets) to algorithmically extract user intent, synthesize realistic personas, and generate deterministic goals for simulation.
• Evaluation & "What-If" Benchmarking: Solid experience building evaluation frameworks for non-deterministic AI systems, including establishing metrics, guardrails, scoring rubrics, and regression testing methodologies for LLM configurations.
• Data Fluency & Orchestration: Strong understanding of managing complex state in asynchronous architectures, streaming LLM tokens, handling rate limits, and manipulating h

Benefits

Additional Information

Dive in and do the best work of your career at DigitalOcean. Journey alongside a strong community of top talent who are relentless in their drive to build the simplest scalable cloud. If you have a growth mindset, naturally like to think big and bold, and are energized by the fast-paced environment of a true industry disruptor, you'll find your place here. We value winning together-while learning, having fun, and making a profound difference for the dreamers and builders in the world.

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at digitalocean98? Share your experience