Staff Testing Developer

Responsibilities

• Craft & Delivery
• Evaluate the platform, produce a thorough test coverage plan, and design a scalable testing architecture for the Python/AWS stack (Lambda, DynamoDB single-table design, SQS, S3, EventBridge, CDK) across unit, integration, E2E, agentic eval, and synthetic learner layers
• Bring an AI-forward mindset to your daily work, using tools like Claude Code and Cursor to ship higher-quality work at pace
• Collaboration & Communication
• Partner with engineering and product leadership to align test strategy with delivery goals and platform architecture decisions
• Translate test coverage status, quality risks, and recommended investments into terms technical and non-technical stakeholders can act on
• Lead test planning sessions and release readiness assessments, driving clear go/no-go signals across the team
• Leadership & Influence
• Establish the testing standards, frameworks, and patterns the broader engineering team adopts and extends, mentoring junior and mid-level engineers on testing practices to raise quality ownership across the team rather than centralizing it on yourself
• Take ownership of quality end-to-end, including the unglamorous work of stabilizing flaky suites and paying down test debt
• Evaluate and introduce emerging tools and methodologies, continuously improving testing quality and velocity without chasing novelty for its own sake

Requirements

• 5+ years of professional software engineering experience with a strong focus on testing-unit, integration, E2E, and/or AI/ML system testing
• Strong Python programming skills; this role writes test code, not just test plans
• Hands-on experience with AWS services including Lambda, DynamoDB, SQS, S3, and EventBridge; CDK experience a strong plus
• Deep expertise with PyTest and Python-native testing frameworks, with a track record of designing and scaling test automation infrastructure
• Experience writing and maintaining E2E and integration tests for event-driven, serverless, or microservices architectures
• Familiarity with DynamoDB single-table design and the specific challenges of testing against it
• Experience building or validating agentic or LLM-based systems; comfort with evals, output consistency testing, and hallucination/accuracy validation
• Str

Benefits

Additional Information

Company Overview Robots & Pencils is an applied AI engineering firm building the next frontier of business architecture. We design and ship AI co-workers that integrate into enterprise operations and deliver measurable results for our clients. We're all in on AWS, combining deep UX capability with senior engineering talent to get AI into production fast and keep it there. We've earned the trust of leaders across Consumer Products and Retail, Education, Energy, Financial Services, Healthcare, and Manufacturing and more, and earned a reputation as the nimble alternative to traditional global systems integrators. Founded in 2009, with delivery centers in Canada, the United States, Eastern Europe, and Latin America, we are smaller, faster, and more senior by design. Our teams average 15+ years of experience. We move fast, sweat the details, and build things that actually ship. Position Overview We're looking for a Staff Testing Developer to join our team and own quality across a cloud-native AI/ML platform built on AWS. This is not a QA position. It is a hands-on engineering role for someone who writes test code across a production Python and AWS stack. In this role, you will evaluate the platform, create a comprehensive test coverage plan, and drive best practices across the team. You'll propose and implement improvements to our testing infrastructure, modify production code to improve testability when necessary, and work with the broader engineering team to establish patterns that other developers can adopt and carry forward. You will be the authoritative voice on testing automation and test engineering on this engagement, working with minimal supervision. Why This Role Matters At Robots & Pencils, we design AI systems for a human world. Our name says it all. Robots and pencils means engineering paired with creativity, because every agent we ship has to work for real people in real workflows. That balance is baked into how we operate. This platform delivers AI-powered learning and automation to real users-and its reliability depends on the quality infrastructure you build. You won't be auditing a test suite. You'll be building one from the ground up, shaping what "done" means across every layer of the stack: Lambdas, DynamoDB, SQS, event-driven flows, and agentic AI pipelines. When this platform works, people learn better and move faster. That's what's on the line.

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at robotsandpencils? Share your experience