Lead Quality Engineer

About the role

Make 'fast' stay 'fast' without turning into 'fragile' - the first Quality Engineer in a newly established Agentic AI team Build our Automation from the ground-up! Join the Public Claims Engineering team Sydney-based, in-office three days a week About CORTO We are CORTO, a cutting-edge software company dedicated to revolutionising the legal industry. Our mission is to empower legal practitioners with AI-driven solutions that streamline their workflow, boost productivity, and provide more efficient client service. Our team of AI experts and engineers collaborate to develop intelligent software tailored to the unique needs of lawyers, paralegals, and legal assistants. Our innovative AI solutions automate routine tasks, simplify document management, and enhance decision-making, allowing legal professionals to focus on what they do best-providing expert legal counsel. We're rapidly scaling from 80 to 150+ employees, with a highly technical workforce where around 90% of the team are developers and engineers. Working alongside our Sydney-based team of passionate high achievers, you'll join a fast-growing technology business where things rarely stay the same for long - and if you're smart, caring, and ambitious, you'll be in great company. What you'll do You'll be the first Quality Engineer in a newly formed Public Claims team, reporting to Dale Hurley, Head of Agentic AI Engineering, working on the delivery of agentic AI products directed by the founder of ATI Group, global LegalTech leader Christian Beck. You'll design and build the quality foundations that let a small team of AI-augmented engineers keep moving at pace while shipping products that inhouse lawyers trust with their work. Christian is the visionary leader who founded LEAP Legal Software in 1992 and is pioneering the future of AI in legal. The Agentic AI team works closely with Christian to understand his vision and bring it to life, rapidly iterating based on his feedback to launch products that help lawyers. To make this happen you will Partner directly with engineers to build the testing strategy, tooling, and automation that matches an AI-assisted development workflow. Design and implement evaluation frameworks for agentic and generative AI features - regression suites for prompts, models, retrieval quality, tool use, and end-to-end agent behaviour. Own the automated test stack across unit, integration, contract, and end-to-end layers, making pragmatic calls on coverage, tooling, and where human review still matters most. Build the CI/CD quality gates that let the team ship multiple times a day without breaking customer trust - pre-merge checks, canary strategies, and production observability for AI behaviour. Establish the feedback loops between production signals (errors, user corrections, eval drift, cost and latency regressions) and the development cycle so the team learns fast and fixes faster. Shape how the team uses AI-assisted coding tools safely - spotting the failure modes (plausible-but-wrong code, missing edge cases, silent regressions) and building the guardrails that catch them. Present your thoughts, findings, and progress clearly and confidently to internal teams and leadership. What you'll bring You've shipped quality frameworks for products that went from zero to thousands of users, on small, fast-moving teams where you had to build the tooling yourself rather than hand specs to a QA team. Hands-on experience testing Generative AI or agentic systems in production - evals, LLM-as-judge, golden datasets, regression detection on non-deterministic output, cost and latency budgets, safety and hallucination checks. You've seen AI products break in ways traditional QA doesn't catch, and you know how to catch them. Deep fluency with modern test automation - unit, integration, contract, and end-to-end - across cloud services, APIs, and data pipelines. You write code, not just test plans. You've worked on teams using AI-assisted coding (Claude Code, Cursor, Copilot, or similar) and have opinions on how quality practices need to adapt when humans are reviewing AI-generated code at volume. Even better if you have Experience in process-heavy workflows, or SaaS Experience testing scraping automation, or structured extraction systems where correctness really matters Experience in internal venture studios, innovation teams, startups, or product incubation environments Background building out QE practice as the first quality hire on a team You are the type of person who Defaults to the customer - cares less about test coverage percentages and more about whether a lawyer using the product is going to trust what it does Credible and clear with engineers, product, and senior leadership alike - gets quality work done across teams without direct authority, and owns problems end-to-end rather than throwing them over a wall Holds yourself and those around you to a high bar, because you care deeply about what gets shipped Defaults to 'how do we ship th

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at Corto Pty Ltd? Share your experience