Skip to main content

Associate Lead - Testing (QA + MLOps) at Quantiphi

Pricing Companies Market Trends For Employers FAQ Sign in

Associate Lead - Testing (QA + MLOps)

External

Quantiphi · Ka Bengaluru, India

ContractHybridToday

API GatewayAWSCI/CDClassificationCloudFormationExcel

Cover Letter Connect

Prepare for this interview

AI-generated questions, company research, and talking points tailored to this role

Responsibilities

AI/ML & GenAI Testing Strategy (AWS Ecosystem)
Define testing approaches for AI systems built on AWS services such as:
Amazon SageMaker

© 2026 Hiro by Mistiq

Privacy Terms FAQ Employer FAQ Support

Amazon Bedrock

AWS Lambda

Amazon API Gateway

Amazon Kinesis

AWS Glue

Amazon S3

Amazon CloudWatch

Design validation frameworks covering:

Model accuracy & performance validation

Data drift & concept drift detection

Hallucination detection for LLMs

Prompt robustness testing

RAG validation (retrieval accuracy + grounding)

Bias & fairness validation

Safety & toxicity testing

MLOps Quality Engineering (AWS-Centric)

Validate the end-to-end ML lifecycle including:

Data ingestion & feature pipelines

Model training & hyperparameter tuning

Model versioning & registry

Deployment validation

Canary & blue/green release validation

Work with AWS-native services such as:

SageMaker Pipelines

SageMaker Model Monitor

SageMaker Feature Store

Bedrock model evaluation workflows

CloudWatch-based observability

Implement CI/CD quality gates for ML pipelines integrated with AWS DevOps tools.

GenAI & Agentic AI Testing

Define quality engineering approaches for:

LLM-based applications using Amazon Bedrock

Prompt engineering validation

Multi-agent orchestration testing

Chatbot & Voice bot conversational testing

Intent classification validation

Conversation drift & fallback validation

API contract validation for LLM integrations

Build reusable evaluation harnesses for:

BLEU / ROUGE scoring

Embedding similarity scoring

Response consistency

Safety scoring frameworks

Framework & Capability Development

Design reusable AI testing accelerators

Create AWS-aligned AI test automation frameworks (Python-first)

Develop synthetic data generation strategies

Establish AI quality scorecards

Build an internal AI QA Center of Excellence

Client Engagement & Leadership

Lead AI/ML quality strategy workshops

Perform AI risk & readiness assessments

Present quality architecture to CXOs

Drive QA transformation programs

Mentor QA teams on AWS-based AI testing

Own delivery for AI testing engagements end-to-end

Must have skills:

Testing Expertise

8-12+ years in Quality Engineering

Strong test strategy, automation & governance experience

Experience leading QA transformation initiatives

Experience building frameworks from scratch AI/ML & GenAI Expertise

Deep understanding of ML lifecycle

Experience testing ML models (NLP preferred)

Hands-on experience validating LLM applications

Strong understanding of:

Prompt engineering

RAG architecture

Embeddings

Bias & explainability AWS AI/ML Expertise

Hands-on experience with:

Amazon SageMaker (training, deployment, monitoring)

Amazon Bedrock (LLM integration & evaluation)

S3-based data pipelines

AWS IAM (security validation)

CloudWatch monitoring

Lambda & API Gateway integrations

AWS CI/CD (CodePipeline / CodeBuild preferred)

Understanding of:

Infrastructure as Code (Terraform / CloudFormation)

Observability in AI systems

Cost monitoring for ML workloads

Technical Skills

Python (mandatory)

Experience with ML libraries (Scikit-learn, TensorFlow, PyTorch)

Experience with LLM frameworks (LangChain, etc.)

API & automation testing frameworks

Git-based workflows

Leadership & Communication

Strong client-facing communication

Experience leading QA teams

Ability to create strategy decks & solution proposals

Strong stakeholder management

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us !

Additional Information

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth. If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi! Role: Lead/Associate Lead - QA + MLOps & Generative AI Experience: 10+ years Location: Mumbai/Bangalore (Hybrid)

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at quantiphi? Share your experience

Interested in this role?

Apply on the company's website.

Cover Letter Connect