Design and develop scalable infrastructure for large-scale ML training, inference, and Generative AI systems.
Build distributed systems and cloud-native platforms supporting GPU clusters, fault-tolerant training, and high-performance AI workloads.
Develop evaluation frameworks for LLMs and agentic AI systems, including hallucination detection, safety validation, robustness testing, and tool-calling reliability.
Architect and optimize retrieval-augmented generation (RAG) pipelines, knowledge management systems, and scalable AI data workflows.
Build backend services, APIs, and production AI infrastructure using technologies such as FastAPI, Kubernetes, Docker, and modern cloud platforms.
Develop automated benchmarking, orchestration, and asynchronous processing systems for enterprise AI applications and evaluation platforms.
Collaborate cross-functionally with research, product, and engineering teams to improve scalability, reliability, observability, and developer productivity across AI systems.
Contribute to full-stack AI applications, developer tooling, and production deployment pipelines supporting next-generation AI-powered workflows.
What We Need to See:
BS, MS, or PhD in Computer Science, Computer Engineering, Electrical Engineering, Statistics, or related technical field (or equivalent experience).
Minimum of 2+ years of related industry experience in software engineering, AI/ML systems, distributed systems, cloud infrastructure, or Generative AI applications.
Strong programming skills in Python and/or C++ with experience building scalable software systems.
Experience developing distributed systems, cloud infrastructure, backend services, or ML systems infrastructure.
Hands-on experience with machine learning frameworks such as PyTorch, TensorFlow, JAX, or DeepSpeed.
Experience with Kubernetes, Docker, and cloud platforms such as AWS, GCP, or Azure.
Familiarity with large language models (LLMs), RAG systems, prompt engineering, evaluation frameworks, or agentic AI workflows.
Experience building APIs and scalable services using frameworks such as FastAPI, Node.js, TypeScript, or related technologies.
Strong understanding of software engineering best practices including CI/CD, automated testing, debugging, observability, and production system reliability.
Ways to Stand Out from the Crowd:
Experience building infrastructure for distributed ML training or large-scale inference systems.
Background in high-performance distributed systems, GPU scheduling, or fault-tolerant training architectures.
Experience developing LLM evaluation frameworks, AI safety systems, hallucination detection pipelines, or agentic AI benchmarking platforms.
Familiarity with knowledge graphs, retrieval systems, vector databases, or scalable RAG architectures.
Experience building Kubernetes-based ML platforms, asynchronous evaluation systems, or cloud-native AI infrastructure.
Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 152,000 USD - 241,500 USD for Level 3, and 184,000 USD - 287,500 USD for Level 4. You will also be eligible for equity and benefits .
Applications for this job will be accepted at least until May 31, 2026. This posting is for an existing vacancy.
NVIDIA uses AI tools in its recruiting processes.
Additional Information
NVIDIA is seeking a highly motivated Software Engineer to join our growing AI and Generative AI engineering team. In this role, you will contribute to the design, development, and evaluation of large-scale AI systems powering next-generation applications in LLMs, agentic AI, retrieval-augmented generation (RAG), and intelligent automation.
You will work closely with cross-functional teams to build scalable AI infrastructure, develop robust evaluation methodologies, and improve the reliability, safety, and performance of production AI services. The ideal candidate combines strong software engineering fundamentals with hands-on experience in machine learning systems, distributed infrastructure, and modern GenAI workflows.
Nvidia · Santa Clara, CA