Skip to main content
Back to jobs

Software Engineer, Inference AI/ML

External
CoreWeave logoCoreweave · Sunnyvale, CA
$92K–$135K/yrFull-timeOn-site1w ago
CachingGitGrafanaIncident ResponseKubernetesLinux
Cover LetterConnect

Prepare for this interview

Elite

AI-generated questions, company research, and talking points tailored to this role

About the role

Implement well-scoped features and fixes in Python/Go/C++ for model-serving services (e.g., Triton, vLLM, TensorRT-LLM, Ray Serve). Write tests, code comments, and short design docs; participate in code reviews. Add basic metrics and dashboards; assist with alarms and runbooks. Follow on-call runbooks and learn incident response in a guided rotation. Contribute to performance experiments (e.g., request batching, concurrency, caching) with guidance.

Responsibilities

  • Join the Inference team to ship production features that improve latency, reliability, and cost for model serving on our GPU platform. As an IC1, you'll implement well-scoped changes, learn our operational practices, and grow quickly with mentorship from experienced engineers.

Requirements

  • BS/MS in CS, EE, or related field, or equivalent practical experience.
  • Foundations in data structures, algorithms, and networked services.
  • Experience with Python or Go (C++ a plus) and Linux fundamentals; Git/CI basics.
  • Exposure to containers and Kubernetes (coursework or projects welcome).
  • Curiosity about GPU inference concepts (micro-batching, KV cache, streaming).
  • Preferred:
  • Internship or project that deployed a microservice or ML inference demo.
  • Coursework/research with PyTorch or TensorFlow; simple CUDA projects a plus.
  • Familiarity with Grafana/Prometheus/OpenTelemetry or similar tooling.
  • Why CoreWeave?
  • Be Curious at Your Core
  • Act Like an Owner
  • Empower Employees
  • Deliver Best-in-Class Client Experiences
  • Achieve More Together

Benefits

The range we've posted represents the typical compensation range for this role. To determine actual compensation, we review the market rate for each candidate which can include a variety of factors. These include qualifications, experience, interview performance, and location.In addition to a competitive salary, we offer a variety of benefits to support your needs. The benefits below reflect our US-based offerings; for roles in other locations, benefits vary and are shared during the hiring process. These include:Medical, dental, and vision insurance - 100% paid for by CoreWeaveCompany-paid Life InsuranceVoluntary supplemental life insuranceShort and long-term disability insuranceFlexible Spending AccountHealth Savings AccountTuition ReimbursementAbility to Participate in Employee Stock Purchase Program (ESPP)Mental Wellness Benefits through Spring HealthFamily-Forming support provided by CarrotPaid Parental LeaveFlexible, full-service childcare support with Kinside401(k) with a generous employer matchFlexible PTOCatered lunch each day in our office and data center locationsA casual work environmentA work culture focused on innovative disruptionCalifornia ApplicantsCalifornia Consumer Privacy ActEqual Opportunity & AccommodationsAs part of this commitment and consistent with the Americans with Disabilities Act (ADHealth insuranceDental insuranceVision insurance401(k)Paid time offFlexible scheduleEquity / stock optionsPerformance bonusParental leave

Additional Information

CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology, tools, and teams that enables innovators to build and scale AI with confidence. Trusted by leading AI labs, startups, and global enterprises, CoreWeave combines superior infrastructure performance with deep technical expertise to accelerate breakthroughs and turn compute into capability. Founded in 2017, CoreWeave became a publicly traded company (Nasdaq: CRWV) in March 2025. Learn more at www.coreweave.com .

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at CoreWeave? Share your experience

Interested in this role?

Apply on the company's website.

Cover LetterConnect