Skip to main content
Back to jobs

Staff Software Engineer, Inference

External
CoreWeave logoCoreweave · Sunnyvale, CA
$188K–$275K/yrFull-timeOn-site2w ago
CachingKubernetesLeadershipMovePerformance OptimizationPython
Cover LetterConnect

Prepare for this interview

Elite

AI-generated questions, company research, and talking points tailored to this role

Requirements

  • 8-12+ years of experience building and operating large-scale distributed systems or cloud platforms
  • Proven experience leading cross-team technical initiatives impacting multiple services or organizations
  • Strong programming skills in Go, Python, or C++
  • Deep expertise in Kubernetes at production scale, including orchestration, scheduling, and service design
  • Strong understanding of distributed systems, networking, and performance optimization
  • Experience designing and operating low-latency, high-throughput systems with strict P95/P99 latency requirements
  • Hands-on experience with inference systems, including batching or micro-batching strategies, caching, and memory optimization
  • Experience improving system performance using metrics-driven approaches (e.g., latency, throughput, utilization)
  • Familiarity with mixed precision (BF16, FP8) and streaming inference workloads
  • Preferred:
  • Experience with inference frameworks such as vLLM, Triton, TensorRT-LLM, Ray Serve, or TorchServe
  • Experience with GPU systems and performance optimization (CUDA, NCCL, RDMA, NUMA, GPU interconnects)
  • Experience leading multi-team or org-level technical initiatives
  • Exposure to large-scale AI/ML infrastructure or hyperscale cloud environments
  • You love to design and optimize high-performance distributed systems at scale
  • You're curious about AI inference, GPU systems, and emerging performance techniques
  • You're an expert in building reliable, low-latency infrastructure and driving system-wide improvements
  • Be Curious at Your Core
  • Act Like an Owner
  • Empower Employees
  • Deliver Best-in-Class Client Experiences
  • Achieve More Together

Benefits

The range we've posted represents the typical compensation range for this role. To determine actual compensation, we review the market rate for each candidate which can include a variety of factors. These include qualifications,Equity / stock optionsPerformance bonus

Additional Information

CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology, tools, and teams that enables innovators to build and scale AI with confidence. Trusted by leading AI labs, startups, and global enterprises, CoreWeave combines superior infrastructure performance with deep technical expertise to accelerate breakthroughs and turn compute into capability. Founded in 2017, CoreWeave became a publicly traded company (Nasdaq: CRWV) in March 2025. Learn more at www.coreweave.com .

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at CoreWeave? Share your experience

Interested in this role?

Apply on the company's website.

Cover LetterConnect