Skip to main content
Back to jobs

Lead Assessment Developer

External
Outsmart logoOutsmart · Lehi Office
$120K–$180K/yrFull-timeOn-site2mo ago
Documentation
Cover LetterConnect

Prepare for this interview

Elite

AI-generated questions, company research, and talking points tailored to this role

Responsibilities

  • End-to-end assessment development operations for assigned assessments
  • SME network - recruitment, training, calibration, and quality
  • Standard-setting studies - method selection, panel facilitation, and final reporting
  • Vendor and external partner relationships, SLAs, and security protocols
  • Field testing and pilot operations
  • Scoring operations for constructed-response and performance-based assessments
  • Project timelines, risk identification, and stakeholder communication
  • Core assessment documentation - specs, blueprints, training materials, technical reports
  • What Helps You Succeed
  • You thrive in ambiguity and are energized by building from scratch
  • You balance psychometric rigor with startup pragmatism - you know when to ship and iterate
  • You communicate with unusual clarity across technical and non-technical audiences
  • You take ownership without being asked and surface risks early
  • You're excited about AI's role in assessment and know where validation is non-negotiable
  • You care deeply about fairness and the learners behind every score
  • Salary Range: $120K-$180K

Requirements

  • Master's degree (Ph.D. preferred) in educational measurement, assessment , or a related field
  • 3+ years managing operational, high-stakes assessment lifecycles end-to-end
  • Proven experience leading standard-setting studies and managing diverse SME panels
  • Experience managing external vendors for field tests and pilot studies
  • Working knowledge of validity, reliability, and fairness standards (e.g., AERA/APA/NCME)
  • Familiarity with cognitive complexity frameworks (DOK, Bloom's)
  • Strong communicator who can translate psychometric concepts for non-technical audiences
  • Bonus: knowledge of CTT, IRT, CDM, DIF,, AI-assisted item generation, or accreditation reviews

Benefits

Competitive salary and equityHealthcare for you and your dependents (medical, dental, and vision)Parental leave for any full-time employee who has been with us for at least 6 months (birthing, non-birthing, and adoptive parents)15 PTO days per year, a winter break, and 12+ paid company holidays per yearRemote work daysLunch provided in-office dailyWorking with super talented people who want to make a difference in the worldHealth insuranceDental insuranceVision insurancePaid time offRemote work optionsEquity / stock optionsPerformance bonusParental leave

Additional Information

Outsmart is a mission-driven, early-stage startup reimagining higher education for the age of AI by creating an entirely new collegiate experience that expands access and opportunity. Backed by $38M from top venture firms including Khosla Ventures, Forerunner, DST Global Partners, and Lightspeed, the company is led by former Duolingo executives and a founding team that also includes Emmy Award winners. With a strong culture centered around high-impact, detail-oriented team members, Outsmart is already seeing significant traction, generating 1-5 million weekly organic views on social media.

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at Outsmart? Share your experience

Interested in this role?

Apply on the company's website.

Cover LetterConnect