Director, Free Response Rating Capability Analyst

About the role

The AP Reading and Scoring team is responsible for delivering high-quality, reliable, and scalable scoring operations for millions of Advanced Placement (AP) free response exam submissions annually. The team partners across Scoring, Reader Management, Psychometrics, Assessment Production, and Technology to continuously improve scoring quality, operational effectiveness, and digital scoring capabilities that support AP's mission and growth. As the Director, FR Rating Capability Analyst, you will play a critical role in designing, building, and improving the operational and analytic capabilities that enable high-quality AP Reading scoring. This role sits at the intersection of scoring operations, data analysis, digital scoring systems, and process improvement. You will translate scoring performance insights into scalable tools, workflows, reporting frameworks, and operational improvements that strengthen scoring accuracy, efficiency, and reader effectiveness. You will collaborate closely with cross-functional partners to support continuous improvement across the AP Reading lifecycle and help scale digital scoring capabilities for the future. In this role, you will: FR Rating Capability Design & Ownership (35%) Define and maintain the capability roadmap for FR rating operations, including tools, workflows, reporting structures, and performance measurement systems that support high-quality scoring at scale. Translate operational needs and scoring insights into clearly defined business requirements, system enhancements, reporting frameworks, and process improvements. Partner with Technology and Product teams to design, test, and implement enhancements to scoring platforms and data pipelines. Develop scalable frameworks for evaluating scoring effectiveness, including consistency, accuracy, throughput, and reader-level performance metrics. Establish and maintain expertise in AP exam item types, rubric structures, allowable response formats, and annual assessment changes that impact scoring operations. Serve as a liaison across Scoring, Reader Management, Assessment Production, Psychometrics, and Technology teams regarding FR rating operational systems and processes. Operational Integration & Reading Support (30%) Support the end-to-end AP Exam scoring workflow, including reader training, scoring operations, psychometric data collection, and operational reporting. Partner with cross-functional teams to ensure scoring systems are configured appropriately for annual AP Reading operations across multiple subjects, forms, and item types. Monitor scoring performance signals during live AP Reading and enable rapid operational interventions through data-driven insights. Collaborate with Reader Management and Workforce Operations teams to align reader performance insights with recruitment, training, and retention strategies. Support onboarding and training for stakeholders on scoring tools, scoring system processes, and calibration strategies. Support AP Reader communications related to scoring platform usage, engagement portals, messaging, and operational processes. Provide operational support during peak scoring periods and help resolve implementation or escalation issues related to scoring systems and workflows. Scoring Data Analysis & Performance Insights (20%) Develop analytic models, dashboards, and reporting tools that provide real-time and post-Reading insights into scoring performance. Analyze scoring trends to identify operational risks, anomalies, reliability concerns, training gaps, and workflow inefficiencies. Translate complex data into actionable insights and recommendations for scoring leadership and operational stakeholders. Partner with Psychometrics and Research teams to align operational metrics with scoring validity and reliability standards. Establish standardized reporting and postmortem analysis processes that support continuous improvement across Reading cycles. Process Standardization & Continuous Improvement (15%) Develop and maintain standardized processes, governance documentation, and operational frameworks for FR rating analytics and reporting. Create and maintain cross-functional RACI documentation related to digital scoring platform deployment, configuration, and operational support. Lead retrospective reviews following Reading cycles and synthesize findings into prioritized operational and capability improvements. Drive automation and efficiency initiatives that reduce manual processes and improve operational accuracy and speed. Identify and implement best practices across subjects to improve consistency in scoring operations and performance management. Contr