Senior Efficacy Research and Psychometrics Analyst, Khan Kids (24 months fixed-term)

United StatesRemoteFixed termsenior

OtherResearch

3 views0 saves0 applied

Apply Now

Quick Summary

Requirements Summary

Advanced degree, such as a PhD or Master’s, in psychometrics, educational measurement, quantitative methods, or a related discipline, or equivalent applied experience.

Technical Tools

OtherResearch

As a Senior Efficacy Research and Psychometrics Analyst, you will strengthen the evidence base behind assessments and learning content designed for young children in PreK through 2nd grade. You’ll combine rigorous psychometric research with analytics and reporting to help improve learning experiences and measure product impact. The role includes managing external research partnerships, analyzing assessment data, and translating findings into actionable product improvements. You’ll also own KPI dashboards and recurring reporting used by leadership, philanthropy, marketing, and school partnerships teams. Working within a small, interdisciplinary team, you’ll collaborate with product managers, engineers, designers, academic experts, and external research organizations. This is a hands-on opportunity to apply quantitative expertise to meaningful education research and high-impact products.

Serve as the primary technical point of contact for external psychometric research partners, supporting study design, data sharing, analysis, and interpretation.
Extract, clean, validate, and securely share assessment data with research partners on recurring schedules.
Analyze psychometric findings, including item difficulty, score distributions, growth analyses, concurrent validity, and Item Response Theory (IRT)-based scoring.
Translate research findings into actionable product improvements, including potential changes to scoring models and assessment discontinuation rules.
Collaborate with research partners to develop technical documentation, including technical manuals for assessments.
Recruit and onboard school and district research sites, execute MOUs, align on consent and participant incentive procedures, and coordinate site visits for direct child assessment data collection.
Partner with legal and external organizations to develop and negotiate data-sharing agreements involving pseudonymized and identified data.
Coordinate participation in ongoing efficacy studies and contribute to the design and planning of future research initiatives.
Own and maintain the product KPI dashboard in partnership with engineering, including integrations with App Store Connect and Google Play Console.
Ensure analytics dashboards remain useful for marketing, school partnerships, philanthropy, and other internal teams.
Lead recurring app performance reporting, including monthly and quarterly business reviews and periodic target-setting activities.
Respond to ad hoc data requests by developing SQL queries and analyses to support initiatives such as district outreach and grant reporting.
Coordinate multiple external partners, recurring deliverables, data-sharing schedules, and research activities while maintaining strong attention to detail.

Requirements

~2 min read

Advanced degree, such as a PhD or Master’s, in psychometrics, educational measurement, quantitative methods, or a related discipline, or equivalent applied experience.
Strong knowledge of psychometric methods, including item analysis, reliability and validity evidence, growth modeling, and Item Response Theory (IRT).
Proficiency with SQL and a statistical programming language such as R or Python for data extraction, cleaning, analysis, and visualization.
Experience with, or willingness to learn, business intelligence tools such as Looker, along with experience building dashboards and data pipelines.
Familiarity with responsible data-sharing practices in education research, including student privacy considerations such as FERPA and COPPA and the use of pseudonymized or deidentified data.
Strong ability to communicate complex quantitative and psychometric findings clearly to technical and non-technical audiences, including product teams, funders, and school partners.
Excellent project management and organizational skills, with the ability to coordinate external partners, research activities, recurring deliverables, and multiple priorities.
Strong analytical thinking, attention to detail, and commitment to data quality and research rigor.
Demonstrated cross-cultural competency, self-awareness, and ability to contribute to inclusive and collaborative working environments.
Ability to translate research evidence into practical recommendations that influence product or program decisions.
Strong interest in using data, research, and technology to improve learning outcomes for young children.
Experience with early childhood education or assessment of young learners is a plus.
Experience designing quasi-experimental studies in school or program settings is advantageous.
Experience with Spanish-language assessment or dual-language learner measurement is a plus.