Research Scientist – Frontier Data — AfterQuery

Location: San Francisco, CA (Onsite)
Compensation: $150,000 – $250,000 base | $250,000 – $450,000 total cash (including profit sharing) + competitive equity
Visa Sponsorship: None available
Employment Type: Full-Time
Headcount: 2 open seats

AfterQuery builds training data infrastructure and evaluation systems used by frontier AI labs to improve large language models and next-generation AI systems. The company is post-Series A with a small, high-impact technical team, direct partnerships with top frontier AI labs, and a focus on training data quality, evaluation rigor, and post-training optimization — with direct model improvement impact.

This is a hands-on applied research role focused on model behavior, data quality, evaluation design, and RL training systems. This is not a pure academic research seat. This is not theoretical ML research. This is a fast-moving experimental execution role where you design experiments that directly improve frontier AI systems.

Core ownership includes:

• Designing high-signal datasets and identifying model failure modes
• Data slicing strategy, benchmark design, and evaluation framework construction
• Reward signal design, RLHF pipeline support, and RLVR experimentation
• Annotator behavior modeling and dataset diversity measurement
• Alignment capability measurement and quantitative experimentation
• Translating frontier lab training objectives into concrete datasets
• Fast experiment iteration under ambiguity across domains including finance, software engineering, enterprise workflows, and policy-related reasoning

• Strong quantitative research instincts with genuine curiosity about model behavior
• LLM training familiarity — RLHF understanding and RLVR familiarity
• Evaluation methodology fluency
• Research depth equivalent to a strong BS/MS researcher
• Ability to reason about how data structure changes model performance
• Lightweight experimentation ability and comfort with messy, incomplete data
• Cross-domain reasoning ability, fast iteration mindset, and builder mentality

• RL environment company experience (METR, Artificial Analysis, or similar)
• AI safety organization or benchmarking organization exposure
• Evaluation methodology work, dataset curation, or annotator/reward modeling
• Lab research experience and alignment research adjacency
• SWE + research hybrid profile

• Pure theorists or PhD-only slow execution profiles
• Candidates who need clean specs or clean datasets to operate
• Profiles with low experimentation velocity

1. Pending approval
2. Initial screen
3. Take-home
4. Take-home review
5. Onsite
6. Offer

• Role is fully onsite in San Francisco — please only apply if you can commit to this
• No visa sponsorship available

Shortlisted candidates will be contacted by David Joseph & Co., the recruiting partner managing this search on behalf of AfterQuery.