Senior / Staff AI Engineer — Evals & Model Quality

At FurtherAI, we’re building the AI agents that run insurance — a trillion-dollar industry still powered by manual work. By combining frontier agentic AI with deep domain expertise, we’re reshaping how the industry’s most critical work gets done.

We’ve raised more than $30M from top investors (Andreessen Horowitz, YC, Nexus, South Park Commons), grown more than XY00% in the last 12 months, and work with some of the largest names in insurance — top 10 carriers/brokers, Fortune 100 companies, and public companies.

We are a deeply technical company. Both founders are engineers, and engineers have an outsized say on product direction.

Now, we’re looking for an exceptional AI Engineer to own how we evaluate and continuously improve the quality of AI agents across our platform.

• →

• Build an end-to-end evaluation framework for FurtherAI agents, measuring success across multi-step workflows, tool calls, and complex insurance tasks.

• Create a library of golden datasets and benchmarks for common insurance workflows, reusable across customer implementations and out-of-the-box AI skills.

• Build an automated benchmarking system that evaluates new frontier and open-weight model releases and identifies the best models for different tasks.

• Develop production evaluation pipelines that detect quality regressions, identify failure patterns, and turn them into actionable improvements.

• 5+ years of software engineering experience, with strong hands-on experience building production AI/ML systems.

• Experience building evaluation frameworks, LLM-based applications, model benchmarking systems, or applied AI infrastructure.

• Strong understanding of modern LLMs, agent architectures, evaluation methodologies, and model performance tradeoffs.

• Strong Python engineering skills and the ability to build reliable, production-grade systems.

• Ability to independently own ambiguous technical problems, from research and experimentation through production.

• Clear communicator — both in person and in writing.

• Bonus: Experience with LLM-as-judge, human evaluation systems, agent trajectory evaluations, or large-scale benchmarking.

• Bonus: Experience with open-weight models, fine-tuning, post-training, or model routing.

• Bonus: Background in B2B SaaS and 0-to-1 experience.

• Above all: drive, grit, and ownership. If you excel here, other requirements are secondary.

At FurtherAI, we set a high bar. We’re not looking for someone who just wants a job. We’re looking for someone who wants to build something transformative. If you thrive in environments where the pace is fast, expectations are high, and the rewards are outsized, you’ll love it here.