Legal Domain Expert - AI Training & Evaluation

• Education: Juris Doctor (JD) from an accredited law school; education from a highly regarded institution is strongly preferred.
• Experience: 5+ years of substantive post-qualification legal experience at an established law firm, corporate legal department, regulatory organization, court, or comparable institution. Internships and clerkships alone do not qualify.
• Legal Specialization: Demonstrated expertise in at least one substantive practice area, such as:
• Corporate & Transactional Law
• Litigation & Dispute Resolution
• Regulatory & Compliance
• Intellectual Property
• Employment & Labor Law
• Tax Law
• Other specialized legal practice areas
• Seniority: Demonstrated career progression into a senior position such as Partner, Of Counsel, Counsel, Senior Associate, Senior In-House Counsel, or General Counsel, with meaningful ownership of legal matters.
• Licensure: Active admission to at least one U.S. state bar and in good standing.
• AI Fluency: Practical familiarity with large language models and AI tools, along with the ability to distinguish genuinely sound legal reasoning from plausible but inaccurate AI-generated responses.
• Communication: Excellent written communication skills and the ability to provide clear, concise, and well-structured legal analysis and feedback.
• Availability: Ability to commit reliably to 40 hours per week for an initial six-month engagement.
• Location: Currently based in the Bay Area, California, with the ability to work on-site with the team multiple days per week when required. Candidates outside the Bay Area must be willing to relocate at their own expense before the engagement begins. Relocation assistance is not provided.

• → Review & Quality Assurance: Evaluate legal tasks and AI-generated outputs for accuracy, completeness, reasoning quality, and professional relevance. Identify gaps, unsupported conclusions, weak reasoning, and responses that may appear credible but would not meet professional legal standards.
• → Develop Instructions & Reference Solutions: Create detailed task specifications, develop high-quality reference answers, and design legal scenarios that accurately represent real-world legal practice.
• → Build Evaluation Benchmarks: Develop challenging legal tasks and evaluation datasets that can effectively measure AI capabilities across different legal domains.
• → Legal Reasoning & Calibration: Translate professional legal judgment into clear, structured, and teachable evaluation criteria that can be consistently applied across AI outputs.
• → Collaborate with Research Teams: Work closely with AI researchers, program managers, and other subject-matter experts to identify knowledge gaps and improve legal reasoning capabilities.
• → Provide Structured Feedback: Deliver precise written feedback explaining not only whether an AI response is correct, but why it meets or fails to meet professional standards.

The ideal candidate combines deep practical legal expertise with strong analytical and communication skills. You should be comfortable evaluating complex legal work, identifying nuanced errors, explaining your reasoning clearly, and helping transform professional legal judgment into measurable standards for AI training and evaluation.