Join a technically driven team focused on developing high-quality scientific and engineering tasks for AI training and evaluation. In this leadership role, you will guide a team of 5–10 technical trainers responsible for creating realistic, terminal-based challenges that reflect authentic engineering workflows. You will oversee the entire task development lifecycle, from technical design and implementation to validation, testing, and final approval. Working across computer science, scientific computing, and electrical engineering, you will ensure that each task demonstrates technical accuracy, reproducibility, and meaningful problem-solving complexity. You will also establish quality standards, mentor technical contributors, and improve evaluation methodologies to support reliable AI assessment. This fully remote, full-time opportunity is ideal for an experienced technical leader passionate about engineering excellence, scientific rigor, and advancing AI capabilities through robust technical benchmarks.
Team Leadership and Mentorship: Lead, mentor, and onboard a team of approximately 5–10 technical trainers. Allocate assignments, monitor productivity, resolve blockers, review performance, and ensure the timely delivery of high-quality technical tasks.
Technical Review and Validation: Oversee the complete task lifecycle, reviewing problem statements, codebases, circuit netlists, hardware description languages (HDLs), datasets, computational environments, reference solutions, and automated tests.
Scientific and Engineering Task Assessment: Validate tasks across high-performance computing (HPC), parallel computing, algorithms, distributed systems, compilers, operating systems, databases, machine learning systems, GPU computing, circuit design, signal processing, power systems, and embedded systems.
Technical Accuracy and Correctness: Verify algorithmic correctness, computational complexity, numerical precision, memory management, concurrency behavior, circuit laws, power balance, signal processing accuracy, timing constraints, and system stability.
Environment and Reproducibility: Validate compilers, simulation tools, software dependencies, build processes, and computational environments to ensure consistent, reliable, and reproducible task execution.
Testing and Evaluation Quality: Review automated tests, graders, and evaluation criteria to ensure they measure meaningful technical outcomes, including functional correctness, performance, frequency response, waveforms, timing, efficiency, and overall system behavior.
Reliability and Edge-Case Analysis: Identify nondeterministic tests, race conditions, hardware-dependent performance thresholds, simulation convergence issues, incorrect tolerances, flawed assumptions, and potential ways to bypass computational or simulation requirements.
Quality Assurance and Feedback: Provide precise, actionable technical feedback, track revisions, and ensure identified issues are resolved before tasks receive final approval.
Technical Documentation and Standards: Develop and maintain technical guidelines, review criteria, documentation, and best practices that promote consistency, accuracy, and quality across the team.
Process Improvement and Benchmark Development: Improve task creation and evaluation workflows, strengthen quality assurance practices, and contribute to reliable benchmarks for AI training and assessment.
Cross-Domain Technical Collaboration: Apply deep engineering knowledge to assess complex implementations and ensure that tasks require authentic, multi-step technical reasoning rather than superficial or easily bypassed solutions.