AI Research Scientist – Scientific Evaluation (ML/AI)

Join an advanced AI research evaluation project focused on assessing scientific work produced by autonomous AI research agents. In this remote, freelance role, you will apply your expertise in machine learning and artificial intelligence to evaluate research papers against the rigorous standards of leading academic conferences. You will assess scientific novelty, technical correctness, experimental design, reproducibility, and the validity of research conclusions. Your expert judgment will help distinguish meaningful scientific advances from flawed methodologies or unsupported claims. Working independently with structured evaluation guidelines, you will contribute to the development and assessment of frontier AI systems capable of conducting end-to-end scientific research. This opportunity offers flexible hours, competitive hourly compensation, and the chance to apply your research expertise to emerging AI capabilities.

The ideal candidate is an accomplished ML/AI researcher with a strong publication record, deep familiarity with current research, and the ability to critically evaluate complex scientific contributions. Candidates must demonstrate the following qualifications:

• A PhD in Machine Learning, Artificial Intelligence, Computer Science, Statistics, or a closely related technical discipline, or equivalent demonstrated research expertise.

• Ideally, first-author publications or other significant research contributions demonstrating substantial scientific involvement.

• Extensive hands-on ML/AI research experience, including developing novel methods, designing experiments, analyzing results, and publishing original research.

• Strong knowledge of scientific methodology and experimental rigor, including baseline selection, statistical validity, benchmark design, ablation studies, and reproducibility.

• Ability to independently assess the novelty, significance, technical soundness, and scientific validity of advanced ML/AI research.

• Familiarity with state-of-the-art research and the ability to evaluate new approaches against established literature and benchmarks.

• Experience reviewing research papers for leading ML/AI conferences or journals, particularly NeurIPS, ICML, or ICLR; this is strongly preferred.

• Ability to verify technical arguments, experimental evidence, scientific claims, citations, and code implementations where relevant.

• Excellent analytical skills and scientific writing abilities, with the capacity to communicate nuanced technical judgments through clear, concise, evidence-based feedback.

• Ability to apply detailed evaluation guidelines and scoring rubrics consistently while maintaining independent scientific judgment.

• Exceptional attention to detail, particularly when identifying subtle methodological weaknesses, unsupported conclusions, and discrepancies between claims and evidence.

• Willingness to provide a CV, a brief overview of your research background and areas of expertise, and links to at least three qualifying peer-reviewed publications.

• Relevant academic peer-review experience, including service as a reviewer or area chair at major research conferences, should be highlighted where applicable.