Lead Prompt Engineer: Strategic LLM Migration

RemotePart Timelead

Prompt EngineerData & AI

0 views0 saves0 applied

Apply Now

Quick Summary

Overview

Are you a veteran of the AI landscape with a Doctorate or Master's and a track record of shaping LLM performance? Welo Data is seeking a Lead Prompt Engineer to spearhead the technical migration of template architectures into state-of-the-art LLM autoraters. This is a high-stakes leadership role.

Technical Tools

sqlmentoring

Are you a veteran of the AI landscape with a Doctorate or Master's and a track record of shaping LLM performance? Welo Data is seeking a Lead Prompt Engineer to spearhead the technical migration of template architectures into state-of-the-art LLM autoraters.

This is a high-stakes leadership role. You will not only engineer complex prompts but also guide a specialized team, drive the migration strategy, and certify the launch readiness of our next-generation AI solutions.

Strategic Oversight: Manage the full end-to-end technical migration workflow, ensuring a seamless transition from human-rated templates to automated LLM systems.
Team Mentorship: Guide a team of engineers through the complexities of parent-child template clusters and automated optimization.
Advanced Optimization: Direct the use of APG and APO tools, providing the high-level intervention needed when automated systems reach plateaus.
Launch Certification: Take final responsibility for accuracy metrics, running prompt versions against gold data to verify $F_1$ scores, precision, and recall.
Edge-Case Resolution: Architect manual prompt strategies to solve the most difficult "anti-patterns" in legacy architectures.

Schedule: Part-Time (Flexible hours within project milestones)
Location: 100% Remote (Must be based in the United States)
Employment Type: Freelance / Independent Contractor
Duration: Long-term technical migration project

Elite Expertise: At least 7 years of experience as a Prompt Engineer with a proven history of tuning LLMs for structured outputs and complex classification.
Advanced Academic Background: Master’s or Doctorate (PhD) in Computer Science, Computational Linguistics, HCI, or a related analytical field.
Data Mastery: Strong proficiency in SQL and deep-dive error pattern analysis to monitor and enhance model performance.
Technical Agility: Expert-level ability to master proprietary tools and enterprise-grade interfaces like the Goose API.
Leadership Excellence: A natural mentor with the verbal and written communication skills required to justify technical launches to stakeholders.

Nice to Have

~1 min read

Experience in software engineering or AI model evaluation at an enterprise scale.
Deep linguistic expertise, specifically in semantics and formal logic.
Hands-on experience with Chain-of-Thought (CoT) and shadowbot disagreement tracking.

APPLY