Senior AI Platform and Harness Engineer

• →Design, build, and maintain enterprise AI platform capabilities supporting Large Language Models (LLMs), AI agents, RAG, and Generative AI applications.
• →Lead data preparation, training and fine-tuning of LLMs
• →Develop reusable AI harnesses to automate testing, prompt evaluation, model benchmarking, regression testing, and quality assurance.
• →Build AI evaluation frameworks to measure model accuracy, retrieval quality, hallucination detection, latency, throughput, cost, and overall application performance.
• →Implement observability and monitoring solutions for AI applications, including telemetry, tracing, logging, dashboards, and operational metrics.
• →Build and maintain LLMOps pipelines supporting model deployment, versioning, evaluation, experimentation, rollback, and continuous improvement.
• →Design automated workflows for prompt testing, retrieval evaluation, AI system validation, and performance benchmarking.
• →Develop internal tools for prompt management, model experimentation, AI performance optimization, and developer productivity.
• →Build scalable backend services and APIs supporting AI platforms and enterprise AI integrations.
• →Collaborate with AI architects and engineering teams to integrate LLMs, RAG pipelines, vector databases, and agentic AI solutions into enterprise applications.
• →Support deployment of AI services across AWS, Azure, or Google Cloud using containerized and cloud-native architectures.
• →Implement CI/CD pipelines and infrastructure automation supporting enterprise AI development and deployment.
• →Apply security, governance, and Responsible AI controls throughout the AI development lifecycle.
• →Evaluate emerging AI frameworks, LLMOps technologies, evaluation methodologies, and automation tools to improve engineering productivity.
• →Troubleshoot production AI issues and continuously improve platform reliability, scalability, security, and user experience.
• →Document engineering standards, AI platform architecture, evaluation methodologies, and operational best practices.

• Bachelor's degree in Computer Science, Software Engineering, Artificial Intelligence, Data Science, or a related technical field.
• 7+ years of experience in software engineering, platform engineering, backend engineering, DevOps, cloud engineering, or infrastructure engineering.
• 2+ years building or supporting Generative AI, Large Language Model (LLM), or machine learning applications.
• Experience implementing LLMOps or MLOps platforms and deployment pipelines
• LLM Training and Fine-Tuning with demonstrated hands-on experience training, fine-tuning, and optimizing Large Language Models (LLMs), including data preparation, supervised fine-tuning (SFT), parameter-efficient techniques (LoRA/QLoRA), model evaluation, and performance optimization using frameworks such as PyTorch, Hugging Face, and related ML tools.
• Strong programming experience in Python/Typescript
• Experience developing APIs, backend services, and distributed systems.
• Experience with cloud platforms including AWS, Azure, or Google Cloud Platform.
• Experience deploying applications using Docker and Kubernetes.
• Experience working with Git, CI/CD pipelines, Infrastructure as Code (IaC), and infrastructure automation.
• Strong understanding of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), Prompt engineering, Embeddings, Vector databases, AI agents and agentic workflows
• Familiarity with AI evaluation techniques, automated testing, benchmarking, regression testing, and model validation.
• Experience building scalable, production-grade software platforms.
• Strong problem-solving, debugging, and performance optimization skills.

• Experience with AI orchestration frameworks such as LangChain, LangGraph, LlamaIndex, Semantic Kernel, or AutoGen.
• Experience with AI observability tools such as LangSmith, OpenTelemetry, Prometheus, Grafana, Evidently AI, or Arize AI.
• Experience with vector databases including Pinecone, Qdrant, Weaviate, Azure AI Search, or pgvector.
• Experience with OpenAI, Azure OpenAI, AWS Bedrock, Anthropic Claude, Google Vertex AI, or similar enterprise AI platforms.
• Experience implementing Responsible AI, AI governance, model security, and AI safety best practices.
• Experience supporting Federal Government or other regulated environments.
• Experience evaluating AI systems for quality, reliability, accuracy, explainability, latency, and cost optimization.
• Familiarity with healthcare, enterprise modernization, or mission-critical systems.

• The Opportunity to support high-visibility federal missions
• A culture that values innovation, growth, and collaboration
• Access to cutting-edge tools and technologies
• Comprehensive benefits for you and your family
• A career path that rewards ambition and performance

If you’re ready to push boundaries, sharpen your skills, and join a team that is passionate about building what’s next, we’d love to meet you. Apply today and let’s build a future together!