Azure OpenAI & RAG Engineer

SkyePoint Decisions is seeking an experienced AI Engineer with Azure OpenAI experience to design and implement a AI inference and Retrieval-Augmented Generation (RAG) pipeline for Department of State. This role is responsible for provisioning and configuring Azure OpenAI Service on Azure Government, building the RAG pipeline that grounds AI responses in agency playbooks and live vulnerability feeds, and implementing the confidence scoring mechanism that drives the human validation queue. The engineer works in close coordination with a Power Platform Developer on prompt context assembly and with the agency SME on response quality. 

• →Provision and configure Azure OpenAI Service on Azure Government in compliance with DOS GovCloud FedRAMP High requirements. 
• →Design and implement the RAG pipeline using Azure AI Search: vector index configuration, document chunking strategy for agency playbooks and runbooks, embedding model selection, and semantic search tuning. 
• →Build and maintain automated data ingestion pipelines for CISA KEV (daily), NVD CVE database (daily), Microsoft Security Update Guide (weekly), and agency SharePoint document library. 
• →Implement ServiceNow ticket context injection: when a matching SNOW ticket is retrieved, structure the vulnerability description, CVE identifier, asset details, and current ticket status into the AI prompt context for response generation. 
• →Design and implement the AI confidence scoring mechanism — per-response confidence scores surfaced to agency analysts in the validation queue. 
• →Implement CUI-safe prompt construction: ensure sanitized email content and structured SNOW ticket data are assembled into prompts without including raw CUI elements that have been vaulted. 
• →Configure Azure AI Content Safety filters appropriate to the advisory use case. 
• →Monitor model performance, token consumption, and response latency in the Azure Government environment. 
• →Support the monthly review of AI confidence score distribution, override rate, and coverage metrics. 
• →Produce AI system documentation including prompt templates, RAG configuration, and model monitoring procedures. 

• 3+ years of experience with Azure OpenAI Service including API integration, prompt engineering, and deployment in enterprise environments.
• Demonstrated experience building RAG pipelines using Azure AI Search or equivalent vector search technology. 
• Experience with Python or JavaScript/TypeScript for AI pipeline development. 
• Familiarity with LLM prompt engineering techniques including few-shot prompting, chain-of-thought, and system prompt design. 
• Experience with FedRAMP-authorized cloud environments, specifically Azure Government. 
• Experience with REST API integrations for data feed ingestion (JSON APIs, rate limiting, error handling). 
• Active Top Secret security clearance OR ability to obtain TS within program timeline (interim clearance accepted to start).
• US Citizenship is required.

• Active Top Secret clearance.
• Experience with LangChain, Semantic Kernel, or Azure AI Foundry for agentic AI pipeline orchestration. 
• Experience integrating AI systems with ServiceNow or similar ITSM platforms for context enrichment. 
• Familiarity with CISA KEV, NVD CVE data structures, and cybersecurity vulnerability terminology. 
• Experience with Microsoft Copilot Studio or Power Platform AI Builder integrations. 
• Azure AI certifications (AI-102, DP-100, or equivalent). 
• Experience with GCC High M365 environments and their AI service constraints. 
• Familiarity with NIST AI RMF, OMB M-25-21 AI governance requirements, or federal AI policy. 

• At SkyePoint, we go B.I.G. (beginning in GRATITUDE) by recognizing all we have and giving back to our employees, families, and communities. It instills a positive mindset that permeates all we do. By beginning in gratitude, SkyePoint can continue to spread living in gratitude each day.
• Great Benefits: Several insurance options including HMO and High Deductible plans with Health Savings Accounts [HSAs], Flex Spending Accounts [FSAs], Full Dental Plans, ST/LT Disability, Life Insurance, floating federal holiday options, and 401k matched
• Certificate Incentive Program: To promote professional development, we recognize and reward employees who obtain new certifications aligned with business needs.
• Flexible Work Environment

SkyePoint Decisions is an established ISO 9001:2015 and ISO/IEC 27001:2013 certified small business and appraised at CMMI Level 3 for Services and Development. We possess a common vision of excellence and foster a collaborative team culture built upon individual performance and accountability. We invest in our people and systems to create value for our clients. It is the SkyePoint Way. We are grateful for the opportunity to work with exceptional people and give back to the communities we serve. Our employees value the flexibility at SkyePoint that allows them to balance quality work and their personal lives.

SkyePoint Decisions is a participating E-Verify Employer. 

U.S. Citizenship is required for most positions.

Equal Opportunity Employer/Veterans/Disabled.

CCPA Disclosure Notice Here