Senior Software Developer III

• You build it, you run it. End-to-end accountability for the AI platform in production: architecture, delivery, operations, cost, and quality, with on-call for what you ship. No hand-off to a separate ops or QA function. 

• Design, build, and operate agentic AI systems: LLM services, retrieval pipelines, multi-agent orchestration, agent execution runtimes, and human-in-the-loop capabilities. 

• Build and operate memory infrastructure for agent systems (storage, retrieval, promotion/demotion mechanics). Applied Science owns promotion criteria and scientific soundness. 

• Own LLMOps: the harnesses, pipelines, and infrastructure that keep non-deterministic systems reliable and correct in production, including dynamic model selection by task, cost, and latency. 

• Build automated eval infrastructure that continuously compares offline and online metrics and alerts on drift or regression. Applied Science owns the methodology and acceptance thresholds it checks against. 

• Evaluate, adopt, and integrate third-party LLMOps and eval platforms where they accelerate delivery over building in-house. 

• Build the technical controls, telemetry, and guardrails that enforce compliance frameworks (e.g., ISO 42001, AIUC-1) in the architecture itself. 

• Build and maintain the agent definition management APIs and the tool registry definitions reference, keeping definitions stable as tools and models version independently. 

• Lead the engineering build of proof-of-concepts. Applied Science defines the architecture bet and validation criteria being tested. 

• Use AI-assisted development tools — inline completion, conversational assistants, and agentic workflows — as a standard part of your daily workflow.  

• Critically evaluate and take ownership of all AI-generated output before it ships — verify correctness, security, and alignment with architecture and team standards.  

• Identify opportunities for agentic automation within your team, and lead the rollout of new AI-assisted workflows and tooling.  

• Mentor other developers on effective, responsible use of AI-assisted development tools.  

• Help establish team-level practices and guardrails for using AI tools safely and effectively.  

• Build and refine feedback loops that improve the team's collective use of AI tooling over time. 

• 2+ years building and operating production AI systems, with a working grasp of the latency, cost, accuracy, and reliability trade-offs involved. 

• Experience with agent frameworks, agent memory systems, or orchestration of tool-using AI systems. 

• Production AWS experience, including Infrastructure as Code (CDK, CloudFormation, or Terraform). 

• Degree in Computer Science, Software Engineering, or equivalent practical experience; 8+ years of professional software development experience with demonstrated impact beyond a single team. 

• Proven experience designing and operating distributed, cloud-native, SaaS/multi-tenant platforms at scale, ideally on AWS. 

• Strong software fundamentals: OOP, design patterns, SOLID principles, and a track record of advocating for code quality. 

• Experience with agentic-AI software development practices. 

• Experience designing and consuming well-designed HTTP APIs (REST, GraphQL, or similar). 

• Experience mentoring developers and influencing architectural decisions across teams. 

• Strong written and verbal communication skills; comfortable operating in fast-moving environments with ambiguity and evolving requirements. 

• Strong initiative to improve processes, tools, methodologies, and product quality. 

Caseware takes the security of candidates seriously. All legitimate communication from us will come from email addresses ending in @caseware.com and our open positions are always listed on reputable job boards and on our website https://jobs.lever.co/caseware. We will NEVER ask for payment or financial information from you. If you receive an unsolicited job offer, proceed with extreme caution.