Manager, AI Analytics

National Debt Relief is seeking a data-savvy Manager of AI Analytics to serve as an individual-contributor analytics expert supporting the measurement, testing, and optimization of agent-replacement AI across voice and chat channels. This role will execute the analytics framework that governs AI interaction performance by designing tests, tracking KPIs, and building the dashboards that give the business real-time visibility into how AI is performing against quality, containment, and client experience goals.

Reporting to the Director of Digital & AI Analytics, this individual will be a hands-on analytical partner to AI Analytics and Engineering teams, translating complex model and interaction data into clear, actionable recommendations that inform the pace and scope of AI scale-up.

• →Execute the measurement framework for AI Analytics, tracking performance of agent-replacement AI interactions across voice and chat channels.
• →Design and run test analysis for AI model and prompt experiments (e.g., model version A/B tests, containment vs. escalation testing), translating results into go/no-go recommendations.
• →Build and maintain dashboards and scorecards tracking AI-handled interaction KPIs — containment rate, accuracy, quality/CSAT, and escalation rate.
• →Partner with AI Analytics and Engineering teams to ensure AI interaction data (transcripts, model outputs, outcomes) flows cleanly into unified reporting.
• →Analyze AI-handled interaction volume and performance trends to support decisions on scaling AI across channels, balancing performance thresholds and client experience.
• →Collaborate with data engineering to validate and maintain data flows into Snowflake for AI analytics use cases.
• →Communicate findings and recommendations clearly to both technical and non-technical stakeholders, including Analytics leadership.
• →Identify data quality issues or gaps in AI interaction tracking and drive resolution with cross-functional partners.
• →Stay current on conversational AI/LLM measurement practices and evolving best practices in AI performance analytics.

Education/Experience:

• Bachelor's degree in Business, Economics, Data Science, or related field required.
• 6 years of experience in analytics, business intelligence, or similar roles.
• Experience in financial services or fintech, especially client onboarding or customer support journeys preferred.
• Previous exposure to leading or mentoring junior analysts, even without formal management experience preferred.

Required Skills/Abilities:

• Strong SQL skills (Snowflake preferred) and comfort with data modeling and manipulation.
• Experience analyzing conversational AI / LLM-driven interactions (chat or voice bots), including containment, accuracy, and quality metrics.
• Experience with A/B testing frameworks and experimentation design, ideally for AI models or prompts.
• Experience with visualization and dashboarding tools like Tableau, Sigma, Looker, or Power BI.
• Ability to communicate complex data clearly and drive action across technical and non-technical teams.
• Ability to consolidate, understand, and communicate large data sets independently.
• Familiarity with server-side tracking, CDPs (e.g., Segment, mParticle), or customer data governance frameworks preferred.
• Experience with Python, BigQuery ML, or Airflow is a plus.

National Debt Relief Role Qualifications:

• Computer competency and ability to work with a computer.
• Prioritize multiple tasks and projects simultaneously.
• Exceptional written and verbal communication skills.
• Punctuality expected, ready to report to work on a consistent basis.
• Attain and maintain high performance expectations on a monthly basis.
• Work in a fast-paced, high-volume setting.
• Use and navigate multiple computer systems with exceptional multi-tasking skills.
• Remain calm and professional during difficult discussions.
• Take constructive feedback.
• Available for full-time position.

National Debt Relief was founded in 2009 with the goal of helping an expanding number of consumers deal with overwhelming debt. We are one of the most-trusted and best-rated consumer debt relief providers in the United States. As a leading debt settlement organization, we have helped over 450,000 people settle over $10 billion of debt, while empowering them to lead a healthier financial lifestyle and feel free to live their best life. At National Debt Relief, we treat our clients like real people. Our purpose is to elevate, empower, and transform their lives.

Rated A+ by the Better Business Bureau, our goal is to help individuals and families get out of debt with the least possible cost through conducting financial consultations, educating the consumer and recommending the appropriate solution. We become our clients' number one advocate to help them reestablish financial stability as quickly as possible.

Want to learn more about who we are? Connect with us on social!