Senior Data Engineer

Why we're hiring:

 Our organization is embarking on a transformative journey to unify our global data landscape through a new central initiative. We are moving away from fragmented systems to a standardized, cloud-agnostic, and AI-ready data platform built on a modern tech stack: dbt, Databricks, Python (dlt), and GitHub Actions. 

As a foundational member of our central platform team in Chennai, you will be a hands-on builder responsible for bringing our strategic data blueprint to life. This is a role for an execution-focused engineer who loves building high-quality, governed, and reusable data assets that will be deployed across our global markets and wants to make a tangible impact on a global scale. This role combines data engineering along with an opportunity to enable next-generation AI.

What you'll be doing:

• Develop, test, and deploy robust, scalable, and optimized data transformation pipelines using dbt and SQL on the Databricks platform. 

• Design and maintain scalable dimensional models (SCD1/SCD2), implement advanced partitioning, and ensure high-performance query execution. 

• Enforce and automate data governance, data quality checks, and security (RLS/CLS) using Databricks Unity Catalog and dbt's testing framework. 

• Implement and manage data ingestion from diverse sources (REST APIs, SFTP, SQL databases, Cloud Storage) using Python, preferably with frameworks like dlt. 

• Leverage the full potential of the Databricks Lakehouse Platform, with a focus on Databricks SQL and Unity Catalog for governance, security (RLS/CLS), and data discovery. 
• Design and build data products that support Agentic AI and Generative AI applications, ensuring reliable, governed, and low-latency access to enterprise data.

• Partner with DevOps teams to establish and maintain CI/CD pipelines for data projects using GitHub Actions for automated testing and deployment 

• Document solutions clearly and contribute to operational runbooks and knowledge base. 

• Collaborate with global teams to execute the strategic blueprint, ensuring standards and best practices are followed across markets. 
• Mentor junior and mid-level engineers through code reviews, pair programming, and technical guidance 

What you'll need:

• Experience: 5+ years of dedicated, hands-on data engineering experience, with a strong track record of delivering production-grade solutions 

• dbt Expertise: Expert-level proficiency in dbt (Core), including structuring projects, writing macros, implementing tests, and integrating with CI/CD. This includes experience of resolving real production issues and versed with the best-practices around the same.  

• Databricks Proficiency(with focus on SQL) : Strong, practical experience with the Databricks Lakehouse Platform, specifically Databricks SQL, Unity Catalog, and Databricks Workflows.  Good understanding of Delta Lake , Performance Optimization techniques. 
• Understanding of Generative AI, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), embeddings, vector databases, and semantic layers. Ability to collaborate with AI teams and prepare enterprise data for GenAI use cases.

• Python for Data Engineering: Proficient in Python for data ingestion and automation. Experience with data loading libraries is a major plus. 

• SQL & Data Warehousing Mastery: Deep understanding of advanced SQL, query optimization, MPP engine concepts, and practical experience designing and implementing dimensional models. 

• Diverse Ingestion Patterns: Proven ability to ingest data from a variety of sources (APIs, SFTP, SQL) and big-data file formats. 

• Strong Cloud Fundamentals :  A solid understanding of core cloud services, including object storage (ADLS/GCS), secret management (Key Vault/Secret Manager), and compute concepts (VMs, Functions, Containers) 

• CI/CD for Data: Proven ability to configure and troubleshoot data-specific workflows within a CI/CD framework (GitHub Actions preferred), in partnership with a DevOps team 

• Communication & Collaboration: Excellent communication skills, with the ability to articulate complex technical concepts to diverse audiences and collaborate effectively with distributed, global teams 

Who you are: