Senior Data Engineer

• Architect, design, and implement end-to-end data solutions using Azure Databricks, PySpark, Azure Data Factory, and Azure SQL.
• Design, build, and maintain data pipelines from data sources through integration to consumption for specific use cases.
• Implement robust data modeling standards across bronze, silver, and gold layers in the data lake.
• Develop data models (conceptual, logical, and/or physical) as required.
• Optimize Spark and SQL workloads for performance, scalability, and cost efficiency.
• Manage metadata using data preparation, integration, and AI-enabled tools and techniques.
 

• Drive automation in data integration; recommend and lead implementation of techniques to automate repeatable data preparation and integration tasks.
• Build API-based integrations (REST/JSON) and real-time ingestion frameworks.
• Automate data workflows using Azure DevOps pipelines and Git-based CI/CD practices.
• Implement parameterized, reusable pipeline templates for ingestion and transformation.
• Develop automated unit, regression, and integration testing frameworks for data jobs.
 

• Prepare and curate high-quality datasets for BI, reporting, and advanced analytics.
• Partner with analytics teams using Power BI, Tableau, or similar platforms to define semantic models and KPIs.
• Implement performance-optimized data models for self-service analytics.
• Will occasionally provide support to end users on the use of data visualization solutions.
 

• Lead technical design reviews, mentor junior engineers, and promote best practices.
• Assist cross-functional groups, business analysts, and stakeholders to gather, define, and refine data requirements.
• Collaborate with business and IT stakeholders to align data engineering with organizational objectives.
• Propose innovative data ingestion, preparation, and integration techniques to address stakeholder requirements.
• Contribute to architectural roadmaps and technology evaluations for the data platform.
• In collaboration with functional leaders, identify inefficiencies and recommend improvements to the executive team.

Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field (Master’s preferred)

5–8 years of experience designing and developing enterprise-scale data solutions (data warehouses, data lakes, operational databases)