Senior Data Engineer

India·PuneFull-timesenior

Data EngineerData

1 views0 saves0 applied

Apply Now

Quick Summary

Overview

This role is for one of Weekday’s clients Min Experience: 3.5+ yearsLocation: India, PuneJobType: full-time We are looking for an experienced Senior Data Engineer with 3.

Technical Tools

Data EngineerData

Requirements

~1 min read

Experience: 3.5+ years in data engineering or a related field.
Strong analytical and problem-solving skills, with the ability to troubleshoot complex data issues.
Ability to work collaboratively in a fast-paced environment and take ownership of data engineering deliverables.
A focus on building scalable, secure, maintainable, and production-ready data solutions.

Responsibilities

~1 min read

→Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, Python, and/or Scala.
→Build and optimize data ingestion, transformation, and processing workflows using ETL/ELT methodologies.
→Implement and maintain Lakehouse architecture and Delta Lake solutions to support reliable and efficient data storage and processing.
→Work with Azure cloud services, particularly Azure Data Lake Storage Gen2 (ADLS Gen2), to manage and process large volumes of data.
→Optimize Spark jobs and data pipelines for performance, scalability, and cost efficiency.
→Implement data modeling practices to support analytical workloads, reporting, and downstream data consumption.
→Use Unity Catalog and related governance capabilities to manage data access, security, permissions, and compliance.
→Develop and maintain CI/CD pipelines to automate deployment, testing, and release processes for data engineering solutions.
→Monitor production data pipelines, troubleshoot failures, and resolve performance and data quality issues.
→Collaborate with data scientists, analysts, architects, and other engineering teams to deliver dependable data solutions.
→Follow best practices for data quality, documentation, version control, security, and production support.

Strong hands-on experience with Databricks and Azure data engineering services.
Experience working with Azure Data Lake Storage Gen2 (ADLS Gen2).
Proficiency in Python and/or Scala for data processing and pipeline development.
Minimum 3.5 years of relevant experience in data engineering.
Strong understanding of data pipeline development, data integration, and cloud-based data platforms.
Experience with CI/CD workflows, data modeling, governance, and production support.

Hands-on experience with Apache Spark and distributed data processing.
Knowledge of Delta Lake, Unity Catalog, and Lakehouse architecture.
Experience with ETL/ELT development and data transformation frameworks.
Expertise in Spark performance tuning, query optimization, and efficient resource utilization.
Familiarity with data quality monitoring, pipeline orchestration, and automated testing.