Senior Data Engineer

We are looking for Senior Data Engineers to join our Data Platform team and build scalable, high-performance data platforms that power data processing, analytics, and downstream applications.

The ideal candidate will have strong experience in distributed data processing, ETL pipelines, and Big Data technologies, with hands-on expertise in Apache Spark and Python Scala.

You will be responsible for designing, developing, and optimizing large-scale data pipelines while collaborating closely with cross-functional engineering teams to build reliable, production-grade data solutions.

• →Design, develop, and maintain scalable ETL and data processing pipelines for large-scale datasets.
• →Build and optimize distributed data applications using Apache Spark and Python Scala.
• →Develop reliable, high-performance data pipelines for batch and streaming workloads.
• →Design and manage data workflows using Apache Airflow.
• →Build and operate data workloads on AWS, with strong usage of Amazon S3 for large-scale data storage.
• →Work with large datasets to ensure data quality, consistency, reliability, and performance.
• →Collaborate with engineering, product, analytics, and other platform teams to deliver robust data solutions.
• →Optimize data workflows for scalability, reliability, performance, and cost efficiency.
• →Troubleshoot production issues, identify bottlenecks, and continuously improve platform performance.

Candidates who demonstrate:

• 5+ years of experience in Data Engineering, Big Data Engineering, or a similar role.
• Strong hands-on experience with Apache Spark and Scala.
• Experience designing, building, and maintaining large-scale ETL pipelines.
• Strong hands-on experience with AWS, particularly Amazon S3.
• Hands-on experience with Apache Airflow for workflow orchestration and scheduling.
• Strong SQL skills and a solid understanding of distributed data processing concepts.
• Experience working with batch and/or streaming data pipelines.
• Excellent debugging, problem-solving, and performance optimization skills.
• Strong communication and collaboration skills.
  

• Experience with Databricks and the broader Databricks data platform.

• Familiarity with streaming technologies such as Apache Kafka.

• Experience working on large-scale data platforms handling high-volume data workloads.

• Exposure to additional AWS data services and cloud-native data architectures.