Principal Data Engineer

India·HyderabadEmployeelead

Data EngineerData

0 views0 saves0 applied

Apply Now

Quick Summary

Key Responsibilities

Design and architect scalable data pipelines and solutions using AWS native services to support complex business

Technical Tools

Data EngineerData

Senior Tech Specialist, Applications & Platforms

Experience Range: 7 to 10 years of experience in advanced AWS data engineering, including significant expertise in Master Data Management (MDM) with Stibo and designing large-scale data solutions Key Responsibilities:

Design and architect scalable data pipelines and solutions using AWS native services to support complex business requirements

Develop, implement, and optimize ETL processes leveraging AWS technologies such as EMR, Redshift, Athena, and DMS

Configure and manage real-time data streaming and ingestion frameworks utilizing Kinesis and Amazon API Gateway

Monitor, troubleshoot, and enhance data infrastructure performance using CloudWatch and Open Search

Automate deployment and infrastructure provisioning with CloudFormation and manage schema migrations using SCT

Lead the implementation and integration of Master Data Management (MDM) solutions using Stibo within enterprise data platforms

Collaborate with cross-functional teams to integrate data solutions with enterprise applications and analytics platforms

Ensure data security, quality, and compliance by implementing best practices and leveraging AWS security features

Lead technical reviews, mentor junior engineers, and drive adoption of emerging data technologies within the organization

Required Skills:

Advanced proficiency in AWS services including SNS, SQS, Amazon API Gateway, Athena, CloudFormation, CloudWatch, DMS, DynamoDB, EMR, Kinesis, Open Search, Redshift, SCT

Expertise in designing and managing data pipelines and ETL workflows on AWS

Strong experience with Spark using Scala for large-scale data processing

Hands-on experience with Oozie workflow scheduling and orchestration

Deep knowledge of real-time data streaming with AWS Kinesis

Proven ability to configure and optimize Amazon Redshift for analytics workloads

Experience with DynamoDB for NoSQL database solutions

Ability to automate infrastructure and deployments using AWS CloudFormation

Proficient in monitoring and logging with AWS CloudWatch

Skill in managing data migration and schema transformation using AWS SCT and DMS

Demonstrated expertise in implementing and integrating MDM solutions using Stibo

Preferred Skills:

Experience integrating AWS data solutions with third-party analytics and visualization tools