Consultant

Good Communication & presentation skills.

• Strong Knowledge of Azure Components Azure Data Lake, Azure Data Factory, Azure SQL, Azure Databricks

Data Pipeline Development:

Design, build, and maintain efficient, reusable, and reliable data pipelines.
Ensure data flow between different systems and databases is seamless and error-free.

ETL Processes:

Develop and maintain ETL (Extract, Transform, Load) processes to aggregate data from various sources.
Ensure data quality and integrity throughout the ETL process.

Performance Optimization:

Optimize data pipelines and queries for performance.
Identify and resolve bottlenecks in data processing workflows.

Experience - 2 years or above

Qualification - B.Tech, MTech

• Must have strong knowledge & hands on in Apache Spark and Python programming, working with Delta Tables, Experience in Databricks is must

• Strong SQL Skills: Data modelling, Developing SQL Store Procedures, Functions, Dynamic SQL queries, Joins

• Must be aware of development of components for data fetching from APIs

• Strong knowledge of Data Warehousing Concepts.

• Hands on experience in ingesting data from various data sources and data types & file types

• Should have good knowledge on development lifecycle , best practice, and coding standards. Should be able to help team members, review the code.

• Good knowledge in Azure DevOps, understanding of build and release pipelines