Azure Data Engineer (ID: 4069)

As an Azure Data Engineer, you will: Design, develop, and maintain complex and scalable data pipelines using Azure Data Factory (ADF) and Azure Databricks. Build and optimize large-scale data processing solutions using PySpark and Databricks. Develop robust ETL/ELT solutions in Python for data extraction, transformation, and loading. Collaborate with cross-functional teams to understand data requirements and deliver efficient data engineering solutions. Implement and maintain CI/CD pipelines using Azure DevOps for automated testing, integration, and deployment. Develop, test, and maintain reliable data solutions aligned with technical and business requirements. Monitor, troubleshoot, and optimize data pipelines to ensure performance, scalability, and reliability. Work with large datasets and use SQL for data querying and data management. Ensure data solutions follow applicable security, compliance, and engineering best practices. Stay updated with the latest trends, technologies, and best practices in Azure cloud data engineering. What You Bring to the Table: 8–10 years of overall professional experience in data engineering, cloud data engineering, or related technology roles. Strong hands-on experience with Azure Data Factory for data orchestration, movement, and transformation. Strong experience with Azure Databricks for developing and managing scalable data processing solutions. Strong programming skills in Python. Hands-on experience with PySpark for large-scale data processing. Experience implementing CI/CD pipelines using Azure DevOps. Good knowledge of SQL and experience working with large datasets. Experience developing and maintaining ETL/ELT data pipelines. Understanding of data security, compliance, and cloud data engineering best practices. Strong problem-solving and analytical skills. Experience working independently as well as collaboratively within cross-functional teams. You Should Possess the Ability To: Design and develop scalable data pipelines using Azure Data Factory and Azure Databricks. Develop efficient PySpark applications for processing and transforming large volumes of data. Write clean, maintainable, and reusable Python-based data engineering solutions. Build and manage automated CI/CD workflows in Azure DevOps. Perform data extraction, transformation, loading, validation, and integration across different data sources. Optimize data pipelines and Databricks workloads for performance and scalability. Monitor pipelines, identify failures or bottlenecks, and implement appropriate solutions. Troubleshoot data processing and pipeline-related issues effectively. Write SQL queries for data analysis, transformation, and validation. Work with stakeholders to understand requirements and translate them into technical data solutions. Follow appropriate data security, governance, compliance, and development standards. Document technical solutions, data processes, pipelines, and deployment procedures. Adapt to new Azure services, cloud data engineering technologies, and industry best practices. What We Bring to the Table: An opportunity to work on enterprise-scale Azure cloud data engineering projects. Exposure to Azure Data Factory, Azure Databricks, PySpark, Python, Azure DevOps, and SQL. Opportunities to design and optimize large-scale data pipelines and processing frameworks. A collaborative environment involving cross-functional technical and business teams. Opportunities to contribute to cloud data modernization, automation, and CI/CD initiatives. Let’s Connect Want to discuss this opportunity in more detail? Feel free to reach out. Recruiter: Hema Murali Phone: +31 20 369 0609 ; Extn :148 Email: hema.m@stafide.nl LinkedIn: https://www.linkedin.com/in/hema-murali-315999329/