Data Engineer- Databricks/ SQL/ Python/ PySpark

Data Engineer required for one of our global clients in the UK for their BI & Analytics Team. You will be working with the leading tools and platforms such as Power Platform, Databricks, ADF.

• →Data Pipeline development/orchestration/maintenance
• →Creating robust, fit-for-purpose data modelling solutions to address multiple business cases
• →Root cause analysis and pro-active investigation to identify potential issues and optimize ETL pipelines, notify end-users and propose adequate solutions
• →Prepare documentation for further reference

• Demonstratable delivery experience using Azure Databricks
• Solid delivery experience using ADF, ADLS Gen2, and the Azure Cloud Ecosystem
• Solid SQL skills – Querying syntax, DDL and DML statements
• Solid Python skills – able to use the Python programming concepts in Databricks Notebooks for data loading, transformation and exploration including familiarity with Pandas
• Strong understanding of core Databricks and Spark concepts such as Delta Live Tables, Delta Sharing, Medallion Architecture, Dataframes, Workflows, Unity Catalogue, Delta Live Tables, and UDF’s
• Understands the key principles of Data Warehouse design including Star Schema, F&D design, Dimension and Fact loading patterns, SCD and CDC concepts
• Experience with use of MS DevOps for Scrum management and with Repos and use of CI / CD Pipelines between Azure and DevOps
• Proven track record in a data engineering role where the focus has been on developing data pipelines, data warehouses, data lakehouses, and similar data repositories for BI, Reporting and Analytics use

• Any relevant Azure certifications / exams such as: AZ-900, DP-900, DP-203, or DP-500 
• Any relevant Databricks certifications: Lakehouse Data Engineer Associate, : Lakehouse Data Engineer Professional