Senior Data Engineer

Job Description

We are seeking a highly skilled Senior Data Engineer to join our data and analytics team. The ideal candidate will have extensive experience in working with diverse data sources, performing data profiling, and identifying matching records across systems to create master data records. This role requires strong analytical and technical skills, along with expertise in data quality, integration, transformation, and cloud-based data platforms.

Roles & Responsibilities

Data Source Analysis:

•             Analyze multiple structured and unstructured data sources to understand schemas, data quality, and relationships.

•             Assess and document the metadata, formats, and business rules of incoming data sets.

Data Profiling & Matching:

•             Perform comprehensive data profiling to assess completeness, accuracy, uniqueness, and consistency of data.

•             Develop logic to match and reconcile records across different systems using keys, rules, or fuzzy matching techniques.

Master Data Management (MDM):

•             Design and implement processes for building and maintaining golden master records.

•             Support creation and maintenance of master data across domains such as customer, product, asset, etc.

•             Handle data deduplication and standardization.

ETL & Data Pipelines:

•             Build scalable data pipelines using Azure Data Factory for ingesting, transforming, and cleansing data from multiple sources.

•             Ensure efficient data movement across staging, processing, and analytical layers.

•             Leverage Azure Synapse Analytics (optional) for advanced data processing and analytics

Collaboration & Governance:

•             Work closely with business analysts, data stewards, and application owners to understand data definitions and ownership.

•             Enforce data governance standards and best practices.

Performance & Optimization:

•             Monitor data workflows and optimize performance for large-scale processing.

•             Troubleshoot data issues and implement robust logging and monitoring.

Requirements

Qualification and Education Requirements:

•             Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.

Minimum Experience/Skills Required: 

•             5+ years of experience in data engineering, ETL development, or data integration.

•             Proven experience in data profiling, matching, and MDM (Master Data Management).

•             Expertise in SQL for querying, transforming, and optimizing complex datasets.

•             Strong proficiency in Python or PySpark for data processing and pipeline development.

•             Experience with Azure Data Factory for building and managing data pipelines.

•             Familiarity with Azure Synapse Analytics is a plus.

•             Strong expertise in ETL/ELT tools (e.g., Talend, Informatica, SSIS, Apache NiFi).

•             Familiarity with data quality frameworks and metadata management.

•             Strong communication skills in both Arabic and English.

Preferred Skills

•             Experience with data cataloging tools (e.g., Azure Purview, Alation).

•             Knowledge of data modeling and normalization techniques.

•             Understanding of data privacy and compliance standards (GDPR, HIPAA, etc.).

•             Familiarity with REST APIs and integrating with third-party data services.