Lingarogroup5mo ago

Data Engineering Architect

IndiaRemotemid

EngineeringData EngineeringData EngineerArchitectData & AIConstruction & Real Estate

0 views0 saves0 applied

Apply Now

Quick Summary

Requirements Summary

implementing data ingestion pipelines from multiple data sources using Azure Data Factory, Azure Databricks and other ETL tools. developing scalable and re-usable,

Technical Tools

EngineeringData EngineeringData EngineerArchitectData & AIConstruction & Real Estate

Acting as a Senior Consultant/Solution Advisor:

Technology partner: Advise on designing data architectures in Microsoft Azure.

Empowering customer to spot and leverage business opportunities by unlocking the power of data.

Providing technical and problem-solving expertise for client representatives and Lingaro project teams.

Data Integration: Lead data integration efforts, including ETL processes, to ensure seamless data flow.

Cloud Services Utilization: Leverage Azure, as well as tools like dbt and Databricks, for efficient data pipelines and analytics solutions.

Advise on security measures and compliance standards in cloud environments.

Continuously monitor and optimize data solutions for cost-efficiency.

Data Governance: Advise data governance and quality standards.

Collaboration: Work with cross-functional teams to understand requirements and provide data solutions.

Mentoring junior team members in cloud data architecture best practices.

Daily tasks including:

implementing data ingestion pipelines from multiple data sources using Azure Data Factory, Azure Databricks and other ETL tools.

developing scalable and re-usable, self-service frameworks for data ingestion and processing.

designing, building, and managing SQL server databases in the Azure cloud.

performing data modelling and integrating data from various systems

reviewing and implementing best practices for data manipulation

creating and supporting Azure Data Factory pipelines.

Integrating end to end data pipelines to take data from data source to target data repositories ensuring the quality and consistency of data.

11+ years of professional experience in the Data & Analytics area,

5+ years of experience in the Senior Consultant role with a strong focus on Azure data solutions.

Advanced knowledge and understanding of architecture, application design, systems engineering, and integration.

Very good knowledge of cloud data services, data warehousing, big data technologies, and data lakes. Especially Azure, DataBricks

Expertise in data modeling, data integration, and ETL processes.

Proven experience in using of Microsoft ETL tools (Azure Databricks, Azure Data Factory, Data Lake and SSIS)

Databricks Unity Catalog, Asset Bundles, Data Factory, Microsoft Fabric. Logic App, Function App.

Programming skills for data analysis (especially PySpark, SparkSQL, Python, SQL)

Familiarity with data governance, data security, and compliance standards.

Familiarity with agile methodologies and the ability to apply DevOps and DataOps principles to data pipelines.

Ability to use ARM templates

Experience in working with JSON

Good understanding of business processes and the ability to drive automation through effective management.

This role involves designing the project architecture. Setup, monitoring Internal Developer Platform. UC migration. Building Databricks Asset Bundles. PR reviews. Data Model / Semantic Layer setup. Model QA automation.

Stable employment. On the market since 2008, 1500+ talents currently on board in 7 global sites.

“Office as an option” model. You can choose to work remotely or in the office, depending on your location.

Flexibility regarding working hours and your preferred form of contract.

Comprehensive online onboarding program with a “Buddy” from day 1.

Cooperation with top-tier engineers and experts.

Unlimited access to the Udemy learning platform from day 1.

Certificate training programs. Lingarians earn 500+ technology certificates yearly.

Upskilling support. Capability development programs, Competency Centers, knowledge sharing sessions, community webinars, 110+ training opportunities yearly.

Internal Gallup Certified Strengths Coach to support your growth.

Grow as we grow as a company. 76% of our managers are internal promotions.

A diverse, inclusive, and values-driven community.

Autonomy to choose the way you work. We trust your ideas.

Create our community together. Refer your friends to receive bonuses.

Activities to support your well-being and health.

Plenty of opportunities to donate to charities and support the environment.

Modern office equipment. Purchased for you or available to borrow, depending on your location