Lead Data Engineer

 

Location: Hybrid  

Division: Engineering 

Reports to: Chief Architect - Data and Insights 

 

ABOUT US  

Agilisys is at the forefront of digital transformation and innovation in the public services sector. With over two decades of experience, we have established ourselves as a trusted partner for governments, local authorities, and organizations nationwide. Our mission is to empower our clients to deliver exceptional public services by harnessing the full potential of technology and data.  

 

OUR VALUES   

Partnership: we become one team and family with organisations, helping them to navigate change and stay agile.  

Integrity: our people really care, going beyond the brief to make change happen for organisations and citizens.  

Innovation: we bring together the right technologies and services to design solutions that work.  

Passion: we are passionate about - and dedicated to - public services and improving people’s lives.  

  

THE ROLE  

The Lead Data Engineer is a pivotal role within Agilisys’ Data & AI Practice, providing technical leadership across high-profile public sector programmes. This is a hands-on leadership position where you will define engineering standards, shape architectural decisions, and deliver modern, secure, and scalable data platforms using Microsoft Fabric and the Azure Data Platform. 

Working closely with architects, analysts, delivery teams, and client stakeholders, you will lead the design and implementation of enterprise-scale data solutions that enable organisations to unlock value from their data while meeting stringent security, governance, and compliance requirements. 

  

Key responsibilities  

Data Platform Strategy and Technical Leadership  

• Lead the design and evolution of scalable data platforms using Microsoft Fabric, Azure Synapse Analytics, and Azure Data Factory. 

• Bridge legacy systems with modern cloud-native architectures. 

• Define engineering standards, reusable patterns, and architectural best practices. 

• Provide strategic technical leadership to ensure performance, scalability, and long-term platform sustainability. 

• Work across distributed data processing, storage, and compute services within the Azure ecosystem. 

Engineering Delivery & Development Best Practice  

• Lead the hands-on development of end-to-end data solutions using Microsoft Fabric components such as Lakehouses, Warehouses, Notebooks, and Dataflows.   

• Design scalable data models and ingestion frameworks, supporting both real-time and batch workloads using best-practice techniques (e.g. 3NF, Kimball).   

• Enable data consumption through well-structured APIs, semantic models, and data products, with a strong understanding of Power BI integration within Fabric.   

• Drive engineering excellence through robust development practices including version control, automated testing, peer reviews, and documentation.   

• Optimise performance and cost-efficiency, tuning queries, storage formats (e.g. Delta, Parquet), and compute resources across Fabric and Azure workloads.  

ETL/ELT and Data Processing  

We're looking for an expert in building dynamic and efficient data pipelines. This includes demonstrable experience in:  

• Design and implement scalable ETL/ELT solutions for large-scale batch and real-time processing. 

• Build and orchestrate complex data workflows using Azure Data Factory, Fabric Data Pipelines, and related technologies. 

• Apply data engineering best practices including schema design, indexing, query optimisation, change data capture (CDC), and slowly changing dimensions (SCD). 

• Develop high-performance transformation logic using Python, SQL, Spark, and related technologies. 

 

Platform Automation & DevOps  

• Lead CI/CD implementation for Microsoft Fabric components, integrating Git for version control and automating multi-environment deployments using Fabric Deployment Pipelines.  

• Embed automated testing across pipelines and transformations, including unit tests, data quality checks, and schema validation.  

• Manage secure configuration using Fabric Environments and Azure Key Vault for secrets and environment-specific parameters.  

Data Security, Governance & Observability  

• Ensure data solutions are secure by design, implementing appropriate access controls, encryption, and authentication mechanisms across all layers of the data platform.  

• Embed governance and compliance principles into engineering workflows, aligning with public sector standards and regulations (e.g. GDPR, UK DPA).  

• Leverage Microsoft Purview (and Fabric-native governance features) to manage metadata, enforce data classification, and support data discovery across the platform.  

• Establish observability and data quality practices, including data profiling, validation, lineage, and monitoring of pipeline health and freshness, while promoting stewardship across teams.  

Specific Duties 

Depending on the project, you will be expected to: 

• Own and deliver key data engineering workstreams from design through to production deployment. 

• Provide technical mentorship and leadership to engineers across delivery teams. 

• Collaborate closely with stakeholders to align technical solutions with business outcomes. 

• Contribute to the growth and maturity of Agilisys’ Data & AI Practice through innovation, thought leadership, and continuous improvement. 

• Support architectural decision-making and ensure best practices are embedded across all delivery activities. 

 

Opportunity  

This role is an excellent platform for experienced data engineering professionals looking to influence how modern data platforms are delivered within the public sector. 

You will join a growing Data & AI Practice focused on delivering innovative solutions across Data, AI, Analytics, and Cloud technologies, with particular emphasis on Microsoft Fabric and the wider Microsoft ecosystem. This role offers significant exposure to enterprise-scale programmes, complex technical challenges, and opportunities to shape future data engineering standards within Agilisys. 

ABOUT YOU 

The successful candidate will have a proven track record of delivering results while navigating change and ambiguity. Strong stakeholder management skills and the ability to collaborate effectively across technical and business communities are essential. 

The Ideal Candidate  

• Has 3+ years’ experience operating at Senior or Lead Data Engineer level delivering enterprise-scale, production-grade data platforms. 

• Holds active Security Clearance. 

• Leads by example through hands-on engineering, code reviews, and mentoring. 

• Demonstrates deep expertise in Microsoft Fabric and keeps pace with its evolving roadmap and capabilities. 

• Has extensive experience working with Azure data technologies and modern data platform architectures. 

• Is comfortable operating in fast-paced environments with evolving priorities. 

• Has experience delivering solutions within public sector or highly regulated environments. 

• Understands Security by Design principles and secure engineering practices. 

• Possesses strong communication and stakeholder management skills. 

• Enjoys collaborating within multidisciplinary and augmented delivery teams. 

• Is passionate about innovation and committed to improving public services through technology and data. 

  

WHAT WE CAN OFFER YOU:  

This role will offer exposure to the right mix of challenges, within a culture that promotes continuous learning and development.  

Benefits include:  

• Enhanced Pension Scheme  

• Life Assurance  

• Access to exclusive discounts and offers through the company’s “Perks at Work” scheme  

• 25 days annual leave (with the option to buy more)  

  

PROCESS  

Simply submit your CV.  

By submitting your CV, you understand that we have a legitimate interest to use your personal data for the purposes of assessing your eligibility for this role.  This means that we may use your personal data to contact you to discuss your CV or arrange an interview or transfer your CV to the hiring manager(s) of the role you have applied for.  You can ask us at any time to remove your CV from our database by emailing talentacquisition@agilisys.co.uk – but please note that this means we will no longer consider you for the role you have applied for.  

We have a rigorous recruitment process, which we use for all our roles to ensure we attract the very best talent.   

Individuals seeking employment at Agilisys must note that we see diversity as something that creates a better workplace and delivers better outcomes. As such, we are keen to maximise the diversity of our workforce and actively encourage applications from all. We encourage diversity through perspective, background, identity, and thought whilst also fostering an environment where everyone can express themselves regardless of your race, religion, sex, gender, colour, national origin, disability, or any other applicable legally protected characteristic. We are committed to continuing to nurture an inclusive environment and building a diverse workforce.