Senior Data Engineer

As a Senior Data Engineer, you will lead the design and build-out of drivvn’s data platform — the foundation that turns the activity flowing through our automotive e-commerce platform into trusted, reusable data products. This is a foundational role: you will shape the architecture, set the standards, and own the platform end to end as the data function grows around you.

Our data platform is built on Python and Apache Spark for processing, with data stored as Apache Iceberg tables on Azure Data Lake Storage. We use Trino as our distributed query engine and Metabase for BI, and we are introducing cube.dev as a semantic layer with first-class observability. While these are our current technologies, we remain pragmatic and open to adopting the best tools to achieve our goals.

drivvn provides a class-leading automotive e-commerce platform delivering over £2bn in online sales per annum, used by global OEMs to accelerate their direct-to-consumer business. The data you unlock will directly inform how those brands understand and improve the customer journey.

•     Own the lakehouse. Design, build and operate scalable batch and streaming data pipelines in Python and Spark (PySpark), ingesting and transforming data into well-modelled Apache Iceberg tables on Azure Data Lake Storage.

•     Architect for scale and cost. Make sound decisions on table design, partitioning, schema evolution and Iceberg table maintenance (compaction, snapshot expiry), balancing query performance against storage and compute cost.

•     Deliver the query and semantic layers. Optimise the Trino query layer and build out the cube.dev semantic layer so that consistent, governed metrics are available for self-serve analytics.

•     Enable analytics and BI. Partner with stakeholders to model data and surface reliable, performant insight through Metabase, reducing the distance between a question and a trustworthy answer.

•     Embed observability and quality. Build monitoring, alerting, data quality checks and lineage into the platform. We believe in a true DevOps culture where engineers run what they build.

•     Set the engineering standard. Establish best practices for the data function — version control (Git), automated testing, CI/CD, infrastructure-as-code and clear documentation.

•     Champion governance and security. Ensure data is handled securely and in line with GDPR and our obligations across UK and European markets, including appropriate access controls and data retention.

•     Partner across the business. Work closely with product, engineering and commercial teams to understand data needs and translate them into robust, reusable data products.

•     Lead technically. Evaluate trade-offs, run proofs of concept, advocate for the right technologies, and provide the technical direction for data engineering as the team grows.

•     Strong, hands-on data engineering experience, with expert-level Python and distributed data processing using Spark / PySpark.

•     Practical experience of the lakehouse paradigm and modern open table formats — Apache Iceberg (or Delta Lake / Hudi) — including partitioning strategy, schema evolution and table maintenance.

•     Experience building and operating data platforms on a major cloud, ideally Microsoft Azure (Azure Data Lake Storage), within cloud-native, containerised environments.

•     Proficiency with distributed SQL query engines such as Trino / Presto / Starburst, and strong SQL skills at scale.

•     Experience modelling data and enabling self-serve BI (Metabase or similar). Experience with a semantic / metrics layer such as cube.dev, dbt or LookML is a strong advantage.

•     Solid software engineering fundamentals: version control (Git), automated testing and CI/CD within container-based environments. Infrastructure-as-code (e.g. Terraform) is desirable.

•     A working understanding of data observability, data quality, lineage and governance, with an appreciation of security and GDPR considerations.

•     Specialised expertise in at least one key area: streaming and event processing (e.g. Kafka), advanced data modelling, performance and cost optimisation, or data governance.

•     Proven ability to work independently with minimal oversight and make pragmatic architectural trade-offs, thriving in a startup environment that values adaptability and proactive problem-solving.

•     A positive, proactive and energetic attitude, strong communication skills, and a passion for transforming the digital automotive buying experience through data!

We’ll let you know if you’re invited to an interview or not. But, as a small team with a lot of applications to consider, we can’t give individual feedback on each application.