Senior Software Engineer, Big Data

In this role, success is measured by the reliability, scalability, and performance of our data platform. You will:

• Lead Technical Design: Own the end-to-end design and delivery of large-scale data ingestion, warehousing, and processing pipelines across billions of daily events—proactively accounting for scalability, failure modes, and security from the start. This includes core data workflows such as the identity graph, ML feature pipelines, and warehouse workload distribution.
• Elevate Reliability: Monitor, troubleshoot, and improve highly available data systems including low-latency data streams and distributed query workloads. Lead blameless post-mortems and implement long-term systemic fixes to prevent incident recurrence.
• Drive Engineering Excellence: Write and optimize complex SQL and Spark queries, extend modern big data tooling (Spark, Flink, Kafka, Iceberg, ClickHouse, AWS EMR/S3), and strengthen the team through high-quality code reviews, technical mentorship, and exemplary technical artifacts such as design docs and architecture diagrams.
• Navigate Ambiguity: Exercise strong judgment to balance long-term data platform health with rapid development velocity—particularly when driving ML feature projection work across large datasets and solving cross-team issues quickly with a small, focused team.
• Collaborate on Direction: Partner with Science, Machine Learning, Product, and Engineering leadership to align technical solutions with business priorities around AI-driven initiatives, including feature engineering, feature projections, and identity graph development.

• Experienced Senior Engineer. You bring 7+ years of experience working with a managed language such as Java or .NET, building production-grade systems.
• Deep Spark Practitioner. You have extensive hands-on experience with Spark in production environments, including scaling large datasets in both Spark and SQL.
• Strong Backend Foundation. You have a proven track record designing, decomposing, and delivering high-scale production services or distributed systems.
• Cloud-native engineer. You have experience building and operating systems in cloud environments such as AWS, Azure, or GCP.
• Owner and Driver. You independently drive technical initiatives from problem definition to deployment, taking full accountability for outcomes without needing granular direction.
• Clear Communicator. You can articulate technical tradeoffs and decisions to both technical and non-technical stakeholders across engineering, science, and product teams.

• Experience with high-volume, low-latency data systems
• Background in distributed systems design for large-scale architectures
• Proficiency in Python
• Familiarity with tools such as Flink, ClickHouse, or Kafka

• Build a deep understanding of the data platform architecture, core pipelines, and key areas of technical debt across ingestion, warehousing, and ML feature systems
• Establish relationships with key partners across Data Science, Machine Learning, and Engineering teams
• Contribute to debugging and minor pipeline improvements to learn the production environment and data flows
• Deliverable: Document one existing pipeline workflow or data infrastructure gap to demonstrate system understanding

• Own the technical delivery of a meaningful improvement to a core data pipeline, warehouse system, or ML feature workflow
• Identify a gap in data platform documentation or engineering processes and lead the fix
• Participate actively in architectural decision-making and design reviews, particularly around scalability and reliability of high-volume systems
• Deliverable: Propose and implement one improvement to system reliability, query performance, or pipeline efficiency

• Independently own a key component of the data platform—such as a critical ingestion pipeline, the identity graph, or ML feature projection workflows
• Deliver a measurable improvement to scalability, latency, or operational performance across billions of daily events
• Be recognized as a trusted technical leader and go-to partner across data science, ML, and engineering teams for your domain

• Scale: Work with some of the largest datasets in the industry and solve real-world problems at a scale few engineers experience
• Impact: Meaningful influence over system design on critical engineering projects from day one
• Collaboration: A highly collaborative, smart, and supportive team environment
• Mission: Build the data infrastructure powering a leading AI-driven advertising platform

• Festiv – We make work fun with cross-team games, events, and creative team bonding.
• Responsiv – You’ll be close to clients and leadership, influencing real outcomes.
• Inclusiv – Diversity and individuality are celebrated across all levels.
• Inventiv – We reward curiosity and embrace bold ideas.
• Transformativ – We support your growth with training, mentorship, and flexibility.
• Collaborativ – We operate across coasts, connected by purpose and teamwork.