Staff Software Engineer - Autonomy Data & Tooling

We are seeking a highly skilled and experienced Staff Data Engineer to join our growing Data Platform team. This is a critical role responsible for designing, building, and optimizing scalable data pipelines fundamental to our data platform initiatives, with a particular focus on rapidly ingesting and processing the massive volumes of data our vehicles collect. You will play a pivotal role in enabling the efficient and reliable flow of data from vehicles to the systems and teams that depend on it, ensuring timely and complete data availability for downstream analysis and simulation.

• →Lead the design, development, and implementation of highly robust and scalable data pipelines for ingesting, processing, and transforming vehicle data to cloud.
• →Architect and implement real-time streaming solutions for continuous data offloading, prioritizing low-latency delivery, high throughput, and optimal resource utilization (especially CPU) across both vehicle and offload infrastructure.
• →Optimize real-time pipelines, improving data freshness and time-to-insight while maintaining cost-efficiency and medallion architecture principles.
• →Champion best practices for data quality, data governance, and data security throughout the data lifecycle.
• →Implement and manage advanced queuing and prioritization mechanisms for data processing jobs, moving beyond first-come, first-served to enable on-demand data prioritization. 
• →Foster a culture of rigorous testing, documentation, and continuous improvement within the data engineering team.
• →Mentor and provide technical leadership to junior and mid-level data engineers, promoting knowledge sharing and technical growth.
• →Collaborate closely with software engineers, cloud architects, data scientists, and product managers to understand data requirements and deliver impactful solutions.
• →Evaluate and recommend new technologies and approaches to enhance our data platform capabilities.

Candidates most successful in this role typically hold the following qualifications or comparable knowledge or experience:

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, or a related quantitative field.
• 15+ years of experience in data engineering, with at least 5 years in a lead or senior technical role.
• Expert-level proficiency in distributed data processing frameworks.
• Demonstrated expertise in optimizing data transfer from edge devices.
• Strong understanding and practical experience with performance optimization in resource constrained environments.
• Strong programming skills in Python, with a strong preference for candidates with significant experience in lower level languages used in embedded systems.
• Extensive experience with cloud platforms (AWS, Azure, or GCP), particularly with data storage, compute, and messaging services (e.g., S3, Kafka, Kinesis)..
• Proven ability to design and implement robust, fault-tolerant, and scalable data stream processing systems.
• Familiarity with containerization technologies (Docker, Kubernetes).
• Strong understanding of database concepts (SQL and NoSQL).
• Experience with version control systems (Git) and CI/CD pipelines.
• Demonstrated ability to troubleshoot complex data pipeline issues and optimize performance.
• Knowledge of software engineering best practices, including design patterns, clean code and testing methodologies.
  
  

• Standard office working conditions which includes but is not limited to:
• Prolonged sitting
• Prolonged standing
• Prolonged computer use