Woven By Toyota2mo ago

Staff Machine Learning Engineer - World Foundation Model

Palo AltoEmployeelead

Data ScienceOtherMachine Learning EngineerStaff Machine Learning EngineerData & AI

1 views0 saves0 applied

Apply Now

Quick Summary

Overview

Woven by Toyota is enabling Toyota’s once-in-a-century transformation into a mobility company. Inspired by a legacy of innovating for the benefit of others,

Technical Tools

Data ScienceOtherMachine Learning EngineerStaff Machine Learning EngineerData & AI

Woven by Toyota is enabling Toyota’s once-in-a-century transformation into a mobility company. Inspired by a legacy of innovating for the benefit of others, our mission is to challenge the current state of mobility through human-centric innovation — expanding what “mobility” means and how it serves society.

Our work centers on four pillars: AD/ADAS, our autonomous driving and advanced driver assist technologies; Arene, our software development platform for software-defined vehicles; Woven City, a test course for mobility; and Cloud & AI, the digital infrastructure powering our collaborative foundation. Business-critical functions empower these teams to execute, and together, we’re working toward one bold goal: a world with zero accidents and enhanced well-being for all.

TEAM

At Woven by Toyota, we are at the forefront of developing advanced Machine Learning solutions for autonomous driving. Our team tackles groundbreaking challenges in designing state-of-the-art neural networks, pioneering innovative end-to-end architectures, and advancing ML techniques in perception, prediction, and motion planning. We're passionate about pushing the boundaries of autonomous systems through deep learning and optimization, particularly in complex visual scenarios. We're seeking passionate innovators and creative problem-solvers eager to redefine mobility through cutting-edge AI and robotics, contributing directly to shaping the future of self-driving technology.

Woven by Toyota is developing a joint project between Toyota Research Institute (TRI) and Woven by Toyota to research and develop a visual-based world model as a learned simulator to evaluate end-to-end automated driving. This cross-org collaborative project is synergistic with TRI's automated driving advanced development division's efforts in Diffusion Policy and Large Behavior Models (LBM).

WHO ARE WE LOOKING FOR?

A technical lead responsible for the vision and strategy of world foundation models research and development in the automated driving domain. As lead, you will also help bridge connections between research and production programs. This role requires excellent communication skills and a collaborative mindset to navigate the joint nature of the Woven & TRI collaboration. The applicant is expected to have a wide technical knowledge of the state-of-the-art approaches in robotics/autonomous driving to define vision, scope necessary to initiate long-term open-research efforts.

Lead the design, development and benchmarking of state-of-the-art world foundation models for autonomous driving, ranging from data strategy, multistage training, model selection, and eventual deployment and integration with onboard and offboard applications.

Architect visually realistic simulators to evaluate full end-to-end autonomy stack behavior, from simulating sensors to policy rollouts, across a diverse range of scenario conditions.

Research and implement cutting-edge approaches across domains (reinforcement learning, probabilistic & generative modeling, scene representations, sensor fusion, temporal reasoning) and validate their effectiveness in simulation and through real-world driving performance.

Align efforts across various company-internal teams as well as TRI, providing technical mentorship and fostering a collaborative, high-trust engineering culture across organizational boundaries, influencing technical decisions across the partnership, and possibly co-authoring publications for premier conferences and journals.

Increase the scalability of ML pipelines to support the training and inference of large foundation models, and to optimize edge deployment of state-of-the-art architectures.

Curate scenarios, develop system introspection capabilities, and establish frameworks for understanding model behavior and performance at scale.

MS or PhD in computer vision, ML, robotics, or related quantitative fields.

7+ years of professional experience with computer vision, ML, or applied science.

Strong hands-on experience with foundation models, world models, generative AI, multimodal transformers, diffusion, VLAs, or large end-to-end behavior models for robotics or autonomy.

Expertise in PyTorch (preferred), JAX, or TensorFlow; strong Python and C++ skills.

Strong understanding of temporal/sequential modeling, probabilistic modeling, reinforcement learning, Bayesian inference, state-space models, and uncertainty quantification.

Strong understanding of 3D perception, multi-view geometry and sensor fusion.

Hands-on experience with large-scale distributed training, ML workflows (data curation, training, evaluation, deployment), and inference optimization.

Knowledge of debugging, profiling and deploying deep neural networks with NVIDIA tooling (CUDA, Nsight, TensorRT) and ONNX.

Experience with simulation platforms (e.g., CARLA, Applied Intuition, Nvidia DriveSim, etc.), their internal principles and their integration into autonomous system workflows.

Proven track record of leading large, multi-person technical projects and influencing technical direction across organizations, as well as strong communication skills.

Publications at top-tier venues (e.g. NeurIPS, CVPR, ICML, ICRA, RSS).

Experience with closed-loop simulation validation, scenario generation, rare-event or counterfactual testing.

Experience with multi-agent simulation or high-fidelity 3D environments and game engines (e.g Unreal).

Experience with 3D generation or reconstruction (e.g., Gaussian Splatting, NeRFs).

Prior experience in fast-paced R&D environments bridging research and production.