Member of Technical Staff — Pretraining Infra

Nuance Labs is building photorealistic, real-time AI avatars with emotional intelligence: a full-duplex audiovisual system that can listen, speak, react, interrupt, and respond like a real person.

We're a Series A company ($60M raised) backed by Lightspeed, Accel, South Park Commons, NVentures, and Define Ventures, with PhDs from MIT, UW, Oxford, CMU, and Johns Hopkins, and industry experience from Apple, Meta, Amazon AGI, and Discord. The team is small, the work is real, and the problems are unsolved.

Most conversational AI avatars today are hacks — a face slapped on a speech-to-speech pipeline, stuck in the uncanny valley: emotionless, mechanical, one-turn-at-a-time. Current systems take 2–5 seconds to respond; natural conversation requires sub-500ms. That's a 10x improvement, and it demands rethinking the entire stack.

That rethinking starts with full-duplex: an AI that listens and speaks simultaneously, perceives emotion in real time, and responds with a face that actually reflects it. It's an extremely hard problem, and we're developing foundation models designed for it from the ground up.

We're looking for a deeply technical MTS to own distributed training infrastructure for large-scale omni model pretraining.

This role sits at the intersection of research, systems, and GPU-scale execution — building the training stack from 0→1 and scaling it: distributed execution, parallelism, GPU communication, data loading, checkpointing, observability, and debugging.

Our models are omni from the ground up (audio, video, language, real-time full-duplex), which introduces systems challenges beyond standard LLM training: multimodal synchronization, long temporal context, variable sequence lengths, and tight memory/throughput constraints.

High ownership. Direct impact on what models we can train, how fast research can iterate, and how reliably we scale.

• Own the distributed training stack for omni model pretraining, from 0→1 system design to 1→10 scaling across large GPU clusters.
• Build and operate the core training runtime: job orchestration, distributed execution, checkpointing, recovery, monitoring, and debugging for long-running training jobs.
• Optimize large-scale training performance across parallelism strategy, GPU communication, memory usage, data throughput, MFU, step time, and end-to-end training efficiency.
• Build infrastructure for omni training workloads: high-throughput audio/video/text data loading, temporal alignment, variable sequence handling, multimodal synchronization, and memory-efficient training.
• Evolve the platform as model architectures, training recipes, data mixtures, sequence lengths, hardware constraints, and research directions change.

• Hands-on experience running large-scale distributed training jobs across large GPU clusters; experience at hundreds of GPUs minimum, 1,000+ GPUs a strong plus.
• Deep understanding of distributed training mechanics: data/tensor/pipeline/sequence parallelism, gradient communication, collectives, mixed precision, activation checkpointing, optimizer state, memory pressure, and framework-level tradeoffs.
• Strong understanding of GPU communication and performance debugging: NCCL, all-reduce/all-gather/reduce-scatter, communication-computation overlap, topology, synchronization, stragglers, low MFU, OOMs, checkpoint bottlenecks, and data starvation.
• Practical experience with at least one major large-scale training stack such as Megatron, PyTorch FSDP, DeepSpeed, or equivalent internal infrastructure.
• Understanding of omni or multimodal training challenges, especially audio/video/language data, long temporal context, variable sequence lengths, modality-specific bottlenecks, and high-throughput dataloading.
• Strong software engineering fundamentals, curiosity, and adaptability to new model architectures, training frameworks, hardware constraints, and research ideas.

• Prior 0→1 experience building large-scale training infrastructure or deeply modifying core training frameworks, runtimes, checkpointing, or debugging systems.
• Experience training large omni or multimodal models involving audio, video, text, or long-context temporal data.
• Experience with adjacent infrastructure areas such as RL/post-training, data infrastructure, synthetic data generation, evaluation, or serving.
• Publications or substantial open-source contributions in ML systems, distributed systems, HPC, GPU performance, or training infrastructure.

Nuance Labs is an equal opportunity employer. We believe diverse teams build better AI.