Co-op Researcher – Multimodal & Voice Interaction (HCI)

The Human-Machine Interaction Lab unites global talents to redefine the relationship between humans and technology. Focused on innovation and user-centered design, the lab strives to advance human-computer interaction research. Our team includes researchers, engineers, and designers collaborating across disciplines to develop novel interactive systems, sensing technologies, wearable and IoT systems, human factors, computer vision, and multimodal interfaces. Through high-impact products and cutting-edge research, we aim to enhance user experiences and interactions with technology.

• Prototype Implementation: Build simple prototypes (applications or small services) that demonstrate voice-first multimodal interactions (e.g., voice + touch, voice + gesture) on real devices. You’ll work closely with the team to turn ideas into working demos.

• Model Integration & Tuning: Use existing speech tools (e.g., Whisper, Vosk, or similar) to integrate features like speech recognition or text-to-speech. Perform light tuning and testing to improve responsiveness and usability.

• Multimodal Fusion Logic: Work with inputs such as audio, camera, and touch to implement straightforward interaction logic (e.g., combining voice commands with simple gestures or confirmations).

• Collaborative Feature Exploration: Collaborate with researchers and designers to prototype new interaction ideas (e.g., conversational interfaces, voice commands). Help translate concepts into testable demos.

• Toolkit & Framework Usage: Use common tools and frameworks (e.g., PyTorch, TensorFlow). Learn how different tools are used for on-device vs. cloud-based solutions.

• Code Quality & Experimentation: Write clean and readable code. Run small tests (e.g., latency or usability checks) and document what you learn.

The total target annual compensation for this position ranges from $58,000 to $104,000 depending on education, experience, and demonstrated expertise.