Wenlong Huang

I am a second-year Ph.D. student in Computer Science at Stanford, advised by Fei-Fei Li as part of the Stanford Vision and Learning Lab (SVL).

I received my B.A. in Computer Science from UC Berkeley (2018 - 2021), advised by Deepak Pathak, Igor Mordatch, and Pieter Abbeel. After graduation, I interned in the Robotics team at Google Brain (2022). Previously, I also worked with Zhuowen Tu at UC San Diego (2018).

Email (wenlongh [AT] stanford.edu)  /  Google Scholar  /  Twitter  /  GitHub

• [Aug 2023] VoxPoser is accepted at CoRL 2023 for an oral presentation!
• [Jun 2023] Code as Policies received Outstanding Robot Learning Paper Award at ICRA 2023!
• [Nov 2022] Code as Policies is covered by Google AI Blog and TechCrunch.
• [Oct 2022] Invited Talks @ CAIR, Google on language as generalization interface for robotics.
• [Oct 2022] Inner Monologue is covered by Two Minute Papers.
• [Sep 2022] Started at Stanford as a PhD student, generously supported by Stanford SoE Fellowship.
• [Apr 2022] Started my internship at Google Brain as a student researcher, working with the amazing Robotics team.
• [Feb 2022] Interviewed by Yannic Kilcher for our language planner project. Check it out here!
• [Feb 2022] Invited Talks @ Google, FAIR, Sea AI Lab, ByteDance AI Lab on our language planner project.
• [Dec 2021] Invited Talk @ Intel AI Lab on generalization across objects and morphologies in robot learning.

The goal of my research is to endow robots with broad generalization capabilities for open-world manipulation tasks, especially in household environments. Towards this goal, I am interested in 1) developing abstractions that best leverage Internet-scale data or models trained on them, and 2) developing motor skills that exhibit broadly generalizable behaviors.

Large language models and visual-language models can be used to directly label affordances and constraints in the 3D perceptual space. Combined with motion planning, we can enable robots to perform diverse everyday manipulation tasks in a zero-shot manner.

Language models can digest real-world sensor modalities (e.g., images) to be embodied in the physical world. The largest model with 562B parameters is a generalist agent across language, vision, and robot planning.

Large language models can be grounded in embodied environments by using continuous probabilities to guide their token decoding, where the guidance is provided by a set of grounded models, such as affordance, safety, and preference functions.

Using hierarchical code generation, large language models can write robot policy code that exhibits spatial-geometric reasoning when given abstract natural language instructions, without any additional training.

Using various sources to provide textual embodied feedback, frozen large language models can articulate a grounded "thought process" for robots, solving many challenging long-horizon robotics tasks, even under adversarial perturbation.

Large language models (e.g. GPT-3, Codex) contain rich actionable knowledge that can be used to plan actions for embodied agents, even without additional training.

With appropriate object representation, a multi-task reinforcement learning policy can control an anthropomorphic hand to manipulate 100+ diverse objects and achieve SOTA performance on unseen ones.

Expressing robots as collections of modular components that share a control policy can lead to zero-shot generalization across diverse unseen robot morphologies.

Built upon prior Generative via Discriminative Learning and Introspective Learning frameworks, a single neural network can simultaneously perform classification and generation of 3D volumetric shapes.

Reviewer for CoRL, IROS, ICML, NeurIPS, ICLR, RA-L, and Nature Communications.

Template from Jon Barron's website