Wenlong Huang

I am a third-year Ph.D. student in Computer Science at Stanford, advised by Fei-Fei Li as part of the Stanford Vision and Learning Lab (SVL).

I received my B.A. in Computer Science from UC Berkeley (2018 - 2021), advised by Deepak Pathak, Igor Mordatch, and Pieter Abbeel. I have spent time at Seattle Robotics Lab at NVIDIA (2024) and Robotics team at Google Brain (2022). Previously, I also worked with Zhuowen Tu at UC San Diego (2018).

Email (wenlongh [AT] stanford.edu)  /  Google Scholar  /  Twitter  /  GitHub

• [May 2025] Invited Talk at ICRA 2025 Workshop on Foundation Models and Neuro-Symbolic AI for Robotics.
• [May 2025] UAD is nominated as a Best Paper Finalist at ICRA 2025.
• [April 2025] Grateful to be selected as a finalist for the Citadel GQS Fellowship (three in CS/EE).
• [Feb 2025] Co-organizing Tutorial on Foundation Models Meet Embodied Agents at ICCV 2025, NAACL 2025, and AAAI 2025.
• [Feb 2025] Invited Talk at Amazon on "Foundation Models for Spatial Task Representation in Robotics".
• [Dec 2024] Grateful to be selected as a finalist for the NVIDIA Graduate Fellowship.
• [Dec 2024] Co-organizing Workshop on Foundation Models for Embodied Agents (FMEA) at CVPR 2025.
• [Dec 2024] Co-organizing Workshop on Planning in the Era of LLMs (LM4Plan) at AAAI 2025.
• [Nov 2024] ReKep was presented at CoRL 2024 and won the Best Paper Award at the 2024 CoRL LEAP Workshop.
• [Oct 2023] Invited Talks at Stanford IPRL, Tsinghua, TRI on "Foundation Models for Open-World Generalization in Robotics".
• [Aug 2023] VoxPoser is accepted at CoRL 2023 for an oral presentation.
• [Jun 2023] Code as Policies received Outstanding Robot Learning Paper Award at ICRA 2023.
• [Nov 2022] Code as Policies is covered by Google AI Blog and TechCrunch.
• [Oct 2022] Invited Talks at Columbia CAIR, Google on "Language as Generalization Interface for Robotics".
• [Oct 2022] Inner Monologue is covered by Two Minute Papers.
• [Sep 2022] Started at Stanford as a PhD student, generously supported by Stanford SoE Fellowship.
• [Feb 2022] Invited Talks at Google, FAIR, Sea AI Lab, ByteDance AI Lab on our LLMs as Zero-Shot Planners project.
• [Dec 2021] Invited Talk at Intel AI Lab on "Generalization across Objects and Morphologies in Robot Learning".

The goal of my research is to endow robots with broad generalization capabilities for open-world manipulation tasks, especially in household environments. Towards this goal, I am interested in 1) developing structured representations that leverage foundation models and Internet-scale data, and 2) developing algorithms that exhibit broadly generalizable behaviors.

Large vision models and vision-language models can generate keypoint-based constraints, which can be optimized to achieve multi-stage, in-the-wild, bimanual, and reactive behaviors, without task-specific training or environment models.

Fine-grained task-conditioned visual affordance can be distilled from off-the-shelf foundation models, enabling diverse generalization properties in downstream policy learning.

Vision-Language Models can serve as human proxies specify diverse task objectives by writing keypoint-based reward functions, where they can be autonomously and iteratively refined based on environment feedback.

Large language models and visual-language models can be used to directly label affordances and constraints in the 3D perceptual space. Combined with motion planning, this enables robots to perform diverse everyday manipulation tasks in a zero-shot manner.

Language models can digest real-world sensor modalities (e.g., images) to be embodied in the physical world. The largest 562B model is a generalist agent across language, vision, and task planning.

Large language models can be grounded in embodied environments by using continuous probabilities to guide their token decoding, where the guidance is provided by a set of grounded models, such as affordance, safety, and preference functions.

Using hierarchical code generation, language models can write robot policy code that exhibits spatial-geometric reasoning given abstract natural language instructions.

Provided with textual embodied feedback, language models can articulate a grounded "thought process", solving challenging long-horizon robotic tasks, even under disturbances.

Large language models (e.g. GPT-3, Codex) contain rich actionable knowledge that can be used to plan actions for embodied agents, even without additional training.

With appropriate object representation, a multi-task RL policy can control an anthropomorphic hand to manipulate 100+ diverse objects and achieve SOTA performance on unseen ones.

Expressing robots as collections of modular components that share a control policy can lead to zero-shot generalization across diverse unseen robot morphologies.

Built upon Generative via Discriminative Learning and Introspective Learning frameworks, a single neural network can simultaneously perform classification and generation of 3D volumetric shapes.

Template from Jon Barron's website