Geometry-Aware Policy (Ours)
Dexterous manipulation of arbitrary objects, a fundamental daily task for humans, has been a grand challenge for autonomous robotic systems. Although data-driven approaches using reinforcement learning can develop specialist policies that discover behaviors to control a single object, they often exhibit poor generalization to unseen ones. In this work, we show that policies learned by existing reinforcement learning algorithms can in fact be generalist when combined with multi-task learning and a well-chosen object representation. We show that a single generalist policy can perform in-hand manipulation of over 100 geometrically-diverse real-world objects and generalize to new objects with unseen shape or size. Interestingly, we find that multi-task learning with object point cloud representations not only generalizes better but even outperforms the single-object specialist policies on both training as well as held-out test objects.
We introduce simple extensions to existing RL algorithms to train a dexterous manipulation policy that is surprisingly robust to over 100 geometrically-diverse objects and even outperforms single-task oracles on unseen objects when evaluated in a zero-shot manner. We first train an object representation encoder using object point clouds (left). Then we perform multi-task RL training on a large number of objects leveraging the encoded object representation (right).
Along with the paper, we also release a suite of dexterous manipulation tasks built upon OpenAI Gym and MuJoCo. Instead of using simple-geometry objects (e.g. cube, egg, etc), the tasks feature a diverse set of 114 real-world objects collected from the YCB Dataset and the ContactDB Dataset.
A single Geometry-Aware Policy can simultaneously manipulate a large number of complex-shaped objects, with performance significantly better than its vanilla counterpart and even single-task oracles. Notice how the policy knows to utilize all fingers and stretch the thumb when it's blocking the object.
The policy can also manipulate simple-geometry objects with ease. Note that the gains are not significant because these objects often do not require fine-grained geometrical reasoning and can be rotated with similar strategies.
The policy can also generalize to unseen objects with challenging geometries in a zero-shot manner. These objects often have unfamiliar shapes and uncommon shape-ratios compared to those in the training set. Notice that it can even outperform oracles on some objects.
When evaluated zero-shot on unseen simple objects, while the Geometry-Aware Policy has similar high success rates compared to the vanilla policy, it can often achieve the goal much faster. The speed is similar to the oracle policies that are trained on these objects individually.
However, the policy still has trouble generalizing to objects with very uneven/fine-grained geometries, indicating room for future improvement. Note that these objects are proven to be difficult even when trained individually for oracle policies.
@article{huang2021geometry,
title={Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning},
author={Huang, Wenlong and Mordatch, Igor and Abbeel, Pieter and Pathak, Deepak},
journal={arXiv preprint arXiv:2111.03062},
year={2021}
}