School of Mechanical and Materials Engineering Seminar Series, “Towards Building Intelligent Systems for Robots to Interact With the World”

About the event

Presented by: Dr. Lin Shao, PhD, Stanford University

Abstract

Building intelligent systems for robots to interact with the world is a challenging problem. In this talk, I will present our efforts ranging from learning primitive skills to learning manipulation concepts for robots to interact with the world. Grasping and hanging are primitive manipulation skills. Choosing the right robot action representation is of great importance for mastering these skills. I will present work that leverages contact points as an abstraction that can be re-used by a diverse set of robot hands. For complex manipulation skills such as tool manipulation and robotic assembly, I will present an approach that allows a robot to autonomously modify the environment and discover how to ease manipulation skill learning. Specifically, we provide the robot with fixtures that can be freely placed within the environment. These fixtures provide hard constraints of the robot actions and funnel uncertainty of the environment to speed up the learning process. And third, I will present our work that endows robots to acquire various manipulation concepts that act as mental representations of verbs in natural language instructions. We propose to use a learning-from-demonstration approach to learn manipulation actions from large-scale video datasets annotated with natural language instructions. In simulation experiments, we show that the policy learned in the proposed way can perform a large percentage of 78 different manipulation tasks. We show that our policy can also generalize to the novel but similar instructions.

Biography

Lin Shao received his Ph.D. in the Institute for Computational and Mathematical Engineering at Stanford University, advised by Professor Jeannette Bohg. His research lies at the intersection of robotics, artificial intelligence, and cognitive science. He is specifically interested in developing methods to endow robots with the abilities of perception, manipulation, conceptualization, and generalization. He received his M.S. in Computational and Applied Mathematics from Stanford University and his B.S. in Geochemistry from Nanjing University.