Lerrel Pinto, an assistant professor at New York University’s Courant Institute of Mathematical Sciences, has been named a recipient of a 2023 Packard Fellowship for Science and Engineering. Pinto was one of 20 scientific researchers to receive a fellowship.
Pinto, named to MIT Technology Review’s “35 Innovators under 35” list earlier this year, is seeking ways to train robots that can perform a variety of tasks—or, as the magazine put it, create robots that “do a lot more than vacuum.”
“Think of a seemingly easy task of opening an unlocked door—something humans can do effortlessly by adapting their approach to a variety of door designs, handle mechanisms, and opening mechanics,” explains Pinto. “But developing an algorithm that gives a robot the ability to open not just one type of door, but several types of doors, presents a formidable challenge.”
Pinto, who has been at NYU since 2020, sees current limitations as a trade-off between dexterity and generality.
“Traditional robotics has produced technologies that can perform highly dexterous behaviors and have been successfully applied to a variety of intricate tasks,” explains Pinto. “However, they rely on strong assumptions, such as knowing precise physical information about the robot’s environment beforehand. These assumptions are limiting when robots are tasked to operate in new scenarios.”
Conversely, other advances rely on machine learning, under which tools are applicable across a large variety of environments without much human intervention.
“But, the generality of behavior comes at the cost of needing large amounts of robotic data,” Pinto observes. “This, in turn, limits their use to less dexterous behaviors, such as object picking, where collecting large amounts of data is relatively easy.”
“Consequently, we have robots that can either produce highly dexterous behavior on narrow variations of a task or can perform wide variations with simple robotic behaviors,” he adds. “But we lack approaches that can do both—produce dexterous behavior and be useful in a wide range of scenarios.”
Under the Packard Research Fellowship—$875,000 over five years—Pinto aims to address this gap in robotics through a trio of approaches: building robotic systems that can continuously self-collect large quantities of data for dexterous tasks, creating algorithms that capture real-world phenomena in order to optimize generality, and developing human-robot interfaces that will allow robots to reflect human preferences based on our behaviors.
Pinto’s preliminary work may be seen on his website.
Pinto, who has also received an Amazon Research Award, holds master’s and doctoral degrees in robotics from Carnegie Mellon University and a bachelor’s degree in mechanical engineering from the Indian Institute of Technology Guwahati. Prior to coming to NYU, Pinto was a postdoctoral researcher at the University of California, Berkeley.
Editor’s Note:
The 2023 Packard Fellows, which are supported by the David and Lucile Packard Foundation's Fellowships for Science and Engineering program, include faculty members at universities across the United States. These fellows are addressing some of the most important research questions today in science and engineering on topics such as Alzheimer's disease, stem cell function, evolutionary theory, and the relationship between disease and environment.
