Can AI Grasp Related Concepts After Learning Only One?

December 11, 2023

This technique, Meta-learning for Compositionality (MLC), outperforms existing approaches and is on par with, and in some cases better than, human performance. MLC centers on training neural networks—the engines driving ChatGPT and related technologies for speech recognition and natural language processing—to become better at compositional generalization through practice. Developers of existing systems, including large language models, have hoped that compositional generalization will emerge from standard training methods, or have developed special-purpose architectures in order to achieve these abilities. MLC, in contrast, shows how explicitly practicing these skills allow these systems to unlock new powers, the authors note. “We have shown, for the first time, that a generic neural network can mimic or exceed human systematic generalization in a head-to-head comparison.”

Humans have the ability to learn a new concept and then immediately use it to understand related uses of that concept—once children know how to “skip,” they understand what it means to “skip twice around the room” or “skip with your hands up.” 

But are machines capable of this type of thinking? In the late 1980s, Jerry Fodor and Zenon Pylyshyn, philosophers and cognitive scientists, posited that artificial neural networks—the engines that drive artificial intelligence and machine learning— are not capable of making these connections, known as “compositional generalizations.” However, in the decades since, scientists have been developing ways to instill this capacity in neural networks and related technologies, but with mixed success, thereby keeping alive this decades-old debate.

Researchers at New York University and Spain’s Pompeu Fabra University have now developed a technique—reported in the journal Nature—that advances the ability of these tools, such as ChatGPT, to make compositional generalizations. This technique, Meta-learning for Compositionality (MLC), outperforms existing approaches and is on par with, and in some cases better than, human performance. MLC centers on training neural networks—the engines driving ChatGPT and related technologies for speech recognition and natural language processing—to become better at compositional generalization through practice.

Developers of existing systems, including large language models, have hoped that compositional generalization will emerge from standard training methods, or have developed special-purpose architectures in order to achieve these abilities. MLC, in contrast, shows how explicitly practicing these skills allow these systems to unlock new powers, the authors note.

“For 35 years, researchers in cognitive science, artificial intelligence, linguistics, and philosophy have been debating whether neural networks can achieve human-like systematic generalization,” says Brenden Lake, an assistant professor in NYU’s Center for Data Science and Department of Psychology and one of the authors of the paper. “We have shown, for the first time, that a generic neural network can mimic or exceed human systematic generalization in a head-to-head comparison.”

The source of this news is from New York University

Popular in Research

1

Feb 12, 2024

DTU Scientist Co-Authors International Paper on COVID-19

2

5 days ago

Two Simple Words May Help Decide Immigration Case Before the High Court

3

Feb 5, 2024

Evaluating the Truthfulness of Fake News Through Online Searches Increases the Chances of Believing Misinformation

4

Feb 5, 2024

The Opportunities and Pitfalls of Place-Based Philanthropy

5

Feb 9, 2024

Australian researchers discover what turned Earth into a snowball 700m years ago

Sotomayor and Barrett stress Supreme Court camaraderie

14 hours ago

Supreme Court agrees to weigh whether Trump can be kicked off ballot in Colorado

3 days ago

Japan's Nikkei crosses 39,000 as robust earnings, investor-friendly measures drive risk-on sentiment

1 day ago

Trump received millions of dollars from foreign governments while president, House Democrats allege

4 days ago

Climate action, here and now

14 hours ago

Institute Professor Emeritus Robert Solow, pathbreaking economist, dies at age 99

3 days ago