Our paper “Aligning generalization between humans and machines” is now published at Nature Machine Intelligence! This paper has been co-authored with 25 experts in AI and cognitive science and is a product of the Dagstuhl seminar on “Generalization by People and Machines” in May 2024. We thank the many co-authors and the seminar’s organizers for this joint work.
In summary, the paper combines insights across disciplines to identify key commonalities and differences across three dimensions: notions of, methods for, and evaluation of generalization. Some key findings of the paper are:
- Generalization is a multi-dimensional concept.
- Humans and typical (statistical) machines generalize in complementary ways.
- Collaborative and explainable mechanisms are key to achieving alignment in human-AI teaming.
- Given the promise of foundation models, further research is necessary to provide guarantees and theoretical support for their generalization.
- Weaknesses of standard (statistical) AI techniques can be mitigated by other AI families: case-to-case and analytical methods, highlighting the promise of hybrid, neurosymbolic AI.
- The relationship between continual learning and generalization is complex but promising to explore.
- Evaluating the many facets of generalization has improved, yet, many gaps remain.
For more information, read the full paper. The abstract is provided below:
Recent advances in artificial intelligence (AI)—including generative approaches—have resulted in technology that can support humans in scientific discovery and forming decisions, but may also disrupt democracies and target individuals. The responsible use of AI and its participation in human–AI teams increasingly shows the need for AI alignment, that is, to make AI systems act according to our preferences. A crucial yet often overlooked aspect of these interactions is the different ways in which humans and machines generalize. In cognitive science, human generalization commonly involves abstraction and concept learning. By contrast, AI generalization encompasses out-of-domain generalization in machine learning, rule-based reasoning in symbolic AI, and abstraction in neurosymbolic AI. Here we combine insights from AI and cognitive science to identify key commonalities and differences across three dimensions: notions of, methods for, and evaluation of generalization. We map the different conceptualizations of generalization in AI and cognitive science along these three dimensions and consider their role for alignment in human–AI teaming. This results in interdisciplinary challenges across AI and cognitive science that must be tackled to support effective and cognitively supported alignment in human–AI teaming scenarios.