Publications

People are able to learn clever cognitive strategies through trial and error from small amounts of experience. This is facilitated by people's ability to reflect on their own thinking which is known as metacognition. To examine the effects of deliberate systematic metacognitive reflection on how people learn how to plan, the experimental group was guided to systematically reflect on their decision-making process after every third decision. We found that participants assisted by reflection prompts learned to plan better faster. Moreover, we found that reflection led to immediate improvements in the participants' planning strategies. Our preliminary results do suggest that deliberate metacognitive reflection can help people discover clever cognitive strategies from very small amounts of experience. Understanding the role of reflection in human learning is a promising approach for making reinforcement learning more sample efficient in both humans and machines.

Highly influential "dual-process" accounts of human cognition postulate the coexistence of a slow accurate system with a fast error-prone system. But why would there be just two systems rather than, say, one or 93? Here, we argue that a dual-process architecture might reflect a rational tradeoff between the cognitive flexibility afforded by multiple systems and the time and effort required to choose between them. We investigate what the optimal set and number of cognitive systems would be depending on the structure of the environment. We find that the optimal number of systems depends on the variability of the environment and the difficulty of deciding when which system should be used. Furthermore, we find that there is a plausible range of conditions under which it is optimal to be equipped with a fast system that performs no deliberation (``System 1'') and a slow system that achieves a higher expected accuracy through deliberation (``System 2''). Our findings thereby suggest a rational reinterpretation of dual-process theories.

People’s decisions about how to allocate their limited computational resources are essential to human intelligence. An important component of this metacognitive ability is deciding whether to continue thinking about what to do and move on to the next decision. Here, we show that people acquire this ability through learning and reverse-engineer the underlying learning mechanisms. Using a process-tracing paradigm that externalises human planning, we find that people quickly adapt how much planning they perform to the cost and benefit of planning. To discover the underlying metacognitive learning mechanisms we augmented a set of reinforcement learning models with metacognitive features and performed Bayesian model selection. Our results suggest that the metacognitive ability to adjust the amount of planning might be learned through a policy-gradient mechanism that is guided by metacognitive pseudo-rewards that communicate the value of planning.

Goal-directed behaviour is a deeply important part of human psychology. People constantly set goals for themselves and pursue them in many domains of life. In this paper, we develop computational models that characterize how humans pursue goals in a complex dynamic environment and test how well they describe human behaviour in an experiment. Our models are motivated by the principle of resource rationality and draw upon psychological insights about people's limited attention and planning capacities. We found that human goal pursuit is qualitatively different and substantially less efficient than optimal goal pursuit. Models of goal pursuit based on the principle of resource rationality captured human behavior better than both a model of optimal goal pursuit and heuristics that are not resource-rational. We conclude that human goal pursuit is jointly shaped by its function, the structure of the environment, and cognitive costs and constraints on human planning and attention. Our findings are an important step toward understanding humans goal pursuit, as cognitive limitations play a crucial role in shaping people's goal-directed behaviour.

People often fall victim to decision-making biases, e.g. short-sightedness, that lead to unfavorable outcomes in their lives. It is possible to overcome these biases by teaching people better decision-making strategies. Finding effective interventions is an open problem, with a key challenge being the lack of transfer to the real world. Here, we tested a new approach to improving human decision-making that leverages Artificial Intelligence to discover procedural descriptions of effective planning strategies. Our benchmark problem regarded improving far-sightedness. We found our intervention elicits transfer to a similar task in a different domain, but its effects in more naturalistic financial decisions were not statistically significant. Even though the tested intervention is on par with conventional approaches, which also struggle in far-transfer, further improvements are required to help people make better decisions in real life. We conclude that future work should focus on training decision-making in more naturalistic scenarios.

The discovery of decision strategies is an essential part of creating effective cognitive tutors that teach planning and decision-making skills to humans. In the context of bounded rationality, this requires weighing the benefits of different planning operations compared to their computational costs. For small decision problems, it has already been shown that near-optimal decision strategies can be discovered automatically and that the discovered strategies can be taught to humans to increase their performance. Unfortunately, these near-optimal strategy discovery algorithms have not been able to scale well to larger problems due to their computational complexity. In this talk, we will present recent work at the Rationality Enhancement Group to overcome the computational bottleneck of existing strategy discovery algorithms. Our approach makes use of the hierarchical structure of human behavior by decomposing sequential decision problems into two sub-problems: setting a goal and planning how to achieve it. An additional metacontroller component is introduced to switch the current goal when it becomes beneficial. The hierarchical decomposition enables us to discover near-optimal strategies for human planning in larger and more complex tasks than previously possible. We then show in online experiments that teaching the discovered strategies to humans improves their performance in complex sequential decision-making tasks.

Globally 258 million children and youth do not have access to school (Unesco, 2019), while 600 million receive ineffective education (Unesco, 2017). Solve Education! (SE!) is a non-profit organization committed to enable these young people to empower themselves through education, and currently operates in over 7 countries. Their team includes educationists, technologists, and business executives, who work together with governments and local communities to reach young people with disadvantaged backgrounds. Solve Education!’s main mobile application “The Dawn of Civilisation” (DoC), is an open platform that can deliver different learning content, with the focus on English literacy. It is designed to support lower end devices, as well as offline learning. At the Rationality Enhancement Group, we are laying the scientific foundation for helping people do more good in better ways. We combine methods from computational cognitive science, psychology, human-computer interaction, and artificial intelligence for the development of practical tools, strategies, and interventions that support people in their personal growth. In our collaboration with SE!, we aim at learning from and contributing to real-world challenges by applying our research to enhance SE!’s learning platform. We are currently working on two projects. The first project’s goal is to develop a principled approach to incentivize efficient self-directed learning with digital educational resources and to evaluate its effectiveness regarding learners’ behaviors and success in cooperation with SE!. Specifically, SE!’s DoC serves as the digital educational resource and allows to evaluate the approach with very high ecological validity. The planned intervention is based on the concept of optimal brain points developed by Xu, Wirzberger & Lieder (2019). The core idea is to incentivize effort and smart study choices rather than performance and to do so in a way that learners cannot exploit shortcuts to accumulate game points without also moving closer to their actual learning goals. If successful, SE! can build upon the intervention to further enhance the benefits their users draw from DoC. The second project is based on hierarchical goal setting and consists of a digital assistant that helps users set real-world goals and make progress towards them by reaching milestones with DoC. In this talk, in addition to introducing our work together with SE, we will highlight the mutual benefits of the collaboration between scientists and socially impactful organizations.

Human decision-making is sometimes systematically biased toward suboptimal decisions. For example, people often make short-sighted choices because they don't give enough weight to the long-term consequences of their actions. Previous studies showed that it is possible to overcome such biases by teaching people a more rational decision strategy through instruction, demonstrations, or practice with feedback. The benefits of these approaches tend to be limited to situations that are very similar to those used during the training. One way to overcome this limitation is to create general tools and strategies that people can use to improve their decision-making in any situation. Here we propose one such approach, namely directing people to systematically reflect on how they make their decisions. In systematic reflection, past experience is re-evaluated with the intention to learn. In this study, we investigate how reflection affects how people learn to plan and whether reflective learning can help people to discover more far-sighted planning strategies. In our experiment participants solve a series of 30 planning problems where the immediate rewards are smaller and therefore less important than long-term rewards. Building on Wolfbauer et al. (2020), the experimental group is guided by four reflection prompts asking the participant to describe their planning strategy, the strategy's performance, and his or her emotional response, insights, and intention to change their strategy. The control group practices planning without reflection prompts. Our pilot data suggest that systematic reflection helps people to more rapidly discover adaptive planning strategies. Our findings suggest that reflection is useful not only for helping people learn what to do in a specific situation but also for helping people learn how to think about what to do. In future work, we will compare the effects of different types of reflection on the subsequent changes in people's decision strategies. Developing apps that prompt people to reflect on their decisions may be a promising approach to accelerating cognitive growth and promoting lifelong learning.

Beyond merely reacting to their environment and impulses, people have the remarkable capacity to proactively set and pursue their own goals. But the extent to which they leverage this capacity varies widely across people and situations. The goal of this article is to make the mechanisms and variability of proactivity more amenable to rigorous experiments and computational modeling. We proceed in three steps. First, we develop and validate a mathematically precise behavioral measure of proactivity and reactivity that can be applied across a wide range of experimental paradigms. Second, we propose a formal definition of proactivity and reactivity, and develop a computational model of proactivity in the AX Continuous Performance Task (AX-CPT). Third, we develop and test a computational-level theory of meta-control over proactivity in the AX-CPT that identifies three distinct meta-decision-making problems: intention setting, resolving response conflict between intentions and automaticity, and deciding whether to recall context and intentions into working memory. People's response frequencies in the AX-CPT were remarkably well captured by a mixture between the predictions of our models of proactive and reactive control. Empirical data from an experiment varying the incentives and contextual load of an AX-CPT confirmed the predictions of our meta-control model of individual differences in proactivity. Our results suggest that proactivity can be understood in terms of computational models of meta-control. Our model makes additional empirically testable predictions. Future work will extend our models from proactive control in the AX-CPT to proactive goal creation and goal pursuit in the real world.

The translation of abstract, long-term goals, such as “make a contribution to the field of motivation science,” into short-term, actionable intentions is inherently difficult. Hierarchical goal-setting, a goal-setting strategy in which people construct a hierarchy of increasingly more concrete and proximal subgoals is a promising way to support this process. We designed a goal-setting chatbot that helps people craft action hierarchies for achieving their life goals. We conducted a large online field experiment with two follow-up surveys at one week and one month after the intervention to evaluate the effects of a brief hierarchical planning session with our chatbot on goal pursuit. Although there were no main effects of hierarchical planning on goal-related outcomes, exploratory analyses indicated that hierarchical goal-setting enabled people to make more progress towards goals that appeared less actionable. This suggests that supporting hierarchical goal-setting with chatbots is a promising approach to helping people who don’t know how to pursue their goals.

We tested two reflection techniques derived from Acceptance Commitment Therapy for helping people set life goals that are self-determined, communal, and future-minded. Participants were assigned randomly to control, Eulogy, or the Valued Living Questionnaire (VLQ) conditions. Eulogy participants envisioned what they wanted people to say about them at their funeral. In VLQ, participants rated the importance of life domains and how consistent their behavior has recently been with the importance assigned to each domain. Participants then set a life goal, rated it for self-determination, and indicated its time horizon and life domain. Despite only requiring internal reflection, Eulogy was particularly effective for generating self-determined goals that were interpersonal and future-minded. The Eulogy exercise may be a useful and important building block for inspiring the setting and effective pursuit of goals that are simultaneously self-determined, communal, and future-minded. Future research will examine its efficacy in changing experienced well-being and enacted well-doing.

Well-doing, broadly construed, encompasses acting and thinking in ways that contribute to humanity’s flourishing in the long run. This often takes the form of setting a prosocial goal and pursuing it over an extended period of time. To set and pursue goals in a way that is extremely beneficial for humanity (effective well-doing), people often have to employ critical thinking and far-sighted, rational decision-making in the service of the greater good. To promote effective well-doing, we need to better understand its determinants and psychological mechanisms, as well as the barriers to effective well-doing and how they can be overcome. In this article, we introduce a taxonomy of different forms of well-doing and introduce a conceptual model of the cognitive mechanisms of effective well-doing. We view effective well-doing as the upper end of a moral continuum whose lower half comprises behaviors that are harmful to humanity (ill-doing), and we argue that the capacity for effective well-doing has to be developed through personal growth (e.g., learning how to pursue goals effectively). Research on these phenomena has so far been scattered across numerous disconnected literatures from multiple disciplines. To bring these communities together, we call for the establishment of a transdisciplinary research field focussed on understanding and promoting effective well-doing and personal growth as well as understanding and reducing ill-doing. We define this research field in terms of its goals and questions. We review what is already known about these questions in different disciplines and argue that laying the scientific foundation for promoting effective well-doing is one of the most valuable contributions that the behavioral sciences can make in the 21st century.

When making decisions, people often overlook critical information or are overly swayed by irrelevant information. A common approach to mitigate these biases is to provide decisionmakers, especially professionals such as medical doctors, with decision aids, such as decision trees and flowcharts. Designing effective decision aids is a difficult problem. We propose that recently developed reinforcement learning methods for discovering clever heuristics for good decision-making can be partially leveraged to assist human experts in this design process. One of the biggest remaining obstacles to leveraging the aforementioned methods for improving human decision-making is that the policies they learn are opaque to people. To solve this problem, we introduce AI-Interpret: a general method for transforming idiosyncratic policies into simple and interpretable descriptions. Our algorithm combines recent advances in imitation learning and program induction with a new clustering method for identifying a large subset of demonstrations that can be accurately described by a simple, high-performing decision rule. We evaluate our new AI-Interpret algorithm and employ it to translate information-acquisition policies discovered through metalevel reinforcement learning. The results of three large behavioral experiments showed that the provision of decision rules as flowcharts significantly improved people’s planning strategies and decisions across three different classes of sequential decision problems. Furthermore, a series of ablation studies confirmed that our AI-Interpret algorithm was critical to the discovery of interpretable decision rules and that it is ready to be applied to other reinforcement learning problems. We conclude that the methods and findings presented in this article are an important step towards leveraging automatic strategy discovery to improve human decision-making.

Goal contagion is a social-cognitive approach to understanding how other people’s behavior influences one’s goal pursuit: An observation of goal-directed behavior leads to an automatic inference and activation of the goal before it can be adopted and pursued thereafter by the observer. We conducted a meta-analysis focusing on experimental studies with a goal condition, depicting goal-directed behavior and a control condition. We searched four databases (PsychInfo, Web of Science, ScienceDirect, and JSTOR) and the citing literature on Google Scholar, and eventually included e = 48 effects from published studies, unpublished studies and registered reports based on 4751 participants. The meta-analytic summary effect was small − g = 0.30, 95%CI [0.21; 0.40], τ² = 0.05, 95%CI [0.03, 0.13] − implying that goal contagion might occur for some people, compared to when this goal is not perceived in behavior. However, the original effect seemed to be biased through the current publication system. As shown by several publication-bias tests, the effect could rather be half the size, for example, selection model: g = 0.15, 95%CI [–0.02; 0.32]. Further, we could not detect any potential moderator (such as the presentation of the manipulation and the contrast of the control condition). We suggest that future research on goal contagion makes use of open science practices to advance research in this domain.

How do people learn when to allocate how much cognitive control to which task? According to the Learned Value of Control (LVOC) model, people learn to predict the value of alternative control allocations from features of a given situation. This suggests that people may generalize the value of control learned in one situation to other situations with shared features, even when the demands for cognitive control are different. This makes the intriguing prediction that what a person learned in one setting could, under some circumstances, cause them to misestimate the need for, and potentially over-exert control in another setting, even if this harms their performance. To test this prediction, we had participants perform a novel variant of the Stroop task in which, on each trial, they could choose to either name the color (more control-demanding) or read the word (more automatic). However only one of these tasks was rewarded, it changed from trial to trial, and could be predicted by one or more of the stimulus features (the color and/or the word). Participants first learned colors that predicted the rewarded task. Then they learned words that predicted the rewarded task. In the third part of the experiment, we tested how these learned feature associations transferred to novel stimuli with some overlapping features. The stimulus-task-reward associations were designed so that for certain combinations of stimuli the transfer of learned feature associations would incorrectly predict that more highly rewarded task would be color naming, which would require the exertion of control, even though the actually rewarded task was word reading and therefore did not require the engagement of control. Our results demonstrated that participants over-exerted control for these stimuli, providing support for the feature-based learning mechanism described by the LVOC model.

How motivated a person is to pursue a goal may depend on many different properties of the goal, such as how specific it is, how important it is to the person, and how actionable it is. Rigorously measuring all of the relevant goal characteristics is still very difficult. Existing measures are scattered across multiple research fields. Some goal characteristics are not yet covered, while others have been measured under ambiguous terminology. Other conceptually related characteristics have yet to be adapted to goals. Last but not least, the validity of most measures of goal characteristics has yet to be assessed. The aim of this study is to: a) integrate, refine, and extend previous measures into a more comprehensive battery of self-report measures, the Goal Characteristics Questionnaire (GCQ; https://osf.io/3gxk5/?view_only=1ff0e62127c64b82862a0fe7d73c4faf), and b) investigate its evidence of validity. In two empirical studies, this paper provides evidence for the validity of the measures regarding their internal structure, measurement invariance, and convergence and divergence with other relevant goal-related measures, such as the motivation, affect, and the dimensions of Personal Project Analysis. The results show that our goal characteristic dimensions have incremental validity for explaining important outcomes, such as goal commitment and well-being. It concludes with practical recommendations for using the GCQ in research on goal-setting and goal-pursuit, and a discussion about directions for future studies.

Often we find ourselves in unknown situations where we have to make a decision based on reasoning upon experiences. However, it is still unclear how people choose which pieces of information to take into account to achieve well-informed decisions. Answering this question requires an understanding of human metacognitive learning, that is how do people learn how to think. In this study, we focus on a special kind of metacognitive learning, namely how people learn how to plan and how their mechanisms of metacognitive learning adapt the planning strategies to the structures of the environment. We first measured people's adaptation to different environments via a process-tracing paradigm that externalises planning. Then we introduced and fitted novel metacognitive reinforcement learning algorithms to model the underlying learning mechanisms, which enabled us insights into the learning behaviour. Model-based analysis suggested two sources of maladaptation: no learning and reluctance to explore new alternatives.