Benefits of co-learning with an AI agent

2026-05-03

The advent of effective machine learning techniques raises the question of how such procedures might benefit human learning. In this paper we study how humans solve a type of classification puzzle—the Game of Hidden Rules (GOHR)—with vs. without the assistance of a “bot” that provides potentially helpful suggestions about how to proceed. In a GOHR game, the learner attempts to sort colored shapes into categories according to a hidden rule that they must discover, for example “red shapes go to bucket #0”, “blue shapes to bucket #1,” etc. In some conditions, a "bot" made suggestions, which the human learner was free to follow or ignore. Even though human learners did not always take the bot’s advice, we found a consistent performance advantage in bot conditions compared to no-bot conditions, meaning that participants solved these problems more quickly when the bot was present than when it was not. This effect was particularly pronounced in lower-performing subjects, while high-performing subjects were relatively unaffected. We also manipulated the learning speed of the bot, and found that the benefit of bot assistance increased with bot "intelligence." Our results demonstrate that bot assistance can be helpful to human learners, and shed some light on the prospects of AI-supported human learning.

Benefits of co-learning with an AI agent

2026-05-02

The advent of effective machine learning techniques raises the question of how such procedures might benefit human learning. In this paper we study how humans solve a type of classification puzzle—the Game of Hidden Rules (GOHR)—with vs. without the assistance of a “bot” that provides potentially helpful suggestions about how to proceed. In a GOHR game, the learner attempts to sort colored shapes into categories according to a hidden rule that they must discover, for example “red shapes go to bucket #0”, “blue shapes to bucket #1,” etc. In some conditions, a "bot" made suggestions, which the human learner was free to follow or ignore. Even though human learners did not always take the bot’s advice, we found a consistent performance advantage in bot conditions compared to no-bot conditions, meaning that participants solved these problems more quickly when the bot was present than when it was not. This effect was particularly pronounced in lower-performing subjects, while high-performing subjects were relatively unaffected. We also manipulated the learning speed of the bot, and found that the benefit of bot assistance increased with bot "intelligence." Our results demonstrate that bot assistance can be helpful to human learners, and shed some light on the prospects of AI-supported human learning.

Counting Circuits: Mechanistic Interpretability of Visual Reasoning in Large Vision-Language Models

arXiv (Cornell University) · 2026-03-19

Counting serves as a simple but powerful test of a Large Vision-Language Model's (LVLM's) reasoning; it forces the model to identify each individual object and then add them all up. In this study, we investigate how LVLMs implement counting using controlled synthetic and real-world benchmarks, combined with mechanistic analyses. Our results show that LVLMs display a human-like counting behavior, with precise performance on small numerosities and noisy estimation for larger quantities. We introduce two novel interpretability methods, Visual Activation Patching and HeadLens, and use them to uncover a structured "counting circuit" that is largely shared across a variety of visual reasoning tasks. Building on these insights, we propose a lightweight intervention strategy that exploits simple and abundantly available synthetic images to fine-tune arbitrary pretrained LVLMs exclusively on counting. Despite the narrow scope of this fine-tuning, the intervention not only enhances counting accuracy on in-distribution synthetic data, but also yields an average improvement of +8.36% on out-of-distribution counting benchmarks and an average gain of +1.54% on complex, general visual reasoning tasks for Qwen2.5-VL. These findings highlight the central, influential role of counting in visual reasoning and suggest a potential pathway for improving overall visual reasoning capabilities through targeted enhancement of counting mechanisms.

Counting Circuits: Mechanistic Interpretability of Visual Reasoning in Large Vision-Language Models

ArXiv.org · 2026-03-19

Counting serves as a simple but powerful test of a Large Vision-Language Model's (LVLM's) reasoning; it forces the model to identify each individual object and then add them all up. In this study, we investigate how LVLMs implement counting using controlled synthetic and real-world benchmarks, combined with mechanistic analyses. Our results show that LVLMs display a human-like counting behavior, with precise performance on small numerosities and noisy estimation for larger quantities. We introduce two novel interpretability methods, Visual Activation Patching and HeadLens, and use them to uncover a structured "counting circuit" that is largely shared across a variety of visual reasoning tasks. Building on these insights, we propose a lightweight intervention strategy that exploits simple and abundantly available synthetic images to fine-tune arbitrary pretrained LVLMs exclusively on counting. Despite the narrow scope of this fine-tuning, the intervention not only enhances counting accuracy on in-distribution synthetic data, but also yields an average improvement of +8.36% on out-of-distribution counting benchmarks and an average gain of +1.54% on complex, general visual reasoning tasks for Qwen2.5-VL. These findings highlight the central, influential role of counting in visual reasoning and suggest a potential pathway for improving overall visual reasoning capabilities through targeted enhancement of counting mechanisms.

Exploring Human Navigational Strategies in a Dynamic VR Social Wayfinding Task

Journal of Vision · 2025-07-15

How do people navigate through crowded, dynamic environments? We investigated social wayfinding (navigation influenced by other people) in virtual reality (VR). Using a VR headset, we placed our participants in a virtual train station and asked them to physically navigate to one of the two exit gates while under time pressure. The station included several obstacles, and a number of virtual humans ("agents"), some static and others walking. The walking agents passed single file in two rows approximately perpendicular to the participant's path, requiring the participants to navigate through gaps between the agents. In a series of experiments, we manipulated the sitting agents' presence, the walking agents' direction, and the time allotted for subjects to reach the gate. Our analyses focused on the balance between global planning, in which the wayfinder plots the entire path from beginning to end, and local planning, in which the wayfinder continually modifies their path in light of newly encountered obstacles. We found evidence for local wayfinding from several sources. First, though most subjects went directly towards the target gate, following a global plan, some subjects switched midway from the foil to target gate, suggesting a more local strategy. Second, even subjects who consistently headed to the target gate decelerated as they approached the room's midpoint, suggesting local path modifications to pass through a "gap" between agents. In Exp. 3, we collected participants’ eye gaze as they traversed the room. Participants fixated almost exclusively on moving agents and other informative elements of the environment (e.g., the sign that identified the target gate or the countdown timer). In sum, participants’ movements and eyegaze provide evidence for both global and local decision-making in a social wayfinding task. Our study demonstrates an interplay of both strategies, with the balance depending on the progression of available information.

Social Wayfinding in VR: Navigational Decisions and Eye Movements in a Dynamic Environment

2025-07-01

Social wayfinding refers to the process of navigating in the presence of other people. Socialwayfinding entails a complex series of interrelated decisions, such as how closely to approachpeople and when to pass them. In this paper we report two virtual reality (VR) experiments thatinvestigate social wayfinding in a complex, dynamic task. In these experiments, participantsphysically walked from one end of a simulated train station waiting room to the other, avoidingstatic obstacles (e.g., benches, seated and standing people) and dynamic obstacles (two rows ofpeople walking perpendicularly to the participant’s path). We model the task as a hierarchicalcombination of local subgoals (e.g., when and where to pass people) and a global goal (whichgate to navigate toward). Although eye movements are difficult to analyze in such a dynamictask, they prove to be particularly revealing about how participants combined these local andglobal goals efficiently in real time. Overall, the results suggest that adults are experts atsocial navigational tasks, rapidly deploying a flexible combination of local and global decisionstrategies to navigate crowded environments efficiently.

Defining Nonsuicidal Self-Injury in Autistic People: A Framework for Assessment Using Key Elements to Aid in Characterization

2025-05-28 · 1 citations

Nonsuicidal self-injury (NSSI) is the intentional destruction of one’s own body tissue without suicidal intent and for purposes that are not socially or culturally accepted or practiced. Research on NSSI in autistic people is limited but increasing. NSSI is strongly associated with suicide, and it is an important behavior to better understand given the high rates of NSSI and suicide in autistic people. To date, research focused on autistic people has mostly used self-report questionnaires to assess NSSI, with a more limited application of clinical interviews of NSSI. However, researchers and clinicians may find it challenging to determine whether a behavior is categorized as NSSI in autistic people, especially since autistic people may present with other behaviors that cause self-injury. We set forth key elements in defining NSSI to support better reliability of NSSI assessment across studies. We emphasize that when assessing for NSSI the behavior must meet these key elements: (1) not intended to cause death, (2) the self-injury/harm is intentional, (3) there is immediate physical injury following the behavior, (4) the injury is to the external body, not internal body, (5) the injury is self-imposed and not done by another being, and (6) it is not a part of social or cultural practices. It will be important for future work to develop measures that can accurately assess NSSI in autistic people and advance mechanistic and intervention research related to NSSI.

Probabilistic origins of compositional mental representations

2025-01-11

The representation of complex phenomena via combinations of simple discrete features is a hallmark of human cognition. But it is not clear exactly how (or whether) discrete features can effectively represent the complex probabilistic fabric of the environment. This paper introduces information-theoretic tools for quantifying the fidelity and efficiency of a featural representation with respect to a probability model. In this framework a feature or combination of features is “faithful” to the extent that knowing the value of the features reduces uncertainty about the true state of the world. In a single dimension, a discrete feature is faithful if the values of the feature correspond isomorphically to distinct classes in the probability model. But in multiple dimensions, the situation is more complicated: the fidelity of each feature depends on the direction in multidimensional feature space in which the feature is projected from the underlying distribution. More interestingly, distributions may be more effectively represented by combinations of projected features—that is, compositionality. For any given distribution, a variety of compositional forms (features and combination rules) are possible, which can be quite different from one another, entailing different degrees of fidelity, different numbers of features, and even different induced regularities. This paper proposes three specific criteria for a compositional representation: fidelity, simplicity, and robustness. The information-theoretic framework introduces a new and potentially useful way to look at the problem of compositionality in human mental representation.

Perceptual Biases in the Interpretation of Non-Rigid Shape Transformations from Motion

Vision · 2024-07-04 · 2 citations

Most existing research on the perception of 3D shape from motion has focused on rigidly moving objects. However, many natural objects deform non-rigidly, leading to image motion with no rigid interpretation. We investigated potential biases underlying the perception of non-rigid shape interpretations from motion. We presented observers with stimuli that were consistent with two qualitatively different interpretations. Observers were shown a two-part 3D object with the smaller part changing in length dynamically as the whole object rotated back and forth. In two experiments, we studied the misperception (i.e., perceptual reinterpretation) of the non-rigid length change to a part. In Experiment 1, observers misperceived this length change as a part orientation change (i.e., the smaller part was seen as articulating with respect to the larger part). In Experiment 2, the stimuli were similar, except the silhouette of the part was visible in the image. Here, the non-rigid length change was reinterpreted as a rigidly attached part with an "illusory" non-orthogonal horizontal angle relative to the larger part. We developed a model that incorporated this perceptual reinterpretation and could predict observer data. We propose that the visual system may be biased towards part-wise rigid interpretations of non-rigid motion, likely due to the ecological significance of movements of humans and other animals, which are generally constrained to move approximately part-wise rigidly. That is, not all non-rigid deformations are created equal: the visual systems' prior expectations may bias the system to interpret motion in terms of biologically plausible shape transformations.

Visual perception: On the trail of high-level shape aftereffects

Current Biology · 2024-03-01