About

Robert Frank is a professor of Linguistics at Yale University. He received his PhD in 1992 from the University of Pennsylvania in Computer and Information Science. His research explores computational models of language learning and processing as well as the role of computational constraints in linguistic explanation. He has worked extensively on the application of the Tree Adjoining Grammar formalism in syntactic theory. Before coming to Yale, he held positions at Johns Hopkins University in Cognitive Science and at the University of Delaware in Linguistics.

Research topics

• Artificial Intelligence
• Computer Science
• Natural Language Processing
• Mathematics
• Machine Learning
• Statistics
• Data science
• Psychology
• Geography

Selected publications

• Meaning Beyond Truth Conditions: Evaluating Discourse Level Understanding via Anaphora Accessibility

  ArXiv.org · 2025-02-19

  preprintOpen accessSenior author

  We present a hierarchy of natural language understanding abilities and argue for the importance of moving beyond assessments of understanding at the lexical and sentence levels to the discourse level. We propose the task of anaphora accessibility as a diagnostic for assessing discourse understanding, and to this end, present an evaluation dataset inspired by theoretical research in dynamic semantics. We evaluate human and LLM performance on our dataset and find that LLMs and humans align on some tasks and diverge on others. Such divergence can be explained by LLMs' reliance on specific lexical items during language comprehension, in contrast to human sensitivity to structural abstractions.

  PublisherOA PDFDOI

• The Structural Sources of Verb Meaning Revisited: Large Language Models Display Syntactic Bootstrapping

  ArXiv.org · 2025-08-17

  preprintOpen accessSenior author

  Syntactic bootstrapping (Gleitman, 1990) is the hypothesis that children use the syntactic environments in which a verb occurs to learn its meaning. In this paper, we examine whether large language models exhibit a similar behavior. We do this by training RoBERTa and GPT-2 on perturbed datasets where syntactic information is ablated. Our results show that models' verb representation degrades more when syntactic cues are removed than when co-occurrence information is removed. Furthermore, the representation of mental verbs, for which syntactic bootstrapping has been shown to be particularly crucial in human verb learning, is more negatively impacted in such training regimes than physical verbs. In contrast, models' representation of nouns is affected more when co-occurrences are distorted than when syntax is distorted. In addition to reinforcing the important role of syntactic bootstrapping in verb learning, our results demonstrated the viability of testing developmental hypotheses on a larger scale through manipulating the learning environments of large language models.

  PublisherOA PDFDOI

• Is In-Context Learning a Type of Error-Driven Learning? Evidence from the Inverse Frequency Effect in Structural Priming

  2025-01-01

  articleOpen access

  Zhenghao Zhou, Robert Frank, R. Thomas McCoy. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

  PublisherOA PDFDOI

• Meaning Beyond Truth Conditions: Evaluating Discourse Level Understanding via Anaphora Accessibility

  2025-01-01

  articleOpen accessSenior author

  We present a hierarchy of natural language understanding abilities and argue for the importance of moving beyond assessments of understanding at the lexical and sentence levels to the discourse level.We propose the task of anaphora accessibility as a diagnostic for assessing discourse understanding, and to this end, present an evaluation dataset inspired by theoretical research in dynamic semantics.We evaluate human and LLM performance on our dataset and find that LLMs and humans align on some tasks and diverge on others.Such divergence can be explained by LLMs' reliance on specific lexical items during language comprehension, in contrast to human sensitivity to structural abstractions.

  PublisherOA PDFDOI

• Concise One-Layer Transformers Can Do Function Evaluation (Sometimes)

  ArXiv.org · 2025-03-28

  preprintOpen accessSenior author

  While transformers have proven enormously successful in a range of tasks, their fundamental properties as models of computation are not well understood. This paper contributes to the study of the expressive capacity of transformers, focusing on their ability to perform the fundamental computational task of evaluating an arbitrary function from $[n]$ to $[n]$ at a given argument. We prove that concise 1-layer transformers (i.e., with a polylog bound on the product of the number of heads, the embedding dimension, and precision) are capable of doing this task under some representations of the input, but not when the function's inputs and values are only encoded in different input positions. Concise 2-layer transformers can perform the task even with the more difficult input representation. Experimentally, we find a rough alignment between what we have proven can be computed by concise transformers and what can be practically learned.

  PublisherOA PDFDOI

• LIEDER: Linguistically-Informed Evaluation for Discourse Entity Recognition

  2024-01-01

  articleOpen accessSenior author

  Discourse Entity (DE) recognition is the task of identifying novel and known entities introduced within a text.While previous work has found that large language models have basic, if imperfect, DE recognition abilities (Schuster and Linzen, 2022), it remains largely unassessed which of the fundamental semantic properties that govern the introduction and subsequent reference to DEs they have knowledge of.We propose the Linguistically-Informed Evaluation for Discourse Entity Recognition (LIEDER) dataset that allows for a detailed examination of language models' knowledge of four crucial semantic properties: EXISTENCE, UNIQUENESS, PLURALITY, and NOVELTY.We find evidence that state-of-the-art large language models exhibit sensitivity to all of these properties except NOVELTY, which demonstrates that they have yet to reach human-level language understanding abilities.

  PublisherOA PDFDOI

• Brain and grammar: revealing electrophysiological basic structures with competing statistical models

  bioRxiv (Cold Spring Harbor Laboratory) · 2024-02-06

  preprintOpen access

  Abstract Acoustic, lexical and syntactic information is simultaneously processed in the brain. Therefore, distinguishing the electrophysiological activity pertaining to these components requires complex and indirect strategies. Capitalizing on previous works which factor out acoustic information, we could concentrate on the lexical and syntactic contribution to language processing by testing competing statistical models. We exploited EEG recordings and compared different surprisal models selectively involving lexical information, part of speech or syntactic structures in various combinations. EEG responses were recorded in 32 participants during listening to affirmative active declarative sentences and compared the activation corresponding to basic syntactic structures, such as noun phrases vs verb phrases. Lexical and syntactic processing activates different frequency bands, different time windows and different networks. Moreover, surprisal models based on part of speech inventory only do not explain well the electrophysiological data, while those including syntactic information do. Finally, we confirm previous measures obtained with intracortical recordings independently supporting the original hypothesis addressed here in a robust way.

  PublisherOA PDFDOI

• Is In-Context Learning a Type of Error-Driven Learning? Evidence from the Inverse Frequency Effect in Structural Priming

  arXiv (Cornell University) · 2024-06-26

  preprintOpen access

  Large language models (LLMs) have shown the emergent capability of in-context learning (ICL). One line of research has claimed that ICL is functionally equivalent to gradient descent, a type of error-driven learning mechanism. In this paper, we introduce a new way of diagnosing whether ICL is functionally performing error-driven learning. Our approach is based on the inverse frequency effect (IFE) -- a phenomenon in which an agent's behavior is influenced to a greater degree when presented with improbable examples as compared to more likely ones. The IFE has previously been identified in psycholinguistics where humans exhibit the IFE in the context of structural priming (the tendency for people to produce sentence structures they have encountered recently). In that context, the IFE has been used as evidence that human structural priming must involve error-driven learning mechanisms. In our experiments, we simulated structural priming with ICL and found that LLMs indeed display the IFE, with the effect being stronger in larger models. We conclude that at least in the case we studied, ICL is indeed a type of error-driven learning, supporting the hypothesis that an error signal is implicitly computed in the forward pass during ICL. Our results suggest that both humans and LLMs make use of error-driven processing mechanisms in on-line processing.

  PublisherOA PDFDOI

• Introduction: An historical perspective

  Elsevier eBooks · 2024-11-29

  book-chapter1st authorCorresponding
  PublisherDOI

• LIEDER: Linguistically-Informed Evaluation for Discourse Entity Recognition

  arXiv (Cornell University) · 2024-03-10

  preprintOpen accessSenior author

  Discourse Entity (DE) recognition is the task of identifying novel and known entities introduced within a text. While previous work has found that large language models have basic, if imperfect, DE recognition abilities (Schuster and Linzen, 2022), it remains largely unassessed which of the fundamental semantic properties that govern the introduction and subsequent reference to DEs they have knowledge of. We propose the Linguistically-Informed Evaluation for Discourse Entity Recognition (LIEDER) dataset that allows for a detailed examination of language models' knowledge of four crucial semantic properties: existence, uniqueness, plurality, and novelty. We find evidence that state-of-the-art large language models exhibit sensitivity to all of these properties except novelty, which demonstrates that they have yet to reach human-level language understanding abilities.

  PublisherOA PDFDOI

Recent grants

• NIH Grant R29DA004483

  NIH · $513k · 1992

• NIH Grant R01DA004483

  NIH · $464k · 1995

• Collaborative Research: Inductive Biases for the Acquisition of Syntactic Transformations in Neural Networks

  NSF · $343k · 2019–2025

Frequent coauthors

• Jungo Kasai

  Toyota Technological Institute at Chicago

  16 shared

• Owen Rambow

  15 shared

• Andrea Cometa

  IMT School for Advanced Studies Lucca

  13 shared

• Tal Linzen

  13 shared

• R. Thomas McCoy

  Princeton University

  11 shared

• Dan Friedman

  11 shared

• William Merrill

  11 shared

• Dragomir Radev

  10 shared

Labs

• Historical Linguistics and Australian Languages (HistLing)PI

Education

• PhD, Computer and Information Science

  University of Pennsylvania

  1992

• SB, Brain and Cognitive Science

  Massachusetts Institute of Technology

  1987