About

Jaime Arguello is a professor involved in teaching and research related to information retrieval (IR). His course focuses on the analysis, organization, storage, and retrieval of unstructured and semi-structured data, primarily text. The curriculum covers the theory, implementation, and evaluation of IR systems and techniques, exploring how search engines work, interpret human language, and how they can be improved. His teaching includes system-focused and user-focused aspects of IR, emphasizing concepts, basic mathematics, and practical applications.

Research topics

• Computer Science
• Psychology
• Information Retrieval
• Cognitive psychology
• Machine Learning
• Artificial Intelligence
• Medicine
• Knowledge management
• World Wide Web
• Human–computer interaction

Selected publications

• Understanding the Effects of Explaining Predictive but Unintuitive Features in Human-XAI Interaction

  2025-06-23 · 1 citations

  articleOpen access

  Feature importance explanation, which highlights input features that are most influential to the output, is a popular explainable AI (XAI) technique to help users understand machine learning model predictions.However, features deemed predictive by machines can still be puzzling or even appear unintuitive to end-users.Explaining why a feature is predictive is an underexplored area in current XAI research.In this paper, we used deception detection as a case study.We leveraged a large language model (LLM) to explain why a word is predictive of genuine or deceptive reviews.We first validated the LLM-generated explanations to be non-hallucinated through an algorithmic evaluation.Then, we conducted a crowdsourced study ( = 220) to investigate how unintuitive words and LLM-generated explanations influence participants in a deception detection task.Our study results found that showing unintuitive features without explaining why they are predictive was no better than not showing them at all, while explaining why these features are predictive significantly enhanced participants' learning of the task, appropriate reliance on AI assistance, and perceptions of the AI system.

  PublisherOA PDFDOI

• Tip of the Tongue Query Elicitation for Simulated Evaluation

  ArXiv.org · 2025-02-25

  preprintOpen access

  Tip-of-the-tongue (TOT) search occurs when a user struggles to recall a specific identifier, such as a document title. While common, existing search systems often fail to effectively support TOT scenarios. Research on TOT retrieval is further constrained by the challenge of collecting queries, as current approaches rely heavily on community question-answering (CQA) websites, leading to labor-intensive evaluation and domain bias. To overcome these limitations, we introduce two methods for eliciting TOT queries - leveraging large language models (LLMs) and human participants - to facilitate simulated evaluations of TOT retrieval systems. Our LLM-based TOT user simulator generates synthetic TOT queries at scale, achieving high correlations with how CQA-based TOT queries rank TOT retrieval systems when tested in the Movie domain. Additionally, these synthetic queries exhibit high linguistic similarity to CQA-derived queries. For human-elicited queries, we developed an interface that uses visual stimuli to place participants in a TOT state, enabling the collection of natural queries. In the Movie domain, system rank correlation and linguistic similarity analyses confirm that human-elicited queries are both effective and closely resemble CQA-based queries. These approaches reduce reliance on CQA-based data collection while expanding coverage to underrepresented domains, such as Landmark and Person. LLM-elicited queries for the Movie, Landmark, and Person domains have been released as test queries in the TREC 2024 TOT track, with human-elicited queries scheduled for inclusion in the TREC 2025 TOT track. Additionally, we provide source code for synthetic query generation and the human query collection interface, along with curated visual stimuli used for eliciting TOT queries.

  PublisherOA PDFDOI

• Information needs and perceptions of chatbots for hypertension medication self-management: a mixed methods study

  UNC Libraries · 2025-06-20

  articleOpen accessSenior author

  OBJECTIVE: Chatbots have potential to deliver interactive self-management interventions but have rarely been studied in the context of hypertension or medication adherence. The objective of this study was to better understand patient information needs and perceptions of chatbots to support hypertension medication self-management. MATERIALS AND METHODS: Mixed methods were used to assess self-management needs and preferences for using chatbots. We purposively sampled adults with hypertension who were prescribed at least one medication. Participants completed questionnaires on sociodemographics, health literacy, self-efficacy, and technology use. Semi-structured interviews were conducted, audio-recorded, and transcribed verbatim. Quantitative data were analyzed using descriptive statistics, and qualitative data were analyzed using applied thematic analysis. RESULTS: Thematic saturation was met after interviewing 15 participants. Analysis revealed curiosity toward chatbots, and most perceived them as humanlike. The majority were interested in using a chatbot to help manage medications, refills, communicate with care teams, and for accountability toward self-care tasks. Despite general enthusiasm, there were concerns with chatbots providing too much information, making demands for lifestyle changes, invading privacy, and usability issues with deployment on smartphones. Those with overall positive perceptions toward chatbots were younger and taking fewer medications. DISCUSSION: Chatbot-related informational needs were consistent with existing self-management research, and many felt chatbots would be valuable if customizable and compatible with patient portals, pharmacies, or health apps. CONCLUSION: Although most were not familiar with chatbots, patients were interested in interacting with them, but this varied. This research informs future design and functionalities of conversational interfaces to support hypertension self-management.

  PublisherDOI

• Search+Chat: Integrating Search and GenAI to Support Users with Learning-oriented Search Tasks

  2025-03-24 · 10 citations

  articleOpen access

  Generative AI (GenAI) technologies such as ChatGPT are changing the ways people interact with information.To illustrate, popular search engines (e.g., Google) have started integrating responses from GenAI tools with the traditional search results.In this paper, we explore the integration of GenAI technology with traditional search in the context of a learning-oriented task.We report on a between-subjects study ( = 40) in which participants completed a complex, learning-oriented search task.Participants were assigned to one of two conditions.In the SearchOnly condition, participants used a traditional web search system to gather information.In the Search+Chat condition, participants used an experimental system that combined a traditional web search component and an interactive GenAI-based chat component (Chat AI).The study investigated seven research questions.RQ1-RQ3 focused on differences between groups: (RQ1) post-task perceptions, (RQ2) search behaviors, and (RQ3) learning outcomes.To measure learning, participants completed a multiple-choice test before the search task, immediately after, and one week later (to measure retention).RQ4-RQ7 delved deeper into participants' behaviors and experiences in the Search+Chat condition: (RQ4) motivations for (and gains from) engaging with the Chat AI; (RQ5) the phases during which participants engaged with the Chat AI; (RQ6) the types of queries issued to each component; and (RQ7) perceptions about the information returned by each component.

  PublisherOA PDFDOI

• The Effects of Working Memory during a Search and Sensemaking Task

  2025-03-24 · 4 citations

  articleOpen accessSenior author

  Working memory (WM) is involved in high-level cognitive tasks such as comprehension, reasoning, and learning.Search and sensemaking (SSM) is no exception-a wide range of (meta)cognitive activities are involved in the process of making sense of a complex topic by gathering information.Prior studies have found that WM can influence search behaviors, perceptions, and outcomes.However, little work has been done to gain insights into how WM might affect the SSM process.We report on a lab study ( = 44) in which participants were binned into low-and high-WM groups.During the study, participants were asked to learn about a complex and multifaceted topic by gathering information using a web search engine and taking notes.After the search session, participants were asked to produce a summary of everything they learned.The study investigated four research questions.RQ1 and RQ2 investigate the effects of WM on post-task perceptions and search behaviors.RQ3 investigates the effects of WM on the extent to which participants engaged in specific search, sensemaking, and cognitive activities.To address RQ3, the study used a think-aloud protocol.Search sessions (i.e., recorded actions and think-aloud comments) were then analyzed using qualitative techniques.Finally, RQ4 investigates the effects of WM on learning outcomes.Our RQ4 results found that high-WM participants had better learning outcomes.Our RQ2 and RQ3 results point to possible reasons why.Despite differences for RQ2-RQ4, there were no differences in post-task perceptions (RQ1).

  PublisherOA PDFDOI

• Tip of the Tongue Query Elicitation for Simulated Evaluation

  2025-07-13 · 2 citations

  articleOpen access

  Tip-of-the-tongue (TOT) search occurs when a user struggles to recall a specific identifier, such as a document title. While common, existing search systems often fail to effectively support TOT scenarios. Research on TOT retrieval is further constrained by the challenge of collecting queries, as current approaches rely heavily on community question-answering (CQA) websites, leading to labor-intensive evaluation and domain bias. To overcome these limitations, we introduce two methods for eliciting TOT queries-leveraging large language models (LLMs) and human participants-to facilitate simulated evaluations of TOT retrieval systems. Our LLM-based TOT user simulator generates synthetic TOT queries at scale, achieving high correlations with how CQA-based TOT queries rank TOT retrieval systems when tested in the Movie domain. Additionally, these synthetic queries exhibit high linguistic similarity to CQA-derived queries. For human-elicited queries, we developed an interface that uses visual stimuli to place participants in a TOT state, enabling the collection of natural queries. In the Movie domain, system rank correlation and linguistic similarity analyses confirm that human-elicited queries are both effective and closely resemble CQA-based queries. These approaches reduce reliance on CQA-based data collection while expanding coverage to underrepresented domains, such as Landmark and Person. LLM-elicited queries for the Movie, Landmark, and Person domains have been released as test queries in the TREC 2024 TOT track, with human-elicited queries scheduled for inclusion in the TREC 2025 TOT track. Additionally, we provide source code for synthetic query generation and the human query collection interface, along with curated visual stimuli used for eliciting TOT queries.

  PublisherOA PDFDOI

• Referee report. For: Non-canonical odor representation and learning in Dipteran brains [version 2; peer review: 3 approved]

  Faculty of 1000 Research Ltd · 2024-01-01

  articleOpen access1st authorCorresponding
  PublisherDOI

• Why is "Problems" Predictive of Positive Sentiment? A Case Study of Explaining Unintuitive Features in Sentiment Classification

  2024-06-03 · 3 citations

  preprintOpen access

  Explainable AI (XAI) algorithms aim to help users understand how a machine learning model makes predictions. To this end, many approaches explain which input features are most predictive of a target label. However, such explanations can still be puzzling to users (e.g., in product reviews, the word “problems” is predictive of positive sentiment). If left unexplained, puzzling explanations can have negative impacts. Explaining unintuitive associations between an input feature and a target label is an underexplored area in XAI research. We take an initial effort in this direction using unintuitive associations learned by sentiment classifiers as a case study. We propose approaches for (1) automatically detecting associations that can appear unintuitive to users and (2) generating explanations to help users understand why an unintuitive feature is predictive. Results from a crowdsourced study (N = 300) found that our proposed approaches can effectively detect and explain predictive but unintuitive features in sentiment classification.

  PublisherOA PDFDOI

• The Effects of Goal-setting on Learning Outcomes and Self-Regulated Learning Processes

  2024-03-08 · 8 citations

  articleOpen accessSenior author

  We present a user study (N = 40) that investigated the role of goal-setting on learning during search. To this end, we developed a tool called the Subgoal Manager (SM). The SM was designed to help searchers break apart a learning-oriented search task into smaller subgoals. The tool enabled participants to add, delete, and modify subgoals; take notes with respect to subgoals; and mark subgoals as completed. During the study, participants completed a single learning-oriented search task and were assigned to one of two subgoal conditions. In the Subgoals condition, participants had access to the SM; were instructed to develop at least three subgoals before the search session; and could add, delete, and modify subgoals during the search session. In the NoSubgoals condition, participants were not instructed to set subgoals and were simply provided with a text editor to take notes. We investigate the effects of the subgoal condition on: (RQ1) learning and retention and (RQ2) the extent to which participants engaged in specific self-regulated learning (SRL) processes during the search session. Our results found two important trends. First, participants in the Subgoals condition had better learning outcomes, especially with respect to retention. Second, based on a qualitative analysis of participants’ search sessions, participants in the Subgoals condition engaged in more self-regulated learning (SRL) processes. Combined, our results suggest that goal-setting improves learning during search by encouraging and supporting greater engagement with SRL processes.

  PublisherOA PDFDOI

• Understanding the Cognitive Influences of Interpretability Features on How Users Scrutinize Machine-Predicted Categories

  2023-03-19 · 1 citations

  article

  The goal of interpretable machine learning (ML) is to design tools and visualizations to help users scrutinize a system’s predictions. Prior studies have mostly employed quantitative methods to investigate the effects of specific tools/visualizations on outcomes related to objective performance—a human’s ability to correctly agree or disagree with the system—and subjective perceptions of the system. Few studies have employed qualitative methods to investigate how and why specific tools/visualizations influence performance, perceptions, and behaviors. We report on a lab study (N = 30) that investigated the influences of two interpretability features: confidence values and sentence highlighting. Participants judged whether medical articles belong to a predicted medical topic and were exposed to two interface conditions—one with and one without interpretability features. We investigate the effects of our interpretability features on participants’ performance and perceptions. Additionally, we report on a qualitative analysis of participants’ responses during an exit interview. Specifically, we report on how our interpretability features impacted different cognitive activities that participants engaged with during the task—reading, learning, and decision making. We also describe ways in which the interpretability features introduced challenges and sometimes led participants to make mistakes. Insights gained from our results point to future directions for interpretable ML research.

  PublisherDOI

Recent grants

• CAREER: Making Aggregated Search Results More Effective and Useful

  NSF · $519k · 2015–2020

Frequent coauthors

• R. Capra

  32 shared

• Fernando Díaz

  Carnegie Mellon University

  15 shared

• Bogeum Choi

  13 shared

• Jamie Callan

  11 shared

• Sandeep Avula

  9 shared

• Kelsey Urgo

  University of San Francisco

  8 shared

• Yue Wang

  8 shared

• Carolyn Penstein Rosé

  7 shared

Education

• Ph.D., Federated Search in Heterogeneous Environments

  Carnegie Mellon University

  2011

Awards & honors

• NSF CAREER: Making Aggregated Search Results More Effective…
• NSF Small: Search Assistance Using Search Trails
• Methods Recognition (Honorable Mention)