About

Timothy Bickmore is an affiliated faculty member in the Electrical and Computer Engineering department at Northeastern University and holds the position of Professor at the Khoury College of Computer Sciences. He also serves as the Associate Dean for Research at the Khoury College of Computing Sciences. His research focuses on Human-Computer Interaction, Health Informatics, Dialog systems, and Conversational agents. He works with the Relational Agents Group, which specializes in simulating face-to-face counseling, primarily in health education and health behavior change interventions, with a particular emphasis on the relational aspects of these interactions and how they unfold over time.

Research topics

• Computer Science
• Medicine
• Human–computer interaction
• Psychology
• Sociology
• Applied psychology
• Nursing
• Artificial Intelligence
• Medical education
• Political Science
• Psychiatry
• Family medicine
• Physical therapy
• Gerontology
• Surgery
• World Wide Web
• Social psychology
• Biology
• Clinical psychology
• Multimedia
• Genetics
• Demography
• Communication
• Knowledge management

Selected publications

• Neighborhood context shapes physical activity intervention outcomes: a comparison of human vs. virtual advisors

  BMC Public Health · 2026-03-16

  articleOpen access

  Neighborhood conditions are key social determinants of health (SDOH) that play a critical role in healthy aging. Incorporating contextual measures into behavioral medicine interventions is essential for creating adaptable, equity-focused health solutions. Yet, few physical activity (PA) interventions explicitly evaluate how these factors influence effectiveness. We conducted a secondary analysis of the Computerized Physical Activity Support for Seniors (COMPASS) Trial to examine whether neighborhood context, measured by the California Healthy Places Index (HPI), moderated intervention effects on PA among Latino/a older adults. The COMPASS trial was a single-blind, cluster-randomized non-inferiority trial comparing an interactive virtual advisor (reference arm) with a trained human peer advisor. PA outcomes, including weekly minutes of walking and moderate-to-vigorous physical activity (MVPA), were assessed with the CHAMPS questionnaire at baseline and 12 months. Neighborhood conditions were measured with the HPI, a composite index of 25 indicators across eight domains (e.g., housing, education, transportation) that reflect place-based SDOH. Scores range from 0 to 100, with higher values indicating more health-supportive environments. Mixed-effects ANCOVA models adjusted for baseline PA, age, and gender, with a random intercept for study site, and tested effect modification via interaction terms between intervention arm and HPI. Among 245 Latino/a participants (mean age = 62.3 years; 78.8% female), HPI significantly moderated intervention effects. When modeled continuously, significant interactions were observed for 12-month changes in walking (β = − 2.93; P = .041) and MVPA (β = − 2.54 min/week per HPI point; P = .033). These findings indicated that the human advisor was comparatively more effective in lower-HPI neighborhoods, whereas the virtual advisor produced stronger improvements in higher-HPI neighborhoods. In sensitivity analyses using dichotomized HPI, the interaction was significant for MVPA (β = − 112.9; P = .026) and trended for walking (P = .077). Neighborhood context moderated the relative effectiveness of digital versus human-delivered PA interventions. These findings suggest that tailoring delivery strategies, leveraging digital tools in advantaged areas and peer support in under-resourced neighborhoods, may enhance equity in health promotion for older Latino/a adults by explicitly accounting for neighborhood-level social determinants of health. Clinicaltrials.gov NCT02111213 Registered April 2, 2014 https://clinicaltrials.gov/study/NCT02111213.

  PublisherDOI

• Assessing Risks of Large Language Models in Mental Health Support: A Framework for Automated Clinical AI Red Teaming

  ArXiv.org · 2026-01-01

  articleOpen accessSenior author

  Large Language Models (LLMs) are increasingly utilized for mental health support; however, current safety benchmarks often fail to detect the complex, longitudinal risks inherent in therapeutic dialogue. We introduce an evaluation framework that pairs AI psychotherapists with simulated patient agents equipped with dynamic cognitive-affective models and assesses therapy session simulations against a comprehensive quality of care and risk ontology. We apply this framework to a high-impact test case, Alcohol Use Disorder, evaluating six AI agents (including ChatGPT, Gemini, and Character AI) against a clinically-validated cohort of 15 patient personas representing diverse clinical phenotypes. Our large-scale simulation (N=369 sessions) reveals critical safety gaps in the use of AI for mental health support. We identify specific iatrogenic risks, including the validation of patient delusions ("AI Psychosis") and failure to de-escalate suicide risk. Finally, we validate an interactive data visualization dashboard with diverse stakeholders, including AI engineers and red teamers, mental health professionals, and policy experts (N=9), demonstrating that this framework effectively enables stakeholders to audit the "black box" of AI psychotherapy. These findings underscore the critical safety risks of AI-provided mental health support and the necessity of simulation-based clinical red teaming before deployment.

  PublisherOA PDF

• Health Literacy and Usability of Public Health Websites for COVID-19 Vaccine Search

  HLRP Health Literacy Research and Practice · 2026-01-01

  articleOpen access

  BACKGROUND: Millions of United States residents made use of public health websites during the coronavirus disease 2019 (COVID-19) pandemic to obtain information about vaccines and determine vaccine eligibility. OBJECTIVE: For public health websites to be effective, they must be usable. This study aimed to evaluate the usability of government websites for vaccine information, by examining whether these platforms helped users determine COVID-19 vaccine eligibility accurately and satisfactorily, as well as how health literacy (HL) plays a role in accurate eligibility judgments and satisfaction with the media. METHODS: = 39), where each participant used two websites and one embodied conversational agent system to determine their own eligibility for vaccination, as well as that of a fictitious persona as a standardized task. The website conditions in the study included the Centers for Disease Control (CDC) website, as well as vaccines.gov and mass.gov. The accuracy of participant-estimated eligibility and usability were further analyzed for association with HL. KEY RESULTS: Participants' estimate of vaccine eligibility was generally inaccurate for all website conditions, with an overall rate of 53.8% for correct responses. Participants with low HL had more incorrect responses and confusion, and HL was found to be a significant predictor of eligibility correctness when they were determining their own vaccine eligibility using the CDC website. Participants also reported having a significantly higher ability to find information and were more satisfied when interacting with the embodied conversational agent system, compared to the websites. CONCLUSIONS: Government websites-particularly the CDC website-were found to lack usability, especially for those with low HL. High error rates and low satisfaction underscore the need for simplification of public health site content and design and motivate the development of novel education methods for public health communication.

  PublisherOA PDFDOI

• Balancing Efficiency and Empathy: Healthcare Providers' Perspectives on AI-Supported Workflows for Serious Illness Conversations in the Emergency Department

  2026-04-13 · 2 citations

  articleOpen access

  Serious Illness Conversations (SICs)—discussions about values and care preferences for patients with life-threatening illness—rarely occur in Emergency Departments (EDs), despite evidence that early conversations improve care alignment and reduce unnecessary interventions. We interviewed 11 ED providers to identify challenges in SICs and opportunities for technology support, with a focus on AI. Our analysis revealed a four-stage SIC workflow (identification, preparation, conduction, documentation) and barriers at each stage, including fragmented patient information, limited time and space, lack of conversational guidance, and burdensome documentation. Providers expressed interest in AI systems for synthesizing information, supporting real-time conversations, and automating documentation, but emphasized concerns about preserving human connection and clinical autonomy. This tension highlights the need for technologies that enhance efficiency without undermining the interpersonal nature of SICs. We propose design guidelines for ambient and peripheral AI systems to support providers while preserving the essential humanity of these conversations.

  PublisherDOI

• Assessing Risks of Large Language Models in Mental Health Support: A Framework for Automated Clinical AI Red Teaming

  arXiv (Cornell University) · 2026-02-23

  preprintOpen accessSenior author

  Large Language Models (LLMs) are increasingly utilized for mental health support; however, current safety benchmarks often fail to detect the complex, longitudinal risks inherent in therapeutic dialogue. We introduce an evaluation framework that pairs AI psychotherapists with simulated patient agents equipped with dynamic cognitive-affective models and assesses therapy session simulations against a comprehensive quality of care and risk ontology. We apply this framework to a high-impact test case, Alcohol Use Disorder, evaluating six AI agents (including ChatGPT, Gemini, and Character AI) against a clinically-validated cohort of 15 patient personas representing diverse clinical phenotypes. Our large-scale simulation (N=369 sessions) reveals critical safety gaps in the use of AI for mental health support. We identify specific iatrogenic risks, including the validation of patient delusions ("AI Psychosis") and failure to de-escalate suicide risk. Finally, we validate an interactive data visualization dashboard with diverse stakeholders, including AI engineers and red teamers, mental health professionals, and policy experts (N=9), demonstrating that this framework effectively enables stakeholders to audit the "black box" of AI psychotherapy. These findings underscore the critical safety risks of AI-provided mental health support and the necessity of simulation-based clinical red teaming before deployment.

  PublisherDOI

• Scaffolding Empathy: Training Counselors with Simulated Patients and Utterance-level Performance Visualizations

  2025-04-24 · 17 citations

  preprintOpen accessSenior author

  Learning therapeutic counseling involves significant role-play experience with mock patients, with current manual training methods providing only intermittent granular feedback. We seek to accelerate and optimize counselor training by providing frequent, detailed feedback to trainees as they interact with a simulated patient. Our first application domain involves training motivational interviewing skills for counselors. Motivational interviewing is a collaborative counseling style in which patients are guided to talk about changing their behavior, with empathetic counseling an essential ingredient. We developed and evaluated an LLM-powered training system that features a simulated patient and visualizations of turn-by-turn performance feedback tailored to the needs of counselors learning motivational interviewing. We conducted an evaluation study with professional and student counselors, demonstrating high usability and satisfaction with the system. We present design implications for the development of automated systems that train users in counseling skills and their generalizability to other types of social skills training.

  PublisherDOI

• Online Health Information–Seeking in the Era of Large Language Models: Cross-Sectional Web-Based Survey Study

  Journal of Medical Internet Research · 2025-03-01 · 65 citations

  articleOpen accessSenior author

  BACKGROUND: As large language model (LLM)-based chatbots such as ChatGPT (OpenAI) grow in popularity, it is essential to understand their role in delivering online health information compared to other resources. These chatbots often generate inaccurate content, posing potential safety risks. This motivates the need to examine how users perceive and act on health information provided by LLM-based chatbots. OBJECTIVE: This study investigates the patterns, perceptions, and actions of users seeking health information online, including LLM-based chatbots. The relationships between online health information-seeking behaviors and important sociodemographic characteristics are examined as well. METHODS: A web-based survey of crowd workers was conducted via Prolific. The questionnaire covered sociodemographic information, trust in health care providers, eHealth literacy, artificial intelligence (AI) attitudes, chronic health condition status, online health information source types, perceptions, and actions, such as cross-checking or adherence. Quantitative and qualitative analyses were applied. RESULTS: Most participants consulted search engines (291/297, 98%) and health-related websites (203/297, 68.4%) for their health information, while 21.2% (63/297) used LLM-based chatbots, with ChatGPT and Microsoft Copilot being the most popular. Most participants (268/297, 90.2%) sought information on health conditions, with fewer seeking advice on medication (179/297, 60.3%), treatments (137/297, 46.1%), and self-diagnosis (62/297, 23.2%). Perceived information quality and trust varied little across source types. The preferred source for validating information from the internet was consulting health care professionals (40/132, 30.3%), while only a very small percentage of participants (5/214, 2.3%) consulted AI tools to cross-check information from search engines and health-related websites. For information obtained from LLM-based chatbots, 19.4% (12/63) of participants cross-checked the information, while 48.4% (30/63) of participants followed the advice. Both of these rates were lower than information from search engines, health-related websites, forums, or social media. Furthermore, use of LLM-based chatbots for health information was negatively correlated with age (ρ=-0.16, P=.006). In contrast, attitudes surrounding AI for medicine had significant positive correlations with the number of source types consulted for health advice (ρ=0.14, P=.01), use of LLM-based chatbots for health information (ρ=0.31, P<.001), and number of health topics searched (ρ=0.19, P<.001). CONCLUSIONS: Although traditional online sources remain dominant, LLM-based chatbots are emerging as a resource for health information for some users, specifically those who are younger and have a higher trust in AI. The perceived quality and trustworthiness of health information varied little across source types. However, the adherence to health information from LLM-based chatbots seemed more cautious compared to search engines or health-related websites. As LLMs continue to evolve, enhancing their accuracy and transparency will be essential in mitigating any potential risks by supporting responsible information-seeking while maximizing the potential of AI in health contexts.

  PublisherDOI

• A mobile relational agent to enhance atrial fibrillation self-care: Primary and secondary outcomes of a randomized controlled trial

  American Heart Journal · 2025-06-20 · 1 citations

  article
  PublisherDOI

• HealthDial: A No-Code LLM-Assisted Dialogue Authoring Tool for Healthcare Virtual Agents

  ArXiv.org · 2025-09-10

  preprintOpen accessSenior author

  We introduce HealthDial, a dialogue authoring tool that helps healthcare providers and educators create virtual agents that deliver health education and counseling to patients over multiple conversations. HealthDial leverages large language models (LLMs) to automatically create an initial session-based plan and conversations for each session using text-based patient health education materials as input. Authored dialogue is output in the form of finite state machines for virtual agent delivery so that all content can be validated and no unsafe advice is provided resulting from LLM hallucinations. LLM-drafted dialogue structure and language can be edited by the author in a no-code user interface to ensure validity and optimize clarity and impact. We conducted a feasibility and usability study with counselors and students to test our approach with an authoring task for cancer screening education. Participants used HealthDial and then tested their resulting dialogue by interacting with a 3D-animated virtual agent delivering the dialogue. Through participants' evaluations of the task experience and final dialogues, we show that HealthDial provides a promising first step for counselors to ensure full coverage of their health education materials, while creating understandable and actionable virtual agent dialogue with patients.

  PublisherOA PDFDOI

• The mobile health intervention for rural patients with atrial fibrillation a randomized controlled trial

  International Journal of Cardiology · 2025-07-01 · 1 citations

  articleOpen accessSenior author
  PublisherOA PDFDOI

Recent grants

• NIH Grant R01HL116448

  NIH · $3.0M · 2018

• HCC: Large: Collaborative Research: Always-On Relational Agents for Social Support of Older Adults

  NSF · $726k · 2010–2016

• CAREER: A Virtual Laboratory for Studying Long-Term Human-Computer Relationships

  NSF · $500k · 2006–2012

• CHS: Medium: Transforming Scientific Presentations with Co-Presenter Agents

  NSF · $1.2M · 2015–2021

• SCC: Smart and Connected Churches for Promoting Health in Disadvantaged Populations

  NSF · $2.1M · 2018–2024

Frequent coauthors

• Michael K. Paasche‐Orlow

  Tufts Medical Center

  112 shared

• Brian W. Jack

  99 shared

• Elizabeth Limakatso Nkabane-Nkholongo

  Sefako Makgatho Health Sciences University

  66 shared

• David M. Thompson

  University of Oklahoma Health Sciences Center

  65 shared

• Daniel Schulman

  Christus Health

  40 shared

• Everlyne Kimani

  Toyota Industries (United States)

  38 shared

• Stefán Ólafsson

  Reykjavík University

  38 shared

• Hye Sun Yun

  University of Chicago

  36 shared

Awards & honors

• NSF CAREER Award