About

Shamya Karumbaiah is an assistant professor in Educational Psychology at the University of Wisconsin-Madison. She completed her PhD at the University of Pennsylvania in 2021, where she worked as a research fellow in the Penn Center for Learning Analytics with Dr. Ryan Baker. Following her PhD, she was a postdoctoral fellow at Carnegie Mellon University in the Human-Computer Interaction Institute, collaborating with Dr. Vincent Aleven and Dr. Nikol Rummel. Her research focuses on promoting student engagement and learning within adaptive and artificially intelligent educational systems, emphasizing fairness and equity. Her work lies at the intersection of machine learning and learning sciences, involving the development of statistical and machine learning models to understand complex educational constructs such as affect, cognition, motivation, self-identity, help-seeking, and persistence. She also pursues methodological innovations in learning analytics to address challenges in applying current methods to education data, with a critical focus on identifying and mitigating biases to ensure equitable student outcomes. Prior to her doctoral studies, she earned an MS in computer science from UMass Amherst with an emphasis on machine learning, working with Dr. Beverly P. Woolf and Dr. Ivon Arroyo. In 2016, she was a visiting researcher in learning sciences at USC ICT, collaborating with Dr. Benjamin Nye and Dr. Mark Core. She also interned as a data scientist at Cisco's Advanced Technologies and AI lab in 2017 and worked as a software engineer at Cisco from 2011 to 2015 after earning a BE in Computer Science from SJCE India.

Research topics

• Computer Science
• Sociology
• Engineering
• Computer Security
• Artificial Intelligence
• Psychology
• Epistemology
• World Wide Web
• Social psychology
• Engineering ethics
• Internet privacy
• Mathematics education
• Economics
• Data science

Selected publications

• Authenticity or Alienation: Latine Students’ Perceptions of Stereotypes in AI-Generated Educational Materials

  Lecture notes in computer science · 2025-01-01

  book-chapterSenior author
  PublisherDOI

• A Comparative Analysis of LLM and Specialized NLP System for Automated Assessment of Science Content

  Lecture notes in computer science · 2025-01-01

  book-chapter
  PublisherDOI

• Responsible AI in Education: Understanding Teachers’ Priorities and Contextual Challenges

  2025-06-23 · 2 citations

  articleOpen access
  PublisherOA PDFDOI

• Prompt vs. Supervise: A Case of Using Language Models to Assess Students’ Science Explanations

  Lecture notes in computer science · 2025-09-01

  book-chapter1st authorCorresponding
  PublisherDOI

• Bias or Insufficient Sample Size? Improving Reliable Estimation of Algorithmic Bias for Minority Groups

  2025-02-21 · 1 citations

  articleOpen access
  PublisherDOI

• AIBAT: AI Behavior Analysis Tool for Teacher-Driven Contextual Evaluation of Language Models in Education

  Lecture notes in computer science · 2025-01-01

  book-chapter1st authorCorresponding
  PublisherDOI

• Exploring Out-of-Tutor Interventions by Teachers in Supporting Student Learning with AI Tutors

  Proceedings. · 2025-06-10

  articleOpen access1st authorCorresponding

  Learning analytics has embraced the complexities of modeling student learning and engagement in ecological settings like K12 classrooms.While analyzing AI tutor interaction logs remains common, there is a growing need to include events that happen outside the AI tutor.Studies indicate teachers play active roles when students use AI tutors, and teacher support tools improve learning outcomes.However, methodological challenges exist in studying how such out-of-tutor interventions in the classroom shape learning (e.g., no counterfactual data, ethical issues in collecting control data).We propose a simple reconceptualization of the problem with which we could use standard statistical approaches to quantify the relationship between out-of-tutor interventions and student learning in the AI tutor.We study how moments of teacher help relate to improvements in students' performance on the skill or concept they were helped with.Our analysis reveals that teacher help has a significantly positive effect.

  PublisherOA PDFDOI

• Teacher Perceptions on AI Support for Bi/multilingual Learners: Superpowers and Constraints

  Lecture notes in computer science · 2025-01-01

  book-chapter
  PublisherDOI

• Teacher Agency in Implementing Automated Writing Evaluation Systems in a Science Classroom: A Focus Group Study

  Lecture notes in computer science · 2025-09-02

  book-chapter
  PublisherDOI

• Evaluating Language Models Using Linguistic Variations in Multilingual Learners’ Writing: A Teacher Study

  Proceedings. · 2025-06-10

  articleOpen access

  This paper investigates teachers' perceptions on linguistic variations in bi/multilingual learners' (MLs) writing to evaluate the (in)effectiveness of Multilingual Large Language Models (MLLMs), which are artificial intelligence (AI) models that generate texts in multiple languages.Due to their inherent linguistic biases, these models often struggle to interpret MLs' linguistic variations.To address this gap, we elicit teacher feedback on prevalent linguistic variations in MLs' writing and assess how Meta Llama 3.1, a state-of-the-art MLLM, responds to these variations.Using translanguaging as a lens-the fluid use of multiple languages to convey meaning across social contexts-we propose a new approach to evaluate MLLMs in multilingual learning contexts.With the increasing prevalence of AI in K12 classrooms, this paper advocates for the inclusion of bi/multilingual educators to better align the use of AI with progressive pedagogies such as translanguaging.

  PublisherOA PDFDOI

Frequent coauthors

• Ryan S. Baker

  44 shared

• Jaclyn Ocumpaugh

  University of Pennsylvania

  29 shared

• Scott A. Crossley

  Vanderbilt University

  12 shared

• Matthew J. Labrum

  11 shared

• Mark G. Core

  Creative Technologies (United States)

  9 shared

• Kallirroi Georgila

  9 shared

• Benjamin D. Nye

  Creative Technologies (United States)

  9 shared

• Daniel Auerbach

  Maersk (Denmark)

  9 shared

Labs

• Shamya Karumbaiah's LabPI

  Promoting student engagement and learning in adaptive and artificially intelligent educational systems in a fair and equitable manner.

Awards & honors

• Nellie McKay Fellowship, University of Wisconsin–Madison, 20…
• Best Paper Nomination (as first author), ACM LAK, 2021
• Rising Stars in EECS, Massachusetts Institute of Technology,…
• Best Paper Nomination (as first author), ICQE, 2020
• Best Paper Nomination (as first author), EDM, 2018