About

Craig Zilles is a Professor in the Computer Science department at the University of Illinois at Urbana-Champaign and a Severns Faculty Scholar. His current research focuses on applying computing and data analytics to education, including the development of the Computer-based Testing Facility (CBTF). Historically, his research has concentrated on the interaction between compilers and computer architecture, especially in the context of managed and dynamic languages. He received his Ph.D. in 2002 from Wisconsin-Madison for his work with Guri Sohi on Speculative Slices and Master/Slave Speculative Parallelization. Prior to his work on computer architecture and compilers, he developed the first algorithm that allowed rendering arbitrary three-dimensional polygonal shapes for haptic interfaces, which are force-feedback human-computer interfaces. He has been widely recognized for excellent teaching at the undergraduate level, receiving numerous awards including the campus Excellence in Undergraduate Teaching in 2018, the IEEE Education Society's 2010 Mac Van Valkenburg Early Career Teaching Award, the College of Engineering's Rose Award and Everitt Award for Teaching Excellence, and the Illinois Student Senate Teaching Excellence Award. Additionally, he holds five patents, is a recipient of an NSF CAREER award, and his research has been recognized through best paper awards from ASPLOS in 2010 and 2013, as well as being included in the IEEE Micro Top Picks from the 2008 Computer Architecture Conferences.

Research topics

• Computer Science
• Natural Language Processing
• Medical education
• Artificial Intelligence
• Psychology
• Programming language
• Mathematics education
• Medicine
• Software engineering
• Linguistics
• Engineering
• Information Retrieval
• Multimedia
• Statistics
• Pedagogy
• World Wide Web
• Mathematics

Selected publications

• Enabling Open Educational Resource Adoption through Integrated Sharing in PrairieLearn

  2026-02-13

  articleOpen access

  This paper introduces the PrairieLearn Question Sharing System (PQSS), which enables instructors to share question generators with other instructors, either as open educational resources or privately. PQSS is integrated into PrairieLearn, an open-source, problem-driven online learning platform. PQSS addresses a critical need for more open-source assessments by making it easier for instructors to share assessments and for instructors to use those assessments. Instructors often do not share questions due to the time it takes to publish them and the lack of recognition for their work. Because it is directly integrated into PrairieLearn, PQSS reduces the aforementioned friction of sharing and using shared questions, and we can report usage statistics to help question authors receive recognition for their work. In this paper, we share design and implementation details of the system, as well as experiences using it to share course content across courses and between universities.

  PublisherDOI

• Actually Achieving "A's for All" (as Time and Interest Allow)

  2026-02-13

  articleOpen access
  PublisherDOI

• Investigating the Impact of Automated Code Quality Feedback in an Embedded Systems Course

  2025-11-06

  articleOpen accessSenior author

  In this paper, we examine the effect of providing automatic continuous code-quality feedback in a practical, embedded systems course. We conducted a mixed-methods quasi-experimental study, analyzing data from eight individual lab assessments and three group projects in two cohorts: a control group that did not receive automatic continuous code-quality feedback (N=35) and a treatment group that did (N=36). We computed two metrics based on the number and frequency of defects during students’ code development and interviewed students from the treatment group at the end of the semester.

  PublisherDOI

• Do Centralized Testing Centers Influence Test Anxiety forEngineering Students?

  2025-08-21

  article
  PublisherDOI

• Exploring Code Quality in Embedded Systems Education

  2025-03-05

  preprintOpen access

  This study investigates code quality in embedded systems, particularly in an educational context. We highlight five common mistakes frequently recognized by experts yet absent from traditional software engineering quality criteria. By examining bare-metal and RTOS firmware, we assess how these issues affect code quality and analyze their prevalence among students in two embedded systems course cohorts.

  PublisherOA PDFDOI

• Measuring Test Anxiety of Two Computerized Exam Approaches

  2025-02-12

  articleOpen access

  Computerized exams have benefits for large enrollment courses and computer science classes, specifically. In this research paper, we compare student self-reported test anxiety between two modes of administering computerized exams: a computer-based testing facility (CBTF) and a bring-your-own-device (BYOD) setup. We conducted crossover design experiments in two computer science courses, measuring trait anxiety, as well as students' test anxiety and their test performance after each exam.

  PublisherDOI

• On Teaching Novices Computational Thinking by Utilizing Large Language Models Within Assessments

  2025-02-12 · 5 citations

  articleOpen accessSenior author

  Novice programmers often struggle to develop computational thinking (CT) skills in introductory programming courses. This study investigates the use of Large Language Models (LLMs) to provide scalable, strategy-driven feedback to teach CT. Through think-aloud interviews with 17 students solving code comprehension and writing tasks, we found that LLMs effectively guided decomposition and program development tool usage. Challenges included students seeking direct answers or pasting feedback without considering suggested strategies. We discuss how instructors should integrate LLMs into assessments to support students' learning of CT.

  PublisherDOI

• Exploring Different Specifications Grading Policies

  2025-02-12 · 1 citations

  articleOpen access

  Alternative grading practices, such as specifications grading, have been reported to increase student engagement, decrease instructor workload, and ensure a minimum mastery of all course topics. However, there are also reports of decreased student learning and there have been few comparisons between different policies. In this experience report, we explore the effects of adopting different types of specifications grading systems in a project-based Embedded Systems course taught at a South American institution. We tested three different grading policies for exams in three different course offerings, varying both the number of times mastery needed to be demonstrated and whether exams specified two (pass/fail) or five (I, D, C, B, A) proficiency levels. For each policy, we list potential problems we were trying to address and evaluate how the change impacted the course. We use a mixed-methods approach to do a retrospective analysis, focusing on three aspects: (i)exam attendance and pass rates,(ii) grades on labs and projects and, (iii) student perceptions. We find that attendance remains high and pass rates increase with the pass/fail model. Also, lab and project grades decrease after students receive a passing grade in the first exam, indicating a possible decrease in engagement with the course. In terms of student perception, students described a mix of procrastination, low engagement with the course, and a desire to learn and create a challenging project. Finally, we discuss how we believe these findings could be used to increase engagement and learning.

  PublisherOA PDFDOI

• Experiences with Computer-Based Testing (CBT)

  2025-02-18

  articleSenior author

  Delivery of affordable, secure, and scalable assessments is an essential component of large university courses, whether online or in-person. The transition to Computer-Based Testing (CBT) has a transformational effect on pedagogy. Modern CBT systems provide almost unlimited flexibility in the types of questions they can support for manual grading and autograding. In this BoF, faculty interested in learning about various components of CBT and how to implement it at their institution are invited to ask their questions and learn from others who have already done this. To facilitate these discussions, in this BOF we will break into four smaller groups to discuss CBT pedagogy, building and sharing question banks, technical or logistical considerations, and building buy-in from all levels of the institution.

  PublisherDOI

• ReDefining Code Comprehension: Function Naming as a Mechanism for Evaluating Code Comprehension

  2025-06-13 · 1 citations

  articleOpen accessSenior author

  ''Explain in Plain English'' (EiPE) questions are widely used to assess code comprehension skills but are challenging to grade automatically. Recent approaches like Code Generation Based Grading (CGBG) leverage large language models (LLMs) to generate code from student explanations and validate its equivalence to the original code using unit tests. However, this approach does not differentiate between high-level, purpose-focused responses and low-level, implementation-focused ones, limiting its effectiveness in assessing comprehension level. We propose a modified approach where students generate function names, emphasizing the function's purpose over implementation details. We evaluate this method in an introductory programming course and analyze it using Item Response Theory (IRT) to assess the difficulty and discrimination of function naming exercises as exam items and to compare their alignment with traditional EiPE grading standards. We also publish this work as an open source Python package for auto-grading EiPE questions, providing a scalable solution for adoption.

  PublisherDOI

Recent grants

• CAREER: A Framework for Dynamic Self-Tuning of General Purpose Programs

  NSF · $410k · 2004–2009

• Helping Improve and Scale Introductory Programming Courses through Automated Code-Reading Exercises

  NSF · $300k · 2021–2024

• Exploring Second-Chance Testing as a Practical Form of Mastery Learning

  NSF · $600k · 2019–2023

• Investigating the Effects of a Mastery-based Assessment Approach on Undergraduate Engineering Education across Multiple Engineering Courses and Universities

  NSF · $200k · 2020–2023

• Collaborative Research: Development of Concept Inventories for Computer Science

  NSF · $285k · 2006–2010

Frequent coauthors

• Matthew West

  University of Illinois Urbana-Champaign

  95 shared

• Jacob Bailey

  Kwantlen Polytechnic University

  38 shared

• Geoffrey Herman

  Lahore University of Management Sciences

  24 shared

• David Mussulman

  University of Illinois Urbana-Champaign

  21 shared

• Chris Wilkerson

  Intel (United States)

  18 shared

• Engin İpek

  Qualcomm (United Kingdom)

  18 shared

• David Brooks

  18 shared

• Yuanyuan Zhou

  17 shared

Education

• Ph.D., Computer Science

  University of Illinois at Urbana-Champaign

  2000

• M.S., Computer Science

  University of Illinois at Urbana-Champaign

  1996

• B.S., Computer Science

  University of Illinois at Urbana-Champaign

  1994

Awards & honors

• Excellence in Undergraduate Teaching (2018)
• IEEE Education Society's Mac Van Valkenburg Early Career Tea…
• College of Engineering's Rose Award for Teaching Excellence
• Everitt Award for Teaching Excellence
• Illinois Student Senate Teaching Excellence Award