Efficiency of Learning from Proof Blocks Versus Writing Proofs

2023 · 12 citations

• Computer Science
• Computer Science
• Mathematics education

Proof Blocks is a software tool that provides students with a scaffolded proof-writing experience, allowing them to drag and drop prewritten proof lines into the correct order instead of starting from scratch. In this paper we describe a randomized controlled trial designed to measure the learning gains of using Proof Blocks for students learning proof by induction. The study participants were 332 students recruited after completing the first month of their discrete mathematics course. Students in the study took a pretest and read lecture notes on proof by induction, completed a brief (less than 1 hour) learning activity, and then returned one week later to complete the posttest. Depending on the experimental condition that each student was assigned to, they either completed only Proof Blocks problems, completed some Proof Blocks problems and some written proofs, or completed only written proofs for their learning activity. We find that students in the early phases of learning about proof by induction are able to learn just as much from reading lecture notes and using Proof Blocks as by reading lecture notes and writing proofs from scratch, but in far less time on task. This finding complements previous findings that Proof Blocks are useful exam questions and are viewed positively by students.

Analyzing Student SQL Solutions via Hierarchical Clustering and Sequence Alignment Scores

2022 · 11 citations

• Computer Science
• Computer Science
• Data Mining

Structured Query Language (SQL), the de facto standard language for relational database systems management, proves to be a vital skill for a wide array of users, developers, and researchers who interact with databases. Given that there are many diverse ways for people to acquire SQL as a skill set, and various methods to write semantically equivalent SQL queries, this presents to us both the challenge and opportunity of understanding how students learn SQL as they work on homework assignment questions. In this paper, we analyze students’ SQL submissions to the homework assignment problems of the Database Systems course available to upper-level undergraduate and graduate students at the University of Illinois at Urbana-Champaign. For each student, we compute the sequence alignment scores between every submission and their final submission to understand how students reached their final solution, and whether there were any obstacles in their learning process. We also utilize hierarchical clustering techniques to create a class-wide aggregate view to determine the number of different approaches used by students in the course. We compute the resulting dendrogram visualization based upon students’ final attempt to a homework problem. Our system enables instructors with more visibility to identify interesting learning patterns and approaches. These findings aim at supporting instructors to target their instruction in difficult SQL areas for the future so students may learn SQL more effectively.

Analyzing Patterns in Student SQL Solutions via Levenshtein Edit Distance

2021 · 13 citations

• Computer Science
• Computer Science
• Database

Structured Query Language (SQL), the standard language for relational database management systems, is an essential skill for software developers, data scientists, and professionals who need to interact with databases. SQL is highly structured and presents diverse ways for learners to acquire this skill. However, despite the significance of SQL to other related fields, little research has been done to understand how students learn SQL as they work on homework assignments. In this paper, we analyze students' SQL submissions to homework problems of the Database Systems course offered at the University of Illinois at Urbana-Champaign. For each student, we compute the Levenshtein Edit Distances between every submission and their final submission to understand how students reached their final solution and how they overcame any obstacles in their learning process. Our system visualizes the edit distances between students' submissions to a SQL problem, enabling instructors to identify interesting learning patterns and approaches. These findings will help instructors target their instruction in difficult SQL areas for the future and help students learn SQL more effectively.

How engineering students use domain knowledge when problem‐solving using different visual representations

Journal of Engineering Education · 2020 · 17 citations

• Computer Science
• Artificial Intelligence
• Computer Science

Abstract Background Engineering students commonly learn domain knowledge by engaging with visual representations of it. However, at times they have trouble accessing information from these representations due to the way information is encoded in features of the representation. Purpose To describe how students engage with representation features, we explored two research questions: (a) what is the interplay between how concepts are encoded within representations, students' use of those concepts, and how students translate between representations during their problem‐solving and (b) how is the interplay described in Research Question 1 similar and different across students in statics versus those in digital logic? Design/Method We synthesized findings from two of our prior research studies using the constant comparative method. We described the effect of representations on students' ability to access and use domain knowledge during problem‐solving within and across engineering disciplines. Results We identified three themes that describe how visual representations affect students’ reasoning. First, students conflated concepts that were represented using similar features. Second, students consistently failed to use concepts that were not represented with visually salient features. Third, statics students coordinated multiple representations when translating between representations during problem‐solving more often than the digital logic students. Conclusions These themes provide possible domain‐general pathways for redesigning the notation and representations that we use to teach engineering concepts and suggest future avenues of research to further explore the generality of these findings.

Insights from Student Solutions to SQL Homework Problems

2020 · 30 citations

• Computer Science
• Computer Science
• Natural Language Processing

We analyze the submissions of 286 students as they solved Structured Query Language (SQL) homework assignments for an upper-level databases course. Databases and the ability to query them are becoming increasingly essential for not only computer scientists but also business professionals, scientists, and anyone who needs to make data-driven decisions. Despite the increasing importance of SQL and databases, little research has documented student difficulties in learning SQL. We replicate and extend prior studies of students' difficulties with learning SQL. Students worked on and submitted their homework through an online learning management system with support for autograding of code. Students received immediate feedback on the correctness of their solutions and had approximately a week to finish writing eight to ten queries. We categorized student submissions by the type of error, or lack thereof, that students made, and whether the student was eventually able to construct a correct query. Like prior work, we find that the majority of student mistakes are syntax errors. In contrast with the conclusions of prior work, we find that some students are never able to resolve these syntax errors to create valid queries. Additionally, we find that students struggle the most when they need to write SQL queries related to GROUP BY and correlated subqueries. We suggest implications for instruction and future research.

When am I ever going to use this? An investigation of the calculus content of core engineering courses

Journal of Engineering Education · 2020 · 42 citations

• Computer Science
• Mathematics education
• Computer Science

Abstract Background Many engineering students fail to proceed through required prerequisite mathematics courses. Since these courses strongly influence engineering student attrition, we should examine to what degree these courses truly serve as prerequisites for following engineering coursework. Purpose/Hypothesis We examined two research questions: Which concepts and skills learned in calculus are applied in engineering statics and circuits homework assignments? How are calculus skills applied in engineering statics and circuits homework assignments? Design/Method This study analyzes the homework problems of two engineering courses—statics and circuits for nonmajors—using the mathematics‐in‐use method. These courses were chosen since they often require calculus as a direct prerequisite and are taken by most engineering majors. The mathematical content of each homework problem is carefully analyzed, with attention to alternative solution paths that may not match the instructor solution. Results Only 8% of statics problems and 20% of circuits for nonmajors problems applied calculus. Furthermore, these problems applied only the simplest calculus skills (e.g., integration of polynomials). Conclusions Circuits and statics apply relatively little calculus; most problems consist primarily of algebra. We may be able to modify prerequisite structures to ease or speed student progress.

Comparison of Grade Replacement and Weighted Averages for Second-Chance Exams

2020 · 27 citations

• Computer Science
• Mathematics education
• Computer Science

We explore how course policies affect students' studying and learning when a second-chance exam is offered. High-stakes, one-off exams remain a de facto standard for assessing student knowledge in STEM, despite compelling evidence that other assessment paradigms such as mastery learning can improve student learning. Unfortunately, mastery learning can be costly to implement. We explore the use of optional second-chance testing to sustainably reap the benefits of mastery-based learning at scale. Prior work has shown that course policies affect students' studying and learning but have not compared these effects within the same course context. We conducted a quasi-experimental study in a single course to compare the effect of two grading policies for second-chance exams and the effect of increasing the size of the range of dates for students taking asynchronous exams. The first grading policy, called 90-cap, allowed students to optionally take a second-chance exam that would fully replace their score on a first-chance exam except the second-chance exam would be capped at 90% credit. The second grading policy, called 90-10, combined students' first- and second-chance exam scores as a weighted average (90% max score + 10% min score). The 90-10 policy significantly increased the likelihood that marginally competent students would take the second-chance exam. Further, our data suggests that students learned more under the 90-10 policy, providing improved student learning outcomes at no cost to the instructor. Most students took exams on the last day an exam was available, regardless of how many days the exam was available.