About

Mehran Sahami is the Tencent Chair of the Computer Science Department at Stanford University and the James and Ellenor Chesebrough Professor in the School of Engineering. He holds a PhD in Computer Science from Stanford University, earned in 1999. As a Professor (Teaching) in the Computer Science department, he is also a Bass Fellow in Undergraduate Education and has previously served as the Associate Chair for Education in Computer Science. His research interests include computer science education, artificial intelligence, and ethics. Prior to joining Stanford, he was a Senior Research Scientist at Google. Sahami has contributed to the development of curricular guidelines for college programs in Computer Science through his role as co-chair of the ACM/IEEE-CS joint task force on Computer Science Curricula 2013. He has served as chair of the ACM Education Board, an elected member of the ACM Council, and has been involved in advisory roles such as serving on the Advisory Board of Code.org and being appointed by California Governor Jerry Brown to the state's Computer Science Strategic Implementation Plan Advisory Panel. He is also a faculty affiliate at the Institute for Human-Centered Artificial Intelligence (HAI) and has served on various boards and committees related to computer science education and policy.

Research topics

• Computer Science
• Engineering
• Political Science
• Engineering ethics
• Artificial Intelligence
• Sociology
• Computer Security
• Social Science
• Operating system
• Data science
• World Wide Web

Selected publications

• The GPT Surprise: Offering Large Language Model Chat in a Massive Coding Class Reduced Engagement But May Increase Adopters' Exam Performances

  2025-07-17 · 2 citations

  article

  Large language models (LLMs) are quickly being adopted in a wide range of learning experiences, especially via ubiquitous and broadly accessible chat interfaces like ChatGPT. This type of interface is readily available to students and teachers around the world. Coding education is an interesting test case, both because LLMs have strong performance on coding tasks, and because LLM-powered support tools are rapidly becoming part of the workflow of professional software engineers. To help understand the impact of generic LLM use on coding education, we conducted a large-scale randomized control trial with 5,831 students from 146 countries in an online coding class in which we provided some students with access to a chat interface with GPT-4. Under some assumptions, we estimate positive benefits on exam performance for adopters, the students who used the tool, but over all students, the advertisement of GPT-4 led to a significant average decrease in exam participation. We observe similar decreases in other forms of course engagement. However, this decrease is modulated by the student's country of origin. Offering access to LLMs to students from low human development index countries increased their exam participation rate on average. Our results suggest there may be promising benefits to using LLMs in an introductory coding class, but also potential harms for engagement, which makes their longer term impact on student success unclear. Our work highlights the need for additional investigations to help understand the potential impact of future adoption and integration of LLMs into classrooms.

  PublisherDOI

• Infinite Story

  2025-02-18 · 1 citations

  article
  PublisherDOI

• The GPT Surprise: Offering Large Language Model Chat in a Massive Coding Class Reduced Engagement but Increased Adopters Exam Performances

  2024-04-25 · 14 citations

  preprintOpen access

  Large language models (LLMs) are quickly being adopted in a wide range of learning experiences, especially via ubiquitous and broadly accessible chat interfaces like ChatGPT and Copilot. This type of interface is readily available to students and teachers around the world, yet relatively little research has been done to assess the impact of such generic tools on student learning. Coding education is an interesting test case, both because LLMs have strong performance on coding tasks, and because LLM-powered support tools are rapidly becoming part of the workflow of professional software engineers. To help understand the impact of generic LLM use on coding education, we conducted a large-scale randomized control trial with 5,831 students from 146 countries in an online coding class in which we provided some students with access to a chat interface with GPT-4. We estimate positive benefits on exam performance for adopters, the students who used the tool, but over all students, the advertisement of GPT-4 led to a significant average decrease in exam participation. We observe similar decreases in other forms of course engagement. However, this decrease is modulated by the student's country of origin. Offering access to LLMs to students from low human development index countries increased their exam participation rate on average. Our results suggest there may be promising benefits to using LLMs in an introductory coding class, but also potential harms for engagement, which makes their longer term impact on student success unclear. Our work highlights the need for additional investigations to help understand the potential impact of future adoption and integration of LLMs into classrooms.

  PublisherDOI

• The GPT Surprise: Offering Large Language Model Chat in a Massive Coding Class Reduced Engagement but Increased Adopters Exam Performances

  arXiv (Cornell University) · 2024-04-25 · 2 citations

  preprintOpen access

  Large language models (LLMs) are quickly being adopted in a wide range of learning experiences, especially via ubiquitous and broadly accessible chat interfaces like ChatGPT and Copilot. This type of interface is readily available to students and teachers around the world, yet relatively little research has been done to assess the impact of such generic tools on student learning. Coding education is an interesting test case, both because LLMs have strong performance on coding tasks, and because LLM-powered support tools are rapidly becoming part of the workflow of professional software engineers. To help understand the impact of generic LLM use on coding education, we conducted a large-scale randomized control trial with 5,831 students from 146 countries in an online coding class in which we provided some students with access to a chat interface with GPT-4. We estimate positive benefits on exam performance for adopters, the students who used the tool, but over all students, the advertisement of GPT-4 led to a significant average decrease in exam participation. We observe similar decreases in other forms of course engagement. However, this decrease is modulated by the student's country of origin. Offering access to LLMs to students from low human development index countries increased their exam participation rate on average. Our results suggest there may be promising benefits to using LLMs in an introductory coding class, but also potential harms for engagement, which makes their longer term impact on student success unclear. Our work highlights the need for additional investigations to help understand the potential impact of future adoption and integration of LLMs into classrooms.

  PublisherOA PDFDOI

• SimGrade: Using Code Similarity Measures for More Accurate Human Grading

  arXiv (Cornell University) · 2024-02-19

  preprintOpen access

  While the use of programming problems on exams is a common form of summative assessment in CS courses, grading such exam problems can be a difficult and inconsistent process. Through an analysis of historical grading patterns we show that inaccurate and inconsistent grading of free-response programming problems is widespread in CS1 courses. These inconsistencies necessitate the development of methods to ensure more fairer and more accurate grading. In subsequent analysis of this historical exam data we demonstrate that graders are able to more accurately assign a score to a student submission when they have previously seen another submission similar to it. As a result, we hypothesize that we can improve exam grading accuracy by ensuring that each submission that a grader sees is similar to at least one submission they have previously seen. We propose several algorithms for (1) assigning student submissions to graders, and (2) ordering submissions to maximize the probability that a grader has previously seen a similar solution, leveraging distributed representations of student code in order to measure similarity between submissions. Finally, we demonstrate in simulation that these algorithms achieve higher grading accuracy than the current standard random assignment process used for grading.

  PublisherOA PDFDOI

• Teaching Responsible Computing in Context

  2022-03-01 · 2 citations

  article

  Recent news and national reports have significantly increased interest in new approaches for teaching responsible computing to help students understand, evaluate, and address the social impact of existing and emerging computing technologies. This 3-hour workshop will be offered in two workshop format sessions: in-person and online. The first part of each session will introduce responsible computing and its connections to RESPECT and Cultural Competence in Computing (3C). Next, we will provide a short overview of our own work in teaching responsible computing along with frameworks, tools, and best practices. We will showcase four different approaches to teaching responsible computing across institutional settings (high school, college, university), interdisciplinary partnerships (computing, philosophy, STS, digital humanities), and instructional formats (dedicated courses, embedded lessons, design challenges, bootcamps). The workshop presentations will focus on practical advice about how to get started, available resources, securing support from administration and colleagues, and other considerations for this work. In the second half of the workshop, participants will work in small groups to co-design potential lessons based on shared topical interests, institutional settings, and/or learning objectives. Facilitators will provide guidance, recommendations, and classroom examples to help the small groups to complete draft lessons that will be disseminated among workshop participants and on the workshop website. A laptop or internet connected device is needed to participate in the small group activity. Handouts/materials will be provided on the workshop website.

  PublisherDOI

• System Error: Where Big Tech Went Wrong and How We Can Reboot

  Perspectives on Science and Christian Faith · 2022 · 41 citations

  • Computer Science
  • Computer Security
  • Computer Science

  SYSTEM ERROR: Where Big Tech Went Wrong and How We Can Reboot by Rob Reich, Mehran Sahami, and Jeremy M. Weinstein. New York: HarperCollins Publishers, 2021. 352 pages. Hardcover; $27.99. ISBN: 9780063064881. *Remember when digital technology and the internet were our favorite things? When free Facebook accounts connected us with our friends, and the internet facilitated democracy movements overseas, including the Arab Spring? So do the authors of this comprehensive book. "We shifted from a wide-eyed optimism about technology's liberating potential to a dystopian obsession with biased algorithms, surveillance capitalism, and job-displacing robots" (p. 237). *This transition has not escaped the notice of the students and faculty of Stanford University, the elite institution most associated with the rise (and sustainment) of Silicon Valley. The three authors of this book teach a popular course at Stanford on the ethics and politics of technological change, and this book effectively brings their work to the public. Rob Reich is a philosopher who is associated with Stanford's Institute for Human-Centered Artificial Intelligence as well as their Center for Ethics in Society. Mehran Sahami is a computer science professor who was with Google during the startup years. Jeremy Weinstein is a political science professor with experience in government during the Obama administration. *The book is breathtakingly broad, explaining the main technical and business issues concisely but not oversimplifying, and providing the history and philosophy for context. It accomplishes all this in 264 pages, but also provides thirty-six pages of notes and references for those who want to dive deeper into some topics. The most important section is doubtless the last chapter dealing with solutions, which may be politically controversial but are well supported by the remainder of the book. *Modern computer processors have enormous computational power, and a good way to take advantage of that is to do optimization, the subject of the first chapter. Engineers love optimization, but not everything should be done as quickly and cheaply as possible! Optimization requires the choice of some quantifiable metric, but often available metrics do not exactly represent the true goal of an organization. In this case, optimizers will choose a proxy metric which they feel logically or intuitively should be correlated with their goal. The authors describe the problems which result when the wrong proxy is selected, and then excessive optimization drives that measure to the exclusion of other possibly more important factors. For example, social media companies that try to increase user numbers to the exclusion of other factors may experience serious side effects, such as the promotion of toxic content. *After that discussion on the pros and cons of optimization, the book dives into the effects of optimizing money. Venture capitalists (VCs) have been around for years, but recent tech booms have swelled their numbers. The methodology of Objectives and Key Results (OKR), originally developed by Andy Grove of Intel, became popular among the VCs of Silicon Valley, whose client firms, including Google, Twitter, and Uber, adopted it. OKR enabled most of the employees to be evaluated against some metric which management believed captured the essence of their job, so naturally the employees worked hard to optimize this quantity. Again, such a narrow view of the job has led to significant unexpected and sometimes unwanted side effects. *The big tech companies are threatened by legislation designed to mitigate some of the harm they have created. They have hired a great many lobbyists, and even overtly entered the political process where possible. In California, when Assembly Bill 5 reclassified many independent contractors as employees, the affected tech companies struck back with Proposition 22 to overturn the law. An avalanche of very expensive promotion of Proposition 22 resulted in its passage by a large margin. *It is well known that very few politicians have a technical background, and the authors speculate that this probably contributes to the libertarian leaning prominent in the tech industry. The authors go back in history to show how regulation has lagged behind technology and industrial practice. An interesting chapter addresses the philosophical question of whether democracy is up to the task of governing, or whether government by experts, or Plato's "philosopher kings" would be better. *Part II of the book is the longest, addressing the fairness of algorithms, privacy, automation and human job replacement, and free speech. The authors point out some epic algorithm failures, such as Amazon being unable to automate resumé screening to find the best candidates, and Google identifying Black users as gorillas. The big advances in deep learning neural nets result from clever algorithms plus the availability of very large databases, but if you've got a database showing that you've historically hired 95% white men for a position, training an algorithm with that database is hardly going to move you into a future with greater diversity. Even more concerning are proprietary black-box algorithms used in the legal system, such as for probation recommendations. Why not just let humans have the last word, and be advised by the algorithms? The authors remind us that one of the selling points of algorithmic decision making is to remove human bias; returning the humans to power returns that bias as well. *Defining fairness is yet another ethical and philosophical question. The authors give a good overview of privacy, which is protected by law in the European Union by the General Data Protection Regulation. Although there is no such federal law in America, California has passed a similar regulation called the California Consumer Privacy Act. At this point, it's too soon to evaluate the effect of such regulations. *The automation chapter is entitled "Can humans flourish in a world of smart machines?" and it covers many philosophical and ethical issues after providing a valuable summary of the current state of AI. Although machines are able to defeat humans in games like chess, go, and even Jeopardy, more useful abilities such as self-driving cars are not yet to that level. The utopian predictions of AGI (artificial general intelligence, or strong AI), in which the machine can set its own goals in a reasonable facsimile of a human, seem quite far off. But the current state of AI (weak AI) is able to perform many tasks usefully, and automation is already displacing some human labor. The authors discuss the economics, ethics, and psychology of automation, as human flourishing involves more than financial stability. The self-esteem associated with gainful employment is not a trivial thing. The chapter raises many more important issues than can be mentioned here. *The chapter on free speech also casts a wide net. Free speech as we experience it on the internet is vastly different from the free speech of yore, standing on a soap box in the public square. The sheer volume of speech today is incredible, and the power of the social media giants to edit it or ban individuals is also great. Disinformation, misinformation, and harassment are rampant, and polarization is increasing. *Direct incitement of violence, child pornography, and video of terrorist attacks are taken down as soon as the internet publishers are able, but hate speech is more difficult to define and detect. Can AI help? As with most things, AI can detect the easier cases, but it is not effective with the more difficult ones. From a regulatory standpoint, section 230 of the Communications Decency Act of 1996 (CDA 230) immunizes the platforms from legal liability due to the actions of users. Repealing or repairing CDA 230 may be difficult, but the authors make a good case that "it is realistic to think that we can pursue some commonsense reforms" (p. 225). *The final part of the book is relatively short, but addresses the very important question: "Can Democracies Rise to the Challenge?" The authors draw on the history of medicine in the US as an example of government regulation that might be used to reign in the tech giants. Digital technology does not have as long a history as medicine, so few efforts have been made to regulate it. The authors mention the Association for Computing Machinery (ACM) Software Engineering Code of Ethics, but point out that there are no real penalties for violation besides presumably being expelled from the ACM. Efforts to license software engineers have not borne fruit to date. *The authors argue that the path forward requires progress on several fronts. First, discussion of values must take place at the early stages of development of any new technology. Second, professional societies should renew their efforts to increase the professionalism of software engineering, including strengthened codes of ethics. Finally, computer science education should be overhauled to incorporate this material into the training of technologists and aspiring entrepreneurs. *The authors conclude with the recent history of attempts to regulate technology, and the associated political failures, such as the defunding of the congressional Office of Technology Assessment. It will never be easy to regulate powerful political contributors who hold out the prospect of jobs to politicians, but the authors make a persuasive case that it is necessary. China employs a very different authoritarian model of technical governance, which challenges us to show that democracy works better. *This volume is an excellent reference on the very active debate on the activities of the tech giants and their appropriate regulation. It describes many of the most relevant events of the recent past and provides good arguments for some proposed solutions. We need to be thinking and talking about these issues, and this book is a great

  PublisherDOI

• Should the AP Computer Science A Exam Switch to Using Python?

  Proceedings of the 53rd ACM Technical Symposium on Computer Science Education V. 2 · 2022-02-23

  article1st authorCorresponding

  Changing the language used to teach the AP Computer Science A course is an expensive, time-consuming, and ultimately controversial endeavor. However, it is worth raising the question from time to time to determine if such a change may now be appropriate. This panel offers a set of speakers specifically chosen to provide a diverse set of viewpoints and experiences (high school teacher, curriculum developer, exam reader, exam development committee member, university credit-granting administrator) on the question of whether the language in the AP CS A exam should be switched to Python in the near future.

  PublisherDOI

• Code in Place

  2021-03-03 · 16 citations

  articleSenior author

  Could it be the case that the number of people who want to teach computer science, and have the potential, is roughly proportional to the number of people who want to learn? During the time of COVID-19 we offered a free CS1 class to people around the world. Well-aware of the high drop-out rates reported in many massive open-access online courses (MOOCs), we augmented our course with a scalable, human-centered solution: section leading. Section leaders teach small, weekly interactive learning sessions. We hypothesize that the personalized attention adds a sense of responsibility for both student and teacher which drives learning. We recruited over 900 volunteer section leaders and more than 10,000 students in the class. To our knowledge this is the largest group of section leaders in a single CS1 course offering and the most small group interactions. The completion rate in our class was more than 10 times that usually reported for similar MOOCs. Additionally, 99% of the volunteer section leaders taught through the entire span of the course, showing the potential for large scale volunteer-driven education, and the benefit that teachers themselves derive. We also discovered the potential for replication of this model, as 34% of students in a representative-sample survey indicated they would serve as section leaders for a future offering of the course. This level of participation would be more than sufficient to field additional offerings of the course sustainably. We believe this is an intriguing case study of a model for significantly scaling human-centric CS education for all.

  PublisherDOI

• Assignments that Blend Ethics and Technology

  2020 · 31 citations

  Senior authorCorresponding
  • Computer Science
  • Computer Science
  • Engineering ethics

  With the 2018 revision of the ACM Code of Ethics and Professional Conduct, there is a growing interest in how computer science faculty can integrate these principles into the education of future practitioners. This special session illustrates one approach by highlighting assignments that blend ethics and technology. These assignments can be used in a variety of courses, including CS1, CS2, and later courses. Presenters will provide an overview of each assignment and gather feedback from the audience. All materials, including descriptions, starter files, and guidelines for instructors, will be published at https://ethics.acm.org/SIGCSE2020.

  PublisherDOI

Frequent coauthors

• Chris Piech

  Stanford University

  15 shared

• Luis Gravano

  11 shared

• Steve Roach

  Gates (United States)

  11 shared

• Panagiotis G. Ipeirotis

  New York University

  11 shared

• Daphne Koller

  8 shared

• Pat Langley

  Institute for the Study of Learning and Expertise

  7 shared

• Henry M. Walker

  Grinnell College

  6 shared

• Christine Alvarado

  University of California, San Diego

  6 shared

Labs

• Mehran Sahami's LabPI

  Not provided

Awards & honors

• Glushko Prize for Excellence in Undergraduate Research in Sy…
• Barwise Award for Distinguished Contributions to Symbolic Sy…
• Stanford Honors Thesis Prizes - Symbolic Systems
• Symbolic Systems Distinguished Teaching Award