About

R. Luke DuBois is a composer, artist, and performer who explores the temporal, verbal, and visual structures of cultural and personal ephemera. He holds a doctorate in music composition from Columbia University and has lectured and taught worldwide on interactive sound and video performance. His work stems from investigations of 'time-lapse phonography,' revealing the average sonority, visual language, and vocabulary in music, film, text, or cultural information, similar to how long camera exposures fuse motion into a single image. DuBois has collaborated with numerous artists and organizations, including Toni Dove, Todd Reynolds, Jamie Jewett, Bora Yoon, Michael Joaquin Grey, Matthew Ritchie, Elliott Sharp, Michael Gordon, Maya Lin, Bang on a Can, Engine 27, Harvestworks, and LEMUR, and was the director of the Princeton Laptop Orchestra for its 2007 season. His work has been exhibited internationally at venues such as the Institut Valencià d’Art Modern in Spain, Haus der elektronischen Künste in Switzerland, the Smithsonian American Art Museum, and at major festivals including Sundance and Sydney Film Festival. He has also been featured in prominent publications like the New York Times, National Geographic, and Esquire Magazine, and was an invited speaker at the 2016 TED Conference. As a visual and musical collaborator, DuBois is the co-author of Jitter, a software suite for real-time manipulation of matrix data, and appears on nearly twenty-five albums both solo and with the avant-garde electronic group The Freight Elevator Quartet. Currently, he performs as part of Bioluminescence and Fair Use, exploring vocal modalities and electronic remixing of cinema. He is the director of the Brooklyn Experimental Media Center at NYU Tandon School of Engineering and serves on the Board of Directors of the ISSUE Project Room. His artwork is represented by bitforms gallery in New York City, and his research interests include digital media, human-computer interaction, cyber-physical systems, computer music, rehabilitation engineering, and emerging media education.

Research topics

• Computer Science
• Multimedia
• Human–computer interaction
• Visual arts
• Art
• Medicine
• World Wide Web
• Mathematics
• Speech recognition
• Computer graphics (images)

Selected publications

• Empowering Blind Musicians to Compose and Notate Music with SoundCells

  2022 · 14 citations

  Senior authorCorresponding
  • Computer Science
  • Computer Science
  • Speech recognition

  Commercial technologies for notating music pose usage barriers to blind and visually impaired (BVI) musicians because they use graphic user interfaces and only produce visual, print scores. However, more research to date has studied how to make existing scores available in braille or large print rather than understand the needs and workflows of BVI musicians who notate new music. To address this gap, we conducted a six-week remote study in which six BVI musicians with wide-ranging backgrounds wrote original music culminating in a live performance. To create their scores, participants used SoundCells, a product of ongoing co-design and testing with BVI musicians that uses text to generate audio, print, and braille music. Across three interviews, participants offered diverse and nuanced views of how text input could facilitate creative expression. We uncovered how vision ability, music experience, and assistive technology preference affected how music was accessed and traversed. From this research, we provide design recommendations for improving SoundCells’ input and output systems, discuss how visual cues embedded in SoundCells’ syntax make learning and remembering harder for people who can’t view it, and reflect on how our chosen methods resulted in high engagement.

  PublisherDOI

• SoundCells

  2022 · 14 citations

  • Computer Science
  • Computer Science
  • Multimedia

  Technologies for notating music pose usage barriers to blind and visually impaired musicians requiring many to overcome a significant learning curve and/or rely on complicated tool chains with limited screen reader support. To address a need for accessible music notation software, we present SoundCells, a browser-based system designed to make music notation easy, intuitive, and accessible to screen reader users, and output music in audio, print, and braille formats. We share findings from a co-design process, in which two experienced musicians used SoundCells for two months guided by four remote meetings, and from a Design Probe, in which five other musicians tried SoundCells with a screen reader and reflected on its usability and accessibility in the context of their current practices. Finally, we discuss design recommendations relevant to a broader ecosystem of creative technologies, including how text-editing and multi-modal output capabilities could be extended and improved, how SoundCells' current design facilitated remote collaboration between sighted researchers and blind musicians, and future opportunities for learning and sharing music on the web.

  PublisherDOI

• Making Math Graphs More Accessible in Remote Learning: Using Sonification to Introduce Discontinuity in Calculus

  2021 · 17 citations

  Senior authorCorresponding
  • Computer Science
  • Computer Science
  • Multimedia

  Math graphs need to be accessible to People with Visual Impairments (PVI). While tactile graphics are a common way for PVI to access math graphs, their use becomes complicated in remote learning. To make math graphs more accessible in remote education, we focused on sonification, the use of non-speech sound. In this study, we designed techniques of sonification of math graphs to introduce the concept of discontinuity in calculus to PVI. First, we conducted a remote interview with six participants to understand their experiences with math education using graphs. Based on these findings, we developed a series of sonifications of math graphs that we remotely evaluated with three participants from our initial interviews. Our findings reveal that sonification can intuitively convey simple patterns and trends in math graphs with a little practice, be useful to introduce the discontinuities, and be more effective with descriptions of the sound and graphs.

  PublisherDOI

• SONYC

  Communications of the ACM · 2019-01-28 · 190 citations

  article

  SONYC integrates sensors, machine listening, data analytics, and citizen science to address noise pollution in New York City.

  PublisherDOI

• SONYC: A System for the Monitoring, Analysis and Mitigation of Urban Noise Pollution

  arXiv (Cornell University) · 2018-05-02 · 37 citations

  preprintOpen access

  We present the Sounds of New York City (SONYC) project, a smart cities initiative focused on developing a cyber-physical system for the monitoring, analysis and mitigation of urban noise pollution. Noise pollution is one of the topmost quality of life issues for urban residents in the U.S. with proven effects on health, education, the economy, and the environment. Yet, most cities lack the resources to continuously monitor noise and understand the contribution of individual sources, the tools to analyze patterns of noise pollution at city-scale, and the means to empower city agencies to take effective, data-driven action for noise mitigation. The SONYC project advances novel technological and socio-technical solutions that help address these needs. SONYC includes a distributed network of both sensors and people for large-scale noise monitoring. The sensors use low-cost, low-power technology, and cutting-edge machine listening techniques, to produce calibrated acoustic measurements and recognize individual sound sources in real time. Citizen science methods are used to help urban residents connect to city agencies and each other, understand their noise footprint, and facilitate reporting and self-regulation. Crucially, SONYC utilizes big data solutions to analyze, retrieve and visualize information from sensors and citizens, creating a comprehensive acoustic model of the city that can be used to identify significant patterns of noise pollution. These data can be used to drive the strategic application of noise code enforcement by city agencies to optimize the reduction of noise pollution. The entire system, integrating cyber, physical and social infrastructure, forms a closed loop of continuous sensing, analysis and actuation on the environment. SONYC provides a blueprint for the mitigation of noise pollution that can potentially be applied to other cities in the US and abroad.

  PublisherOA PDFDOI

• 2012: To Be Inside Someone Else’s Dream: Music for Sleeping and Waking Minds

  Current research in systematic musicology · 2017-01-01

  book-chapterSenior author
  PublisherDOI

• ARIES: Enabling Visual Exploration and Organization of Art Image Collections

  IEEE Computer Graphics and Applications · 2017-10-05 · 25 citations

  articleOpen access

  Art historians have traditionally used physical light boxes to prepare exhibits or curate collections. On a light box, they can place slides or printed images, move the images around at will, group them as desired, and visual-ly compare them. The transition to digital images has rendered this workflow obsolete. Now, art historians lack well-designed, unified interactive software tools that effectively support the operations they perform with physi-cal light boxes. To address this problem, we designed ARIES (ARt Image Exploration Space), an interactive image manipulation system that enables the exploration and organization of fine digital art. The system allows images to be compared in multiple ways, offering dynamic overlays analogous to a physical light box, and sup-porting advanced image comparisons and feature-matching functions, available through computational image processing. We demonstrate the effectiveness of our system to support art historians tasks through real use cases.

  PublisherOA PDFDOI

• I-VEST: Intelligent visual email search and triage

  2014-10-01

  articleSenior author

  Lawyers and investigators are often presented with large email datasets that contain emails that are not all relevant to any given investigative search. They must often manually comb through information contained within these large datasets in order to find the information they need, expending large amounts of time and money in the process. Our work offers an interactive visual analytic alternative to current methodology. We introduce a method for reducing the number of emails that need to be viewed in a large dataset while also giving the user a quick overview of possible contents and relationships in a set of results.

  PublisherDOI

• To be inside someone else's dream: On Music for Sleeping & Waking Minds

  2012-01-01 · 2 citations

  articleSenior author

• To be inside someone else’s dream: On music for sleeping and waking minds

  2012-01-01 · 1 citations

  article1st authorCorresponding
  Publisher

Frequent coauthors

• Raymond Knapp

  4 shared

• Eric Lyon

  Virginia Tech

  4 shared

• Gascia Ouzounian

  4 shared

• Nasir Memon

  New York University

  3 shared

• Amy Hurst

  New York University

  3 shared

• Claudio Silva

  3 shared

• Charles Mydlarz

  New York University

  2 shared

• Fabiha Ahmed

  2 shared