About

Welcome to the PrivateEye lab webpage! We are a research group in CS @ Virginia Tech studying interaction in virtual/mixed/augmented reality (XR) with a particular focus on eye tracking and protecting privacy.

Research topics

• Computer Science
• Artificial Intelligence
• Computer Security
• Machine Learning
• Human–computer interaction
• Computer vision
• Data Mining
• Engineering
• Computer graphics (images)

Selected publications

• CHOP: Breaking Anonymity in XR through a Novel and Cost-effective Chain of Privacy Attacks and Differential Privacy-Based Defenses

  2026-03-21

  article

  The convergence of artificial intelligence (AI) and extended reality (XR) technologies (AIXR) promises innovative applications across many domains. However, the sensitive nature of data (e.g., eye-tracking) used in these systems also raises significant privacy concerns, as adversaries can exploit this data and these models to infer personal information. Prior research has primarily examined membership inference attacks (MIA) to leak privacy at the model-level and re-identification attacks (RDA) at the dataset-level, separately as individual attacks. While these attacks are relevant to the XR domain, launching these attacks as individual attacks is not practical and incurs more attack cost. To address this gap, we present the first comprehensive study of chain of privacy (CHOP) attacks against AIXR applications. We demonstrate how adversaries can launch such attacks with a high success rate, in a cost-effective way, by sequentially combining MIA and Attribute inference attacks (AIA) to re-identify XR users without access to raw XR data, training distributions, or model parameters. We evaluate our proposed method in realistic AIXR settings by adopting deep learning (DL)-based cybersickness detection as a representative AIXR application. Specifically, we train two state-of-the-art DL models on two open-source datasets: Simulation 2021 and VRWalking, and a new XR cybersickness dataset constructed from 34 participants via a user study. Our findings reveal that the proposed CHOP attacks pose severe risks to DL-based cybersickness detection, achieving re-identification rates of up to 94% and 97% on the open-source and the developed cross-linked datasets, respectively, underscoring the feasibility and severity of cross-dataset privacy violations. Furthermore, cost analysis reveals that the proposed CHOP attack is ≈ 2× more cost-effective than traditional individual attacks for re-identifying XR users. Finally, we propose two ε-differential privacy (DP)-enabled privacy-preserving mechanisms: Differentially Private Stochastic Gradient Descent (DPSGD) and Private Aggregation of Teacher Ensembles (PATE) to mitigate CHOP attacks. Our results show that the proposed defense reduces the re-identification rate by up to 88% and 79% while maintaining high model utility, with classification accuracies of up to 94% and 92% for the same datasets using Transformer models.

  PublisherDOI

• Rethinking Privacy Indicators in Extended Reality: Multimodal Design for Situationally Impaired Bystanders

  2025-10-08 · 2 citations

  articleOpen accessSenior author

  As Extended Reality (XR) devices become increasingly prevalent in everyday settings, they raise significant privacy concerns for bystanders: individuals in the vicinity of an XR device during its use, whom the device sensors may accidentally capture. Current privacy indicators, such as small LEDs, often presume that bystanders are attentive enough to interpret the privacy signals. However, these cues can be easily overlooked when bystanders are distracted or have limited vision. We define such individuals as situationally impaired bystanders. This study explores XR privacy indicator designs that are effective for situationally impaired bystanders. A focus group with eight participants was conducted to design five novel privacy indicators. We evaluated these designs through a user study with seven additional participants. Our results show that visual-only indicators, typical in commercial XR devices, received low ratings for perceived usefulness in impairment scenarios. In contrast, multimodal indicators were preferred in privacy-sensitive scenarios with situationally impaired bystanders. Ultimately, our results highlight the need to move toward adaptable, multimodal, and situationally aware designs that effectively support bystander privacy in everyday XR environments.

  PublisherOA PDFDOI

• Just stop doing everything for now!: Understanding security attacks in remote collaborative mixed reality

  ArXiv.org · 2025-01-27

  preprintOpen accessSenior author

  Mixed Reality (MR) devices are being increasingly adopted across a wide range of real-world applications, ranging from education and healthcare to remote work and entertainment. However, the unique immersive features of MR devices, such as 3D spatial interactions and the encapsulation of virtual objects by invisible elements, introduce new vulnerabilities leading to interaction obstruction and misdirection. We implemented latency, click redirection, object occlusion, and spatial occlusion attacks within a remote collaborative MR platform using the Microsoft HoloLens 2 and evaluated user behavior and mitigations through a user study. We compared responses to MR-specific attacks, which exploit the unique characteristics of remote collaborative immersive environments, and traditional security attacks implemented in MR. Our findings indicate that users generally exhibit lower recognition rates for immersive attacks (e.g., spatial occlusion) compared to attacks inspired by traditional ones (e.g., click redirection). Our results demonstrate a clear gap in user awareness and responses when collaborating remotely in MR environments. Our findings emphasize the importance of training users to recognize potential threats and enhanced security measures to maintain trust in remote collaborative MR systems.

  PublisherOA PDFDOI

• "Just stop doing everything for now!": Understanding security attacks in remote collaborative mixed reality

  2025-03-08 · 4 citations

  articleSenior author

  Mixed Reality (MR) devices are being increasingly adopted across a wide range of real-world applications, ranging from education and healthcare to remote work and entertainment. However, the unique immersive features of MR devices, such as 3D spatial interactions and the encapsulation of virtual objects by invisible elements, introduce new vulnerabilities leading to interaction obstruction and misdirection. We implemented latency, click redirection, object occlusion, and spatial occlusion attacks within a remote collaborative MR platform using the Microsoft HoloLens 2 and evaluated user behavior and mitigations through a user study. We compared responses to MR-specific attacks, which exploit the unique characteristics of remote collaborative immersive environments, and traditional security attacks implemented in MR. Our findings indicate that users generally exhibit lower recognition rates for immersive attacks (e.g., spatial occlusion) compared to attacks inspired by traditional ones (e.g., click redirection). Our results demonstrate a clear gap in user awareness and responses when collaborating remotely in MR environments. Our findings emphasize the importance of training users to recognize potential threats and enhanced security measures to maintain trust in remote collaborative MR systems.

  PublisherDOI

• Eye-Tracked Virtual Reality: A Comprehensive Survey on Methods and Privacy Challenges

  Proceedings of the IEEE · 2025-10-01 · 15 citations

  preprintOpen access

  The latest developments in computer hardware, sensor technologies, and artificial intelligence can make virtual reality (VR) and virtual spaces an important part of human everyday life. Eye tracking offers not only a hands-free way of interaction but also the possibility of a deeper understanding of human visual attention and cognitive processes in VR. Despite these possibilities, eye-tracking data also reveal users’ privacy-sensitive attributes when combined with the information about the presented stimulus. To address all, this survey first covers major works in eye tracking, VR, and privacy areas between 2012 and 2022. While eye tracking in VR part covers the computational eye-tracking pipeline from pupil detection and gaze estimation to offline data analysis, for privacy and security, we focus on eye-based authentication as well as computational methods to preserve the privacy of individuals and their eye-tracking data in VR. Later, we outline three main directions by focusing on privacy. In summary, this survey presents an extensive literature review of the utmost possibilities of eye tracking in VR and their privacy implications.

  PublisherDOI

• Visceral Notices and Privacy Mechanisms for Eye Tracking in Augmented Reality

  IEEE Transactions on Visualization and Computer Graphics · 2025-10-03

  articleSenior author

  Head-worn augmented reality (AR) continues to evolve through critical advancements in power optimizations, AI capabilities, and naturalistic user interactions. Eye-tracking sensors play a key role in these advancements. At the same time, eye-tracking data is not well understood by users and can reveal sensitive information. Our work contributes visualizations based on visceral notice to increase privacy awareness of eye-tracking data in AR. We also evaluated user perceptions towards privacy noise mechanisms applied to gaze data visualized through these visceral interfaces. While privacy mechanisms have been evaluated against privacy attacks, we are the first to evaluate them subjectively and understand their influence on data-sharing attitudes. Despite our participants being highly concerned with eye-tracking privacy risks, we found 47% of our participants still felt comfortable sharing raw data. When applying privacy noise, 70% to 76% felt comfortable sharing their gaze data for the Weighted Smoothing and Gaussian Noise privacy mechanisms, respectively. This implies that participants are still willing to share raw gaze data even though overall data-sharing sentiments decreased after experiencing the visceral interfaces and privacy mechanisms. Our work implies that increased access and understanding of privacy mechanisms are critical for gaze-based AR applications; further research is needed to develop visualizations and experiences that relay additional information about how raw gaze data can be used for sensitive inferences, such as age, gender, and ethnicity. We intend to open-source our codebase to provide AR developers and platforms with the ability to better inform users about privacy concerns and provide access to privacy mechanisms. A pre-print of this paper and all supplemental materials are available at https://bmdj-vt.github.io/project_pages/privacy_notice.

  PublisherDOI

• Demonstration of Visceral Notices and Privacy Mechanisms for Eye Tracking in Augmented Reality

  2025-10-08 · 1 citations

  articleSenior author

  We demonstrate visceral interfaces (VIs) and privacy mechanisms that make eye tracking in augmented reality (AR) more transparent and understandable through data visualization. VIs are visual overlays that indicate when and how gaze data is collected, designed to increase privacy awareness. We implement three privacy mechanisms (Gaussian noise, weighted smoothing, and temporal down-sampling) that perturb gaze data and visualize their impact on user perceptions of data sharing. The demo runs on Magic Leap 2 and includes an art gallery and a gaze selection task scenario. Participants explore combinations of VIs and privacy mechanisms, contributing to more transparent, privacy-aware AR systems.

  PublisherDOI

• Rethinking Privacy Indicators in Extended Reality: Multimodal Design for Situationally Impaired Bystanders

  ArXiv.org · 2025-08-09

  preprintOpen accessSenior author

  As Extended Reality (XR) devices become increasingly prevalent in everyday settings, they raise significant privacy concerns for bystanders: individuals in the vicinity of an XR device during its use, whom the device sensors may accidentally capture. Current privacy indicators, such as small LEDs, often presume that bystanders are attentive enough to interpret the privacy signals. However, these cues can be easily overlooked when bystanders are distracted or have limited vision. We define such individuals as situationally impaired bystanders. This study explores XR privacy indicator designs that are effective for situationally impaired bystanders. A focus group with eight participants was conducted to design five novel privacy indicators. We evaluated these designs through a user study with seven additional participants. Our results show that visual-only indicators, typical in commercial XR devices, received low ratings for perceived usefulness in impairment scenarios. In contrast, multimodal indicators were preferred in privacy-sensitive scenarios with situationally impaired bystanders. Ultimately, our results highlight the need to move toward adaptable, multimodal, and situationally aware designs that effectively support bystander privacy in everyday XR environments.

  PublisherOA PDFDOI

• Swap It Like Its Hot: Segmentation-based spoof attacks on eye-tracking images

  arXiv (Cornell University) · 2024-04-22

  preprintOpen accessSenior author

  Video-based eye trackers capture the iris biometric and enable authentication to secure user identity. However, biometric authentication is susceptible to spoofing another user's identity through physical or digital manipulation. The current standard to identify physical spoofing attacks on eye-tracking sensors uses liveness detection. Liveness detection classifies gaze data as real or fake, which is sufficient to detect physical presentation attacks. However, such defenses cannot detect a spoofing attack when real eye image inputs are digitally manipulated to swap the iris pattern of another person. We propose IrisSwap as a novel attack on gaze-based liveness detection. IrisSwap allows attackers to segment and digitally swap in a victim's iris pattern to fool iris authentication. Both offline and online attacks produce gaze data that deceives the current state-of-the-art defense models at rates up to 58% and motivates the need to develop more advanced authentication methods for eye trackers.

  PublisherOA PDFDOI

• Visceral Interfaces for Privacy Awareness of Eye Tracking in VR

  2024-10-21 · 4 citations

  articleSenior author

  Eye tracking is increasingly being integrated into virtual reality (VR) devices to support a wide range of applications. It is used as a method of interaction, to support performance optimizations, and to create adaptive trainingor narrative experiences. However, providing access to eye-tracking data also introduces the ability to monitor user activity, detect and classify a user’s biometric identity, or otherwise reveal sensitive information such as medical conditions. As this technology continues to evolve, users should be made aware of the amount of information they are sharing about themselves to developers and how it can be used. While traditional terms of service may relay this type of information, previous work indicates they are not accessibly conveying privacy-related information to users. Considering this problem, we suggest the application of visceral interfaces that are designed to inform users about eye-tracking data within the VR experience. To this end, we designed and conducted a user study on three visceral interfaces to educate users about their eye-tracking data. Our results suggest that while certain visualizations can be distracting, participants ultimately found them informative and supported the development and availability of such interfaces even if they are not enabled by default or always enabled. Our research contributes to developing informative interfaces specific to eye tracking that promote transparency and privacy awareness in data collection for VR.

  PublisherDOI

Frequent coauthors

• Eakta Jain

  11 shared

• Kevin Butler

  8 shared

• Olivier Le Meur

  8 shared

• Bo Ji

  Virginia Tech

  6 shared

• Matthew Corbett

  Virginia Tech

  5 shared

• Diane Hosfelt

  5 shared

• Candace E. Peacock

  5 shared

• Jiacheng Shang

  Montclair State University

  5 shared

Labs

• PrivateEye Lab @ VTPI

  studies interaction in virtual/mixed/augmented reality (XR) with a particlar focus on eye tracking and protecting privacy

Education

• PhD, Computer & Information Science & Engineering

  University of Florida

  2022