HapticSeer

Haptic feedback significantly enhances virtual experiences. However, supporting haptics currently requires modifying the codebase, making it impractical to add haptics to popular, high-quality experiences such as best selling games, which are typically closed-source.

We present HapticSeer, a multi-channel, black-box, platform-agnostic approach to detecting game events for real-time haptic feedback. The approach is based on two key insights:

  1. All games have 3 types of data streams: video, audio, and controller I/O, that can be analyzed in real-time to detect game events.
  2. A small number of user interface design patterns are reused across most games, so that event detectors can be reused effectively.

We developed an open-source HapticSeer framework and implemented several real-time event detectors for commercial PC and VR games. We validated system correctness and real-time performance, and discuss feedback from several haptics developers that used the HapticSeer framework to integrate research and commercial haptic devices.

We implemented a system whose components are mostly language-independent, loosely coupled, and encapsulated by utilizing a central message broker. Our system architecture comprises three levels of components, raw data capturers, feature extractors, and event detectors.

(CHI ’21 Full Paper) HapticSeer: A Multi-channel, Black-box, Platform-agnostic Approach to Detecting Video Game Events for Real-time Haptic Feedback [HONORABLE MENTION]

Yu-Hsin Lin, Yu-Wei Wang, Pin-Sung Ku, Yun-Ting Cheng, Yuan-Chih Hsu, Ching-Yi Tsai, and Mike Y. Chen. 2021. HapticSeer: A Multi-channel, Black-box, Platform-agnostic Approach to Detecting Video Game Events for Real-time Haptic Feedback. In CHI Conference on Human Factors in Computing Systems (CHI ’21), May 8–13, 2021, Yokohama, Japan. ACM, New York, NY, USA, 14 pages.

DOI: https://doi.org/10.1145/3411764.344

(Talk) Youtube

(Repository) hapticseer.org

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s