Project Tag: CHI '21

HapticSeer

Haptic feedback significantly enhances virtual experiences. However, supporting haptics currently requires modifying the codebase, making it impractical to add haptics to popular, high-quality experiences such as best selling games, which are typically closed-source.

We present HapticSeer, a multi-channel, black-box, platform-agnostic approach to detecting game events for real-time haptic feedback. The approach is based on two key insights:

  1. All games have 3 types of data streams: video, audio, and controller I/O, that can be analyzed in real-time to detect game events.
  2. A small number of user interface design patterns are reused across most games, so that event detectors can be reused effectively.

We developed an open-source HapticSeer framework and implemented several real-time event detectors for commercial PC and VR games. We validated system correctness and real-time performance, and discuss feedback from several haptics developers that used the HapticSeer framework to integrate research and commercial haptic devices.

We implemented a system whose components are mostly language-independent, loosely coupled, and encapsulated by utilizing a central message broker. Our system architecture comprises three levels of components, raw data capturers, feature extractors, and event detectors.

(CHI ’21 Full Paper) HapticSeer: A Multi-channel, Black-box, Platform-agnostic Approach to Detecting Video Game Events for Real-time Haptic Feedback [HONORABLE MENTION]

Yu-Hsin Lin, Yu-Wei Wang, Pin-Sung Ku, Yun-Ting Cheng, Yuan-Chih Hsu, Ching-Yi Tsai, and Mike Y. Chen. 2021. HapticSeer: A Multi-channel, Black-box, Platform-agnostic Approach to Detecting Video Game Events for Real-time Haptic Feedback. In CHI Conference on Human Factors in Computing Systems (CHI ’21), May 8–13, 2021, Yokohama, Japan. ACM, New York, NY, USA, 14 pages.

DOI: https://doi.org/10.1145/3411764.344

(Talk) Youtube

(Repository) hapticseer.org

JetController

JetController is a novel haptic technology capable of supporting high-speed and persistent 3-DoF ungrounded force feedback. It uses high-speed pneumatic solenoid valves to modulate compressed air to achieve 20-50Hz of full impulses at 4.0-1.0N, and combines multiple air propulsion jets to generate 3-DoF force feedback. Compared to propeller-based approaches, JetController supports 10-30 times faster impulse frequency, and its handheld device is significantly lighter and more compact. JetController supports a wide range of haptic events in games and VR experiences, from firing automatic weapons in games like Halo (15Hz) to slicing fruits in Fruit Ninja (up to 45Hz). To evaluate JetController, we integrated our prototype with two popular VR games, Half-life: Alyx and Beat Saber, to support a variety of 3D interactions. Study results showed that JetController significantly improved realism, enjoyment, and overall experience compared to commercial vibrating controllers, and was preferred by most participants.

JetController is a novel high-speed 3-DoF ungrounded force feedback technology capable of supporting the speed of human button presses and high-speed game events.

Press Coverage Links

山下裕毅 (2021, Aug 25). “ジェット噴射”で衝撃与えるVRコントローラー　台湾大学とお茶大「JetController」開発 https://www.itmedia.co.jp/news/articles/2108/25/news047.html

爆炸哥 (2021, Aug 25). 【有片】手掣噴氣添開槍後座力 台灣大學研發JetController https://www.gameover.com.hk/news/525066

Paper Links

(CHI ’21 Full Paper) JetController: High-speed Ungrounded 3-DoF Force Feedback Controllers using Air Propulsion Jets

Yu-Wei Wang, Yu-Hsin Lin, Pin-Sung Ku, Yoko Miyatake, Yi-Hsuan Mao, Po Yu Chen, Chun-Miao Tseng, and Mike Y. Chen. 2021. JetController: Highspeed Ungrounded 3-DoF Force Feedback Controllers using Air Propulsion Jets. In CHI Conference on Human Factors in Computing Systems (CHI ’21), May 8–13, 2021, Yokohama, Japan. ACM, New York, NY, USA.
DOI: https://doi.org/10.1145/3411764.3445549

(CHI’21 Interactivity) Demonstration of JetController: High-speed Ungrounded Force Feedback Controllers Using Air Propulsion Jets

Yu-Wei Wang, Yu-Hsin Lin, Pin-Sung Ku, Yoko Miyatake, Po-Yu Chen, Chun-Miao Tseng, Ching-Yi Tsai, and Mike Y. Chen. 2021. Demonstration of JetController: High-speed Ungrounded Force Feedback Controllers Using Air Propulsion Jets. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI ’21 Extended Abstracts), May 8–13, 2021, Yokohama, Japan. ACM, New York, NY, USA.

DOI: https://doi.org/10.1145/3411763.3451542 

(SIGGRAPH ’21 Labs Installation) JetController: High-speed Ungrounded 3-DoF Force Feedback Controllers using Air Propulsion Jets

Yu-Wei Wang, Yu-Hsin Lin, Yoko Miyatake, Ching-Yi Tsai, Pin-Sung Ku,and Mike Y. Chen. 2021. JetController: High-speed Ungrounded 3-DoFForce Feedback Controllers using Air Propulsion Jets. InSpecial InterestGroup on Computer Graphics and Interactive Techniques Conference Labs(SIGGRAPH ’21 Labs), August 09-13, 2021.ACM, New York, NY, USA.

DOI: https://doi.org/10.1145/3450616.3464520