Detection and Localisation of Pointing, Pairing and Grouping Gestures for Brainstorming Meeting Applications
METADATA ONLY
Loading...
Author / Producer
Date
2021
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
The detection of gestures and their interpretaton is crucial for blind and visually impaired people (BVIP). In a card-based brainstorming meeting, sighted users use non-verbal communication when referring to cards on a common workspace using pointing, grouping, or pairing gestures. While sighted users could easily interpret such gestures, they remain inaccessible to BVIP. Thus, there is a need for capturing, interpreting and translating gestures for BVIP.
To address this problem, we developed a pointing gesture detection system using Unity with the SteamVR Plugin and HTC Vive. HTC's trackers are attached to a user's hands to measure the hand position in 3D space. With pointing gestures, a user controls a virtual ray that will intersect with a virtual whiteboard. This virtual whiteboard is invisible to the sighted users, but its position and size corresponds to a physical digital whiteboard. The intersection of the ray with the virtual whiteboard is calculated, resulting in a pointing trajectory on it. The shape of the trajectory is analyzed to determine, which artifacts are selected by the pointing gesture. A pointing gesture is detected when a user is pointing at a card on the screen and then ending the gesture by pointing outside of the screen. A pairing gesture is detected when pointing at one artifact and then on another one before leaving the screen. The grouping gesture is detected when performing an encircling gesture around multiple artifacts before leaving the screen.
Permanent link
Publication status
published
External links
Book title
HCI International 2021 - Posters
Journal / series
Volume
1420
Pages / Article No.
22 - 29
Publisher
Springer
Event
23rd International Conference on Human-Computer Interaction (HCII 2021)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Blind and visually impaired; Pointing gestures; Virtual reality
Organisational unit
08844 - Kunz, Andreas (Tit.-Prof.) / Kunz, Andreas (Tit.-Prof.)