Detection and Localisation of Pointing, Pairing and Grouping Gestures for Brainstorming Meeting Applications
Metadata only
Date
2021Type
- Conference Paper
ETH Bibliography
yes
Altmetrics
Abstract
The detection of gestures and their interpretaton is crucial for blind and visually impaired people (BVIP). In a card-based brainstorming meeting, sighted users use non-verbal communication when referring to cards on a common workspace using pointing, grouping, or pairing gestures. While sighted users could easily interpret such gestures, they remain inaccessible to BVIP. Thus, there is a need for capturing, interpreting and translating gestures for BVIP.
To address this problem, we developed a pointing gesture detection system using Unity with the SteamVR Plugin and HTC Vive. HTC's trackers are attached to a user's hands to measure the hand position in 3D space. With pointing gestures, a user controls a virtual ray that will intersect with a virtual whiteboard. This virtual whiteboard is invisible to the sighted users, but its position and size corresponds to a physical digital whiteboard. The intersection of the ray with the virtual whiteboard is calculated, resulting in a pointing trajectory on it. The shape of the trajectory is analyzed to determine, which artifacts are selected by the pointing gesture. A pointing gesture is detected when a user is pointing at a card on the screen and then ending the gesture by pointing outside of the screen. A pairing gesture is detected when pointing at one artifact and then on another one before leaving the screen. The grouping gesture is detected when performing an encircling gesture around multiple artifacts before leaving the screen. Show more
Publication status
publishedExternal links
Book title
HCI International 2021 - PostersJournal / series
Communications in Computer and Information ScienceVolume
Pages / Article No.
Publisher
SpringerEvent
Subject
Blind and visually impaired; Pointing gestures; Virtual realityOrganisational unit
08844 - Kunz, Andreas (Tit.-Prof.) / Kunz, Andreas (Tit.-Prof.)
More
Show all metadata
ETH Bibliography
yes
Altmetrics