Virtual Navigation Environment for Blind and Low Vision People

. For comprehensively participating in society, independent and safe mobility is an important skill for many daily activities. Spatial cognition is one of the most important human capabilities and addresses the acquisition, processing and utilization of knowledge about the spatial layout of environments. Humans predominantly use the visual sense for this and for blind and low vision people, the lack of spatial perception reduces their quality of life and their ability of independent living. In particular the spatial navigation in unknown environ‐ ments imposes challenges, since there is no possibility to train navigation tasks in advance. Today, blind and visually impaired people still rely on traditional navigation aids such as a cane for micro-navigation, which - however - does not help for developing orientation at larger scale or for planning of routes. To over‐ come this problem, this paper introduces the concept of a virtual environment that allows experiencing unknown locations by real walking while still staying in a sage controlled environment. Since this virtual environment can be controlled in its complexity, it can be adjusted from an abstract training scenario to a real-life situation such as train stations or airports.


Introduction
Visually impaired people (VIP) encounter many mobility problems as stated by [1,2].To overcome such problems, many mobility aids were developed in the past decades to help VIP acquiring spatial information by avoiding obstacles [3], reading maps [4,5], and navigating in unfamiliar regions [6,7] by applying 3 main categories of non-visual interfaces: auditory-only (i.e.non-speech sounds and speech), haptic-tactile only (e.g.vibrotactile, electrotactile), and audio-haptic.
Only little research was done in using Virtual Reality (VR) to train VIP in navigation tasks prior to their exposure to real world.VR was shown to be advantageous for personalized orientation and mobility (O&M) trainings.Research outcomes reach from a virtual cane (mouse, keyboard, or force-feedback device) to other devices to navigate in an unfamiliar virtual environment [8][9][10].
The HOMERE system [11] was one of the first VR systems developed for a blind user to navigate a virtual environment (VE) to build a mental model.A real cane was triggered by a force-feedback device depending on the virtual ground.Together with audio and thermal feedback, it provided navigation cues that could help a blind user to orient in a large VE.However, the walking path is predefined, thus prohibiting free navigation by the VIP.A system by [12] offered the possibility of walking, but the range was rather limited due to the tracking that was employed.To answer the question whether a VR system could help to build a cognitive map of the environment, [13] built a VE which could be experienced by a force-feedback joystick together with an audio-output.It turned out that VEs are suitable to train cognitive maps, although there are still differences among users regarding the learned spatial arrangements of objects in the cognitive maps.However, these virtual training environments fail to address the important natural perception of real walking in VEs.
Although [14] reported that real walking in a VE improves sighted users' cognitive maps via visual representation, it is not clear whether such real walking could be applied to VIP, i.e. whether a non-visual representation (i.e.haptic and/or auditory) will improve VIP's spatial cognition as well.In particular, it is challenging for VIP to explore a large VE in a limited physical space while really walking.For an immersive experience for example, a 570 m 2 physical space was built by [15] to implement an early VR system without redirected walking.With redirected walking (RDW) algorithms [16,17], it is possible to achieve a compression of virtual space into physical space of 10:1.However, RDW has never been applied before to VIP.
In this paper, we introduce the concept of using VR to design a multimodal interface (haptic, acoustic, sensomotoric perception) that will help VIP to build a cognitive map of the surrounding by real walking.The VR is controllable and can be adapted from an abstract obstacle representation like steps, walls, etc. to a complex representation of buildings that exist in real life.VIPs can thus train real world scenarios without leaving a safe and controlled physical space.However, experiencing large VEs in a limited physical space requires a subtle redirection, being achieved by gains.They will be applied to translations, rotations, and curvatures of the user's walking trajectory.This redirection technology will be applied for the first time to VIP.

Working Principle
VIPs have experiences and capabilities to learn spatial information and thus need personalized mobility training plans.A suitable training system is proposed in Fig. 1.While the VIP is in the training environment, he perceives acoustic and/or haptic feedback when approaching an obstacle.Since time and user position are continuously measured and analyzed in the "Walking Monitor", changes in speed, stops, or sudden turns give hints how familiar the user is with the experienced environment.After a certain amount of trials, the user performance will not exceed upper or lower thresholds anymore, and the next level of complexity can be loaded from the training plan repository.In other cases, it is also possible to reduce the VE's complexity level.

Technical Setup
The system's hardware consists of an Intersense IS-1200 6 DOF tracking system, which is connected to an Oculus Rift DK 2 head-mounted display (HMD) (although there was no need to display images in the HMD).The tracking system's accuracy is apx. 10 mm in horizontal direction.The HMD is connected to an HP Elitebook 8560w laptop strapped to a backpack that has to be worn by the user.The system also consists of a pair of Sennheiser headphones to provide audio feedback (Fig. 2).For providing haptic feedback to a cane, two different vibration motors are mounted to a printed circuit board and driven by an Arduino Uno microcontroller.The board uses the DRV 2605L from Texas Instruments to generate different waveforms for the individual motors.An electronic switch is used to select and address the desired motor.For the communication between the Arduino board and the DRV 2605L, an I2C bus is user.The first motor is a C10-000 from Precision Microdrives, which uses a linear resonant actuator to generate an impulse feedback.The second motor is the 306-117, which is an eccentric rotating mass motor to provide vibrating feedback (Fig. 2).The virtual world was built in Unity and the walkable space comprised a 5 m × 10 m area, which is centered within the walls of a larger physical space.For the first experiments described here, no haptics and redirection are used.

Obstacle Alert System
A physical cane typically gives binary information whether there is an obstacle in the way or not.The distance to the obstacle is then automatically given by the length of the cane the VIP is familiar with.However, in a VE there is neither a physical cane nor a physical object it could collide with (besides the physical wall boundaries of the tracking space).This imposes the problem that a distance-dependent alert is missing and the VIP's walking speed is not taken into account.Thus, our training system provides a twophase alert system, which takes into account the user's distance to an obstacle as well as his walking speed [18].The three phases are shown in Fig. 3. Phase 1 is velocity-based and takes into account the time until the user would collide with an obstacle.He is warned if the time is below 3.5 s.In an ideal case, this phase would be sufficient.However, there are certain conditions, e.g. when the user stops moving in phase 1, where the velocity-based alert needs to be supplemented by an additional phase.Thus, phase 2 is distance-based and supplements the velocity-based phase 1. Phase 2 becomes necessary, since there might be the case where the user stands still in a close distance to an obstacle.In this case, phase 1 would not deliver any output, but phase 2 is still active.The two-phase warning system also allows for two different warning outputs.For the VE navigation, we rely on haptic and audio output only.With these two modalities, four different combinations are possible: • The audio-audio combination consists of a beep with an increasing pace for phase 1, while it provides a continuous tone or phase 2. • The haptic-haptic combination can make use of a variety of parameters.Since two different motors exist, impulse, continuous vibration, or a combination of both.• The audio-haptic combination could replay an acoustic sound for phase 1, while a haptic feedback is provided for phase 2. • The haptic-audio combination will provide a tactile feedback, followed by a sound output for phase 2.

Preliminary System Evaluation for Audio/Audio
For the preliminary evaluation described in the remainder of the paper, we focus on the audio-audio combination, while the other combinations will follow in future work.Unlike the mechanical sweep of a cane, which mainly detects obstacles in walking direction, the stereoscopic audio replay using the headphones also allows sonifying obstacles sideward, giving him more information of the objects' spatial distribution.The audio-audio alert is modelled after the proximity warning system used to aid drivers in parking cars.If a user moves towards an obstacle at a constant speed, he will hear a beep that will increase its pace while the user approaches the obstacle.When the user enters phase 2, the beep will change to a continuous signal to warn him of the collision.The virtual audio sources are located in the obstacles nearest to the user, as shown in Fig. 4. The audio sources move along with the user in the virtual environment and their signals correspond to distance between user and obstacle.To evaluate the acoustic alerts, a series of travel-to-target navigation scenarios was used (Fig. 5).Besides the audio signals, the subjects had no other navigation aids such as a cane.In Fig. 5, the start and end positions are shown, as well as walls and obstacles.Each of these basic layouts could variations as follows: • Maze level: Each maze consists of 8 90° turns.Each maze also has a box somewhere in its geometry in order to avoid a completely zigzagged path.Prior to the test, participants get an instruction about the task: "There is a navigation scenario that you need to get through….Your task is to come from start to finish as quickly as possible.However, there are obstacles in your way from start to finish".
For each participant, positional data was recorded, as well as a corresponding timestamp.From these measures, it is possible to determine the path walked by the participants together with the walking speed, distance travelled, completion time, the ratio of walking to non-walking time, and the proportions they spent in the phases.
All participants also have to complete a simulator sickness questionnaire immediately prior to and after the experiment.None of the participants showed symptoms of experiencing cybersickness.We used the motion sickness questionnaire [20].For a preliminary technical test, we were not able to recruit VIP.Instead, we had 13 sighted users between 22 to 35 years age.By wearing the HMD, the subjects were blindfolded and could not see the real environment anymore.Seven of the users had prior experience with VR or any other room scale tracking.Four users had experience with low-end VR devices such as Google cardboard, and two users had experience with high-end VR (Rift, HTC Vive).

Preliminary Results
The 13 users performed in total 156 trials with the three different geometry types and its 4 variants.Since we wanted to test the alert efficiency only, none of the users experienced the same geometry twice to avoid learning effects.From totally 147 collisions, each of the 12 geometries had an average of μ = 3.77 collisions.Regarding the time users spent in each phase, it is obvious that phase 1 duration was relatively short compared to phase 0 (no alert) and phase 2 (see Fig. 6).Thus, users either were in the safe zone (phase 0), in which no alert was triggered, or they rapidly crossed phase 1 to be in phase 2 and thus more endangered to experience a virtual collision.From the tracking system, the positional information together with the rime stamps was recorded.From this, a path visualization of when the user is walking versus when he has stopped is generated (see Fig. 7).It is clearly visible that users stopped frequently to acoustically detect the position of obstacles, they only walked 53% of the completion time while they used the other time for orientation.

Discussion and Future Work
The results still show a high number of collisions as well as a long time in which users did not walk but stopped for acoustic orientation.This might have several reasons: • Since the group consisted of sighted users, they might not be trained to use acoustics for navigation as blind users would probably be, e.g. by clicking one's tongue.• The combination of an increasing beep pace (phase 1) and a jump to a continuous beep (phase 2) might have irritated the users.• For certain geometries, ambiguities in the audio output signal exist.
• The cognition process is more complex, since not only the existence of an obstacle is alerted, but also its coarse position (distance, orientation).This requires interpreting the beep pace between the left and the right ear, as well as some active movement of the head to overcome possible ambiguities.• The test environment partly consisted of narrow corridors triggering alerts.
Future work on the acoustic alerts will focus on different syntheses of the sound signals.Since differences in pace between the left and the right ear are difficult to detect, other modalities like frequency or constant alert (for phase 2) will be tested.After these tests, an extensive user study will be performed, verifying how well this VE with real walking will support to train a mental model of the environment.The user studies will also evaluate the other output modalities and their combinations (haptic/haptic, haptic/ audio, audio/haptic).

3(3), 203-220 (1993)
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/),which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material.If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Fig. 2 .
Fig. 2. System hardware (left) with layout of circuit board that drives the two haptic motors (middle) and completed printed circuit board with all components mounted (right).

Fig. 3 .
Fig. 3.The magnitude of the alert is determined by the user's time to impact.

Fig. 4 .
Fig. 4. The audio feedback setup with virtual audio sources placed in the obstacles closest to the user, the fourth audio source behind the user is not shown [19].
Office level: An open layout with three obstacles on the way.Obstacles include a square obstacle (1 × 1 m), a wall of 2.5 m length, and a wall corner of 2.0 × 1.5 m. • Door level: A series of four doors on the way from start to finish.The doors can be in one of three positions: adjacent to either the left or right wall, or in the middle.

Fig. 6 .
Fig. 6.Proportions of total times spent in each phase.

Fig. 7 .
Fig. 7. Visualization of walking versus stopping.The points on the walking path indicate where the subject stopped for orientation.