Gross, Markus H.
Rights / licenseIn Copyright - Non-Commercial Use Permitted
We present a novel framework for the multimodal display of words using topological, appearance, and auditory representations. The methods are designed for effective language training and serve as a learning aid for individuals with dyslexia. Our topological code decomposes the word into its syllables and displays it graphically as a tree structure. The appearance code assigns color attributes and shape primitives to each letter and takes into account conditional symbol probabilities, code ambiguities, and phonologically confusable letter combinations. An additional auditory code assigns midi events to each symbol and thus generates a melody for each input string. The entire framework is based on information theory and utilizes a Markovian language model derived from linguistic analysis of language corpora for English, French, and German. For effective word repetition a selection controller adapts to the user’s state and optimizes the learning process by minimizing error entropy. The performance of the method was evaluated in a large scale experimental study involving 80 dyslexic and non-dyslexic children. The results show significant improvements in writing skills in both groups after small amounts of daily training. Our approach combines findings from 3D computer graphics, visualization, linguistics, perception, psychology, and information theory Show more
Journal / seriesTechnical report / Swiss Federal Institute of Technology Zurich, Department of Computer Science
PublisherETH, Department of Computer Science
SubjectCoding; Learning; Information Theory; Visualization; Dyslexia; LANGUAGE INSTRUCTION (SPECIAL SUBJECTS); MULTIMEDIA (INFORMATION SYSTEMS); Visual Information; Natural Language; COMPUTERUNTERSTÜTZTER UNTERRICHT + NEW MEDIA (UNTERRICHT); MULTIMEDIA (INFORMATIONSSYSTEME); SPRACHUNTERRICHT (FACHUNTERRICHT); COMPUTER-AIDED INSTRUCTION + NEW MEDIA (INSTRUCTION); Entropy
Organisational unit02150 - Departement Informatik / Department of Computer Science
NotesTechnical Reports D-INFK.
MoreShow all metadata