Show simple item record

dc.contributor.author
Hollenstein, Nora
dc.contributor.author
Renggli, Cedric
dc.contributor.author
Glaus, Benjamin
dc.contributor.author
Barett, Maria
dc.contributor.author
Troendle, Marius
dc.contributor.author
Langer, Nicolas
dc.contributor.author
Zhang, Ce
dc.date.accessioned
2021-07-21T09:09:22Z
dc.date.available
2021-07-20T08:54:05Z
dc.date.available
2021-07-21T09:09:22Z
dc.date.issued
2021-07
dc.identifier.issn
1662-5161
dc.identifier.other
10.3389/fnhum.2021.659410
en_US
dc.identifier.uri
http://hdl.handle.net/20.500.11850/496323
dc.identifier.doi
10.3929/ethz-b-000496323
dc.description.abstract
Until recently, human behavioral data from reading has mainly been of interest to researchers to understand human cognition. However, these human language processing signals can also be beneficial in machine learning-based natural language processing tasks. Using EEG brain activity for this purpose is largely unexplored as of yet. In this paper, we present the first large-scale study of systematically analyzing the potential of EEG brain activity data for improving natural language processing tasks, with a special focus on which features of the signal are most beneficial. We present a multi-modal machine learning architecture that learns jointly from textual input as well as from EEG features. We find that filtering the EEG signals into frequency bands is more beneficial than using the broadband signal. Moreover, for a range of word embedding types, EEG data improves binary and ternary sentiment classification and outperforms multiple baselines. For more complex tasks such as relation detection, only the contextualized BERT embeddings outperform the baselines in our experiments, which raises the need for further research. Finally, EEG data shows to be particularly promising when limited training data is available.
en_US
dc.format
application/pdf
en_US
dc.language.iso
en
en_US
dc.publisher
Frontiers Media
dc.rights.uri
http://creativecommons.org/licenses/by/4.0/
dc.subject
EEG
en_US
dc.subject
natural language processing
en_US
dc.subject
frequency bands
en_US
dc.subject
brain activity
en_US
dc.subject
machine learning
en_US
dc.subject
multi-modal learning
en_US
dc.subject
physiological data
en_US
dc.subject
neural network
en_US
dc.title
Decoding EEG Brain Activity for Multi-Modal Natural Language Processing
en_US
dc.type
Journal Article
dc.rights.license
Creative Commons Attribution 4.0 International
dc.date.published
2021-07-13
ethz.journal.title
Frontiers in Human Neuroscience
ethz.journal.volume
15
en_US
ethz.journal.abbreviated
Front. Hum. Neurosci.
ethz.pages.start
659410
en_US
ethz.size
19 p.
en_US
ethz.version.deposit
publishedVersion
en_US
ethz.identifier.wos
ethz.identifier.scopus
ethz.publication.place
Lausanne
ethz.publication.status
published
en_US
ethz.leitzahl
ETH Zürich::00002 - ETH Zürich::00012 - Lehre und Forschung::00007 - Departemente::02150 - Dep. Informatik / Dep. of Computer Science::02663 - Institut für Computing Platforms / Institute for Computing Platforms::09588 - Zhang, Ce (ehemalig) / Zhang, Ce (former)
en_US
ethz.leitzahl.certified
ETH Zürich::00002 - ETH Zürich::00012 - Lehre und Forschung::00007 - Departemente::02150 - Dep. Informatik / Dep. of Computer Science::02663 - Institut für Computing Platforms / Institute for Computing Platforms::09588 - Zhang, Ce (ehemalig) / Zhang, Ce (former)
en_US
ethz.date.deposited
2021-07-20T08:54:11Z
ethz.source
FORM
ethz.eth
yes
en_US
ethz.availability
Open access
en_US
ethz.rosetta.installDate
2021-07-21T09:09:28Z
ethz.rosetta.lastUpdated
2024-02-02T14:23:00Z
ethz.rosetta.versionExported
true
ethz.COinS
ctx_ver=Z39.88-2004&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.atitle=Decoding%20EEG%20Brain%20Activity%20for%20Multi-Modal%20Natural%20Language%20Processing&rft.jtitle=Frontiers%20in%20Human%20Neuroscience&rft.date=2021-07&rft.volume=15&rft.spage=659410&rft.issn=1662-5161&rft.au=Hollenstein,%20Nora&Renggli,%20Cedric&Glaus,%20Benjamin&Barett,%20Maria&Troendle,%20Marius&rft.genre=article&rft_id=info:doi/10.3389/fnhum.2021.659410&
 Search print copy at ETH Library

Files in this item

Thumbnail

Publication type

Show simple item record