Transferring Deep Object and Scene Representations for Event Recognition in Still Images
Open access
Datum
2018-04Typ
- Journal Article
Abstract
This paper addresses the problem of image-based event recognition by transferring deep representations learned from object and scene datasets. First we empirically investigate the correlation of the concepts of object, scene, and event, thus motivating our representation transfer methods. Based on this empirical study, we propose an iterative selection method to identify a subset of object and scene classes deemed most relevant for representation transfer. Afterwards, we develop three transfer techniques: (1) initialization-based transfer, (2) knowledge-based transfer, and (3) data-based transfer. These newly designed transfer techniques exploit multitask learning frameworks to incorporate extra knowledge from other networks or additional datasets into the fine-tuning procedure of event CNNs. These multitask learning frameworks turn out to be effective in reducing the effect of over-fitting and improving the generalization ability of the learned CNNs. We perform experiments on four event recognition benchmarks: the ChaLearn LAP Cultural Event Recognition dataset, the Web Image Dataset for Event Recognition, the UIUC Sports Event dataset, and the Photo Event Collection dataset. The experimental results show that our proposed algorithm successfully transfers object and scene representations towards the event dataset and achieves the current state-of-the-art performance on all considered datasets. Mehr anzeigen
Persistenter Link
https://doi.org/10.3929/ethz-b-000184872Publikationsstatus
publishedExterne Links
Zeitschrift / Serie
International Journal of Computer VisionBand
Seiten / Artikelnummer
Verlag
SpringerThema
Event recognition; Deep learning; Transfer learning; Multitask learningOrganisationseinheit
03514 - Van Gool, Luc / Van Gool, Luc
Anmerkungen
It was possible to publish this article open access thanks to a Swiss National Licence with the publisher.