Show simple item record

dc.contributor.author
Chandran, Prashanth
dc.contributor.author
Zoss, Gaspard
dc.contributor.author
Gross, Markus
dc.contributor.author
Gotardo, Paulo
dc.contributor.author
Bradley, Derek
dc.date.accessioned
2023-03-24T09:48:16Z
dc.date.available
2023-01-27T06:57:24Z
dc.date.available
2023-03-24T09:48:16Z
dc.date.issued
2022-12
dc.identifier.issn
1467-8659
dc.identifier.issn
0167-7055
dc.identifier.other
10.1111/cgf.14641
en_US
dc.identifier.uri
http://hdl.handle.net/20.500.11850/595239
dc.description.abstract
We propose a 3D+time framework for modeling dynamic sequences of 3D facial shapes, representing realistic non-rigid motion during a performance. Our work extends neural 3D morphable models by learning a motion manifold using a transformer architecture. More specifically, we derive a novel transformer-based autoencoder that can model and synthesize 3D geometry sequences of arbitrary length. This transformer naturally determines frame-to-frame correlations required to represent the motion manifold, via the internal self-attention mechanism. Furthermore, our method disentangles the constant facial identity from the time-varying facial expressions in a performance, using two separate codes to represent neutral identity and the performance itself within separate latent subspaces. Thus, the model represents identity-agnostic performances that can be paired with an arbitrary new identity code and fed through our new identity-modulated performance decoder; the result is a sequence of 3D meshes for the performance with the desired identity and temporal length. We demonstrate how our disentangled motion model has natural applications in performance synthesis, performance retargeting, key-frame interpolation and completion of missing data, performance denoising and retiming, and other potential applications that include full 3D body modeling.
en_US
dc.language.iso
en
en_US
dc.publisher
Wiley
en_US
dc.subject
CCS Concepts
en_US
dc.subject
Computing methodologies → Motion processing
en_US
dc.subject
Shape modeling
en_US
dc.subject
Mesh geometry models
en_US
dc.title
Facial Animation with Disentangled Identity and Motion using Transformers
en_US
dc.type
Conference Paper
dc.date.published
2023-03-20
ethz.journal.title
Computer Graphics Forum
ethz.journal.volume
41
en_US
ethz.journal.issue
8
en_US
ethz.journal.abbreviated
Comput. Graph. Forum
ethz.pages.start
267
en_US
ethz.pages.end
277
en_US
ethz.event
21st ACM SIGGRAPH / Eurographics Symposium on Computer Animation (SCA 2022)
en_US
ethz.event.location
Durham, United Kingdom
en_US
ethz.event.date
September 13-15, 2022
en_US
ethz.identifier.wos
ethz.identifier.scopus
ethz.publication.place
Oxford
en_US
ethz.publication.status
published
en_US
ethz.leitzahl
ETH Zürich::00002 - ETH Zürich::00012 - Lehre und Forschung::00007 - Departemente::02150 - Dep. Informatik / Dep. of Computer Science::02659 - Institut für Visual Computing / Institute for Visual Computing::03420 - Gross, Markus / Gross, Markus
en_US
ethz.leitzahl.certified
ETH Zürich::00002 - ETH Zürich::00012 - Lehre und Forschung::00007 - Departemente::02150 - Dep. Informatik / Dep. of Computer Science::02659 - Institut für Visual Computing / Institute for Visual Computing::03420 - Gross, Markus / Gross, Markus
en_US
ethz.date.deposited
2023-01-27T06:57:24Z
ethz.source
FORM
ethz.eth
yes
en_US
ethz.availability
Metadata only
en_US
ethz.rosetta.installDate
2023-03-24T09:48:17Z
ethz.rosetta.lastUpdated
2024-02-02T21:17:25Z
ethz.rosetta.versionExported
true
ethz.COinS
ctx_ver=Z39.88-2004&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.atitle=Facial%20Animation%20with%20Disentangled%20Identity%20and%20Motion%20using%20Transformers&rft.jtitle=Computer%20Graphics%20Forum&rft.date=2022-12&rft.volume=41&rft.issue=8&rft.spage=267&rft.epage=277&rft.issn=1467-8659&0167-7055&rft.au=Chandran,%20Prashanth&Zoss,%20Gaspard&Gross,%20Markus&Gotardo,%20Paulo&Bradley,%20Derek&rft.genre=proceeding&rft_id=info:doi/10.1111/cgf.14641&
 Search print copy at ETH Library

Files in this item

FilesSizeFormatOpen in viewer

There are no files associated with this item.

Publication type

Show simple item record