Deep Partial Updating: Towards Communication Efficient Updating for On-device Inference
Open access
Date
2022Type
- Conference Paper
ETH Bibliography
yes
Altmetrics
Abstract
Emerging edge intelligence applications require the server to retrain and update deep neural networks deployed on remote edge nodes to leverage newly collected data samples. Unfortunately, it may be impossible in practice to continuously send fully updated weights to these edge nodes due to the highly constrained communication resource. In this paper, we propose the weight-wise deep partial updating paradigm, which smartly selects a small subset of weights to update in each server-to-edge communication round, while achieving a similar performance compared to full updating. Our method is established through analytically upper bounding the loss difference between partial updating and full updating, and only updates the weights which make the largest contributions to the upper bound. Extensive experimental results demonstrate the efficacy of our partial updating methodology which achieves a high inference accuracy while updating a rather small number of weights. Show more
Permanent link
https://doi.org/10.3929/ethz-b-000560918Publication status
publishedExternal links
Book title
Computer Vision – ECCV 2022Journal / series
Lecture Notes in Computer ScienceVolume
Pages / Article No.
Publisher
SpringerEvent
Subject
Partial updating; communication constraints; parameter reuse; server-to-edge; deep neural networksOrganisational unit
03429 - Thiele, Lothar (emeritus) / Thiele, Lothar (emeritus)
More
Show all metadata
ETH Bibliography
yes
Altmetrics