Search
Results
-
Third order nonlinear correlation of the electromagnetic vacuum at near-infrared frequencies
(2023)New Journal of PhysicsJournal Article -
Vision Transformers with Hierarchical Attention
(2024)Machine Intelligence ResearchThis paper tackles the high computational/space complexity associated with multi-head self-attention (MHSA) in vanilla vision transformers. To this end, we propose hierarchical MHSA (H-MHSA), a novel approach that computes sell-attention in a hierarchical fashion. Specifically, we first divide the input image into patches as commonly done, and each patch is viewed as a token. Then, the proposed H-MHSA learns token relationships within ...Journal Article