A Scientific Event Camera: Theory, Design, and Measurements
Loading...
Author / Producer
Date
2024
Publication Type
Doctoral Thesis
ETH Bibliography
yes
Citations
Altmetric
Data
Rights / License
Abstract
This thesis explores the physical limits of neuromorphic event cameras and proposes a novel highly-sensitive event camera targeting scientific applications.
Certain scientific applications, such as fluorescent imaging of neural activity (especially voltage imaging), as well as tracking of particles/objects moving at high speed, require vision sensors operating near the limits of physics.
These applications consist in the detection of low contrast changes in light intensity during time intervals of only a few milliseconds or less, in observations that can last several minutes.
Currently, these applications rely on scientific image sensors capable of capturing thousands of frames per second. While the paradigm of frames is highly prevalent in computer vision, it has significant downsides. Namely, for the applications described, the acquisition of thousands of frames per second during several minutes leads to a highly redundant output, resulting in an extremely inefficient data utilization.
Vision in animals is the product of millions of years of evolution through natural selection, resulting in visual systems orders of magnitude more efficient than frame-based cameras. Neuromorphic silicon retinas such as the Dynamic Vision Sensor (DVS) event camera draw inspiration from biological visual
systems to build more efficient vision sensors.
Characteristics of the DVS such as its high-speed performance with low latency and high data efficiency, as well as its high dynamic range, make it an emerging technology with growing popularity over the last years. These characteristics make the DVS a promising candidate for the scientific applications mentioned above. However, DVS implementations proposed before this work did not demonstrate sufficient sensitivity in the light-constrained settings required by these applications.
The main purpose of the work presented in this thesis is the development of a novel DVS event camera with improved sensitivity under dim light. To achieve this goal, this thesis investigates the physical limits of the DVS technology, demonstrating that the DVS is limited to a minimum of 2x shot noise, and providing the conditions for the camera to operate near this limit. It also shows that spatial and temporal integration of light are fundamental to improve sensitivity in the dark - a result known from other visual systems, but never fully exploited in the DVS. This new knowledge, resulting from extensive measurements of DVS cameras and supported by theoretical analysis, resulted in a more realistic model of the DVS pixel, capable of reproducing measured phenomena and aligning with theory.
The results obtained are useful for DVS users, by providing optimal biasing strategies, for algorithm developers, by providing novel interpretations and insight about DVS data encoding, and for DVS designers, by defining the limits of the technology and optimization goals.
Finally, supported by an improved understanding of the DVS pixel and its limits, this thesis proposes SciDVS: A scientific event camera capable of responding to edges of 1.7 % contrast under dim light settings at 0.7 lx on-chip illuminance. SciDVS features an array of 126 × 112 pixels with a pitch of
30 μm, implemented on a 180 nm CMOS Image Sensor process. The SciDVS pixel introduces novelty such as an auto-centering high dynamic range pre-amplifier, improved bandwidth control achieving cutoff frequencies down to 3.5 Hz, and pixel binning.
Permanent link
Publication status
published
External links
Editor
Contributors
Examiner: Delbrück, Tobias
Examiner : Liu, Shih-Chii
Examiner : Serrano-Gotarredona, Teresa
Examiner : Grewe, Benjamin
Book title
Journal / series
Volume
Pages / Article No.
Publisher
ETH Zurich
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Event camera; Dynamic Vision Sensor; Neuromorphic Engineering; Image sensor; Vision
Organisational unit
08836 - Delbrück, Tobias (Tit.-Prof.)