Sizhen Bian

Last Name

Bian

First Name

Sizhen

ORCID

0000-0001-6760-5539

Show all metadata

Search Results

Publications 1 - 10 of 18

Embedding textile capacitive sensing into smart wearables as a versatile solution for human motion capturing
Geißler, Daniel; Zhou, Bo; Bello, Hymalai; et al. (2024)
Scientific Reports
This work presents a novel and versatile approach to employ textile capacitive sensing as an effective solution for capturing human body movement through fashionable and everyday-life garments. Conductive textile patches are utilized for sensing the movement, working without the need for strain or direct body contact, wherefore the patches can sense only from their deformation within the garment. This principle allows the sensing area to be decoupled from the wearer’s body for improved wearing comfort and more pleasant integration. We demonstrate our technology based on multiple prototypes which have been developed by an interdisciplinary team of electrical engineers, computer scientists, digital artists, and smart fashion designers through several iterations to seamlessly incorporate the technology of capacitive sensing with corresponding design considerations into textile materials. The resulting accumulation of textile capacitive sensing wearables showcases the versatile application possibilities of our technology from single-joint angle measurements towards multi-joint body part tracking.
On-device Learning of EEGNet-based Network For Wearable Motor Imagery Brain-Computer Interface
Bian, Sizhen; Kang, Pixi; Moosmann, Julian; et al. (2024)
ISWC '24: Proceedings of the 2024 ACM International Symposium on Wearable Computers
Electroencephalogram (EEG)-based Brain-Computer Interfaces (BCIs) have garnered significant interest across various domains, including rehabilitation and robotics. Despite advancements in neural network-based EEG decoding, maintaining performance across diverse user populations remains challenging due to feature distribution drift. This paper presents an effective approach to address this challenge by implementing a lightweight and efficient on-device learning engine for wearable motor imagery recognition. The proposed approach, applied to the well-established EEGNet architecture, enables real-time and accurate adaptation to EEG signals from unregistered users. Leveraging the newly released low-power parallel RISC-V-based processor, GAP9 from Greeenwaves, and the Physionet EEG Motor Imagery dataset, we demonstrate a remarkable accuracy gain of up to 7.31% with respect to the baseline with a memory footprint of 15.6 KByte. Furthermore, by optimizing the input stream, we achieve enhanced real-time performance without compromising inference accuracy. Our tailored approach exhibits inference time of 14.9 ms and 0.76 mJ per single inference and 20 us and 0.83 uJ per single update during online training. These findings highlight the feasibility of our method for edge EEG devices as well as other battery-powered wearable AI systems suffering from subject-dependant feature distribution drift.
Evaluating Spiking Neural Network on Neuromorphic Platform for Human Activity Recognition
Bian, Sizhen; Magno, Michele (2023)
ISWC '23: Proceedings of the 2023 ACM International Symposium on Wearable Computers
Energy efficiency and low latency are crucial requirements for designing wearable AI-empowered human activity recognition systems, due to the hard constraints of battery operations and closed-loop feedback. While neural network models have been extensively compressed to match the stringent edge requirements, spiking neural networks and event-based sensing are recently emerging as promising solutions to further improve performance due to their inherent energy efficiency and capacity to process spatiotemporal data in very low latency. This work aims to evaluate the effectiveness of spiking neural networks on neuromorphic processors in human activity recognition for wearable applications. The case of workout recognition with wrist-worn wearable motion sensors is used as a study. A multi-threshold delta modulation approach is utilized for encoding the input sensor data into spike trains to move the pipeline into the event-based approach. The spikes trains are then fed to a spiking neural network with direct-event training, and the trained model is deployed on the research neuromorphic platform from Intel, Loihi, to evaluate energy and latency efficiency. Test results show that the spike-based workouts recognition system can achieve a comparable accuracy (87.5%) comparable to the popular milliwatt RISC-V bases multi-core processor GAP8 with a traditional neural network (88.1%) while achieving two times better energy-delay product (0.66 vs. 1.32 ).
Worker Activity Recognition in Manufacturing Line Using Near-Body Electric Field
Suh, Sungho; Rey, Vitor Fortes; Bian, Sizhen; et al. (2024)
IEEE Internet of Things Journal
Manufacturing industries strive to improve production efficiency and product quality by deploying advanced sensing and control systems. Wearable sensors are emerging as a promising solution for achieving this goal, as they can provide continuous and unobtrusive monitoring of workers' activities in the manufacturing line. This article presents a novel wearable sensing prototype that combines IMU and body capacitance sensing modules to recognize worker activities in the manufacturing line. To handle these multimodal sensor data, we propose and compare early, and late sensor data fusion approaches for multichannel time-series convolutional neural networks and deep convolutional LSTM. We evaluate the proposed hardware and neural network model by collecting and annotating sensor data using the proposed sensing prototype and Apple Watches in the testbed of the manufacturing line. Experimental results demonstrate that our proposed methods achieve superior performance compared to the baseline methods, indicating the potential of the proposed approach for real-world applications in manufacturing industries. Furthermore, the proposed sensing prototype with a body capacitive sensor (BCS) and feature fusion method improves by 6.35%, yielding a 9.38% higher macro F1 score than the proposed sensing prototype without a BCS and Apple Watch data, respectively.
Retina: Low-Power Eye Tracking with Event Camera and Spiking Hardware
Bonazzi, Pietro; Bian, Sizhen; Lippolis, Giovanni; et al. (2024)
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
This paper introduces a neuromorphic dataset and methodology for eye tracking, harnessing event data captured streamed continuously by a Dynamic Vision Sensor (DVS). The framework integrates a directly trained Spiking Neuron Network (SNN) regression model and leverages a state-of-the-art low power edge neuromorphic processor - Speck. First, it introduces a representative event-based eye-tracking dataset, "Ini-30," which was collected with two glass-mounted DVS cameras from thirty volunteers. Then, a SNN model, based on Integrate And Fire (IAF) neurons, named "Retina", is described , featuring only 64k parameters (6.63x fewer than 3ET) and achieving pupil tracking error of only 3.24 pixels in a 64x64 DVS input. The continuous regression output is obtained by means of temporal convolution using a non-spiking 1D filter slided across the output spiking layer over time. Retina is evaluated on the neuromorphic processor, showing an end-to-end power between 2.89-4.8 mW and a latency of 5.57-8.01 ms dependent on the time to slice the event-based video recording. The model is more precise than the latest event-based eye-tracking method, "3ET", on Ini-30, and shows comparable performance with significant model compression (35 times fewer MAC operations) in the synthetic dataset used in "3ET". We hope this work will open avenues for further investigation of close-loop neuromorphic solutions and true event-based training pursuing edge performance.
Initial Investigation of Kolmogorov-Arnold Networks (KANs) as Feature Extractors for IMU Based Human Activity Recognition
Liu, Mengxi; Geissler, Daniel; Nshimyimana, Dominique; et al. (2024)
UbiComp '24: Companion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing
In this work, we explore the use of a novel neural network architecture, the Kolmogorov-Arnold Networks (KANs) as feature extractors for sensor-based (specifically IMU) Human Activity Recognition (HAR). Where conventional networks perform a parameterized weighted sum of the inputs at each node and then feed the result into a statically defined nonlinearity, KANs perform non-linear computations represented by B-SPLINES on the edges leading to each node and then just sum up the inputs at the node. Instead of learning weights, the system learns the spline parameters. In the original work, such networks have been shown to be able to more efficiently and exactly learn sophisticated real valued functions e.g. in regression or partial differential equation (PDE) solution. We hypothesize that such an ability is also advantageous for computing low-level features for IMU-based HAR. To this end, we have implemented KAN as the feature extraction architecture for IMU-based human activity recognition tasks, including four architecture variations. We present an initial performance investigation of the KAN feature extractor on four public HAR datasets. It shows that the KAN-based feature extractor outperforms CNN-based extractors from CNN-MLP architecture-based models on all datasets while being more parameter efficient.
Q-Segment: Segmenting Images In-Sensor for Vessel-Based Medical Diagnosis
Bonazzi, Pietro; Li, Yawei; Bian, Sizhen; et al. (2024)
2024 IEEE 6th International Conference on AI Circuits and Systems (AICAS)
This paper addresses the growing interest in deploying deep learning models directly in-sensor. We present "Q-Segment", a quantized real-time segmentation algorithm, and conduct a comprehensive evaluation on a low-power edge vision platform with an in-sensors processor, the Sony IMX500. One of the main goals of the model is to achieve end-to-end image segmentation for vessel-based medical diagnosis. Deployed on the IMX500 platform, Q-Segment achieves ultra-low inference time in-sensor only 0.23 ms and power consumption of only 72mW. We compare the proposed network with state-of-the-art models, both float and quantized, demonstrating that the proposed solution outperforms existing networks on various platforms in computing efficiency, e.g., by a factor of 75x compared to ERFNet. The network employs an encoder-decoder structure with skip connections, and results in a binary accuracy of 97.25 % and an Area Under the Receiver Operating Characteristic Curve (AUC) of 96.97 % on the CHASE dataset. We also present a comparison of the IMX500 processing core with the Sony Spresense, a low-power multi-core ARM Cortex-M microcontroller, and a single-core ARM Cortex-M4 showing that it can achieve in-sensor processing with end-to-end low latency (17 ms) and power consumption (254mW). This research contributes valuable insights into edge-based image segmentation, laying the foundation for efficient algorithms tailored to low-power environments.
iEat: automatic wearable dietary monitoring with bio-impedance sensing
Liu, Mengxi; Zhou, Bo; Rey, Vitor Fortes; et al. (2024)
Scientific Reports
Diet is an inseparable part of good health, from maintaining a healthy lifestyle for the general population to supporting the treatment of patients suffering from specific diseases. Therefore it is of great significance to be able to monitor people’s dietary activity in their daily life remotely. While the traditional practices of self-reporting and retrospective analysis are often unreliable and prone to errors; sensor-based remote diet monitoring is therefore an appealing approach. In this work, we explore an atypical use of bio-impedance by leveraging its unique temporal signal patterns, which are caused by the dynamic close-loop circuit variation between a pair of electrodes due to the body-food interactions during dining activities. Specifically, we introduce iEat, a wearable impedance-sensing device for automatic dietary activity monitoring without the need for external instrumented devices such as smart utensils. By deploying a single impedance sensing channel with one electrode on each wrist, iEat can recognize food intake activities (e.g., cutting, putting food in the mouth with or without utensils, drinking, etc.) and food types from a defined category. The principle is that, at idle, iEat measures only the normal body impedance between the wrist-worn electrodes; while the subject is doing the food-intake activities, new paralleled circuits will be formed through the hand, mouth, utensils, and food, leading to consequential impedance variation. To quantitatively evaluate iEat in real-life settings, a food intake experiment was conducted in an everyday table-dining environment, including 40 meals performed by ten volunteers. With a lightweight, user-independent neural network model, iEat could detect four food intake-related activities with a macro F1 score of 86.4% and classify seven types of foods with a macro F1 score of 64.2%.
MoCaPose: Motion Capturing with Textile-integrated Capacitive Sensors in Loose-fitting Smart Garments
Zhou, Bo; Geissler, Daniel; Faulhaber, Marc; et al. (2023)
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
We present MoCaPose, a novel wearable motion capturing (MoCap) approach to continuously track the wearer's upper body's dynamic poses through multi-channel capacitive sensing integrated in fashionable, loose-fitting jackets. Unlike conventional wearable IMU MoCap based on inverse dynamics, MoCaPose decouples the sensor position from the pose system. MoCaPose uses a deep regressor to continuously predict the 3D upper body joints coordinates from 16-channel textile capacitive sensors, unbound by specific applications. The concept is implemented through two prototyping iterations to first solve the technical challenges, then establish the textile integration through fashion-technology co-design towards a design-centric smart garment. A 38-hour dataset of synchronized video and capacitive data from 21 participants was recorded for validation. The motion tracking result was validated on multiple levels from statistics (R2 ∼ 0.91) and motion tracking metrics (MP JPE ∼ 86mm) to the usability in pose and motion recognition (0.9 F1 for 10-class classification with unsupervised class discovery). The design guidelines impose few technical constraints, allowing the wearable system to be design-centric and usecase-specific. Overall, MoCaPose demonstrates that textile-based capacitive sensing with its unique advantages, can be a promising alternative for wearable motion tracking and other relevant wearable motion recognition applications.
Evaluation of Encoding Schemes on Ubiquitous Sensor Signal for Spiking Neural Network
Bian, Sizhen; Donati, Elisa; Magno, Michele (2024)
IEEE Sensors Journal
Spiking neural networks (SNNs), a brain-inspired computing paradigm, are emerging for their inference performance, particularly in terms of energy efficiency and latency attributed to the plasticity in signal processing. To deploy SNNs in ubiquitous computing systems, signal encoding of sensors is crucial for achieving high accuracy and robustness. Using inertial sensor readings for gym activity recognition as a case study, this work comprehensively evaluates four main encoding schemes and deploys the corresponding SNN on the neuromorphic processor Loihi2 for postdeployment encoding assessment. Rate encoding, time-to-first-spike (TTFS) encoding, binary encoding, and delta modulation are evaluated using metrics such as average fire rate, signal-to-noise ratio (SNR), classification accuracy, robustness, and inference latency and energy. In this case study, the TTFS encoding required the lowest firing rate (2%) and achieved a comparative accuracy (89%) although it was the least robust scheme against error spikes (over 20% accuracy drop with 0.1 noisy spike rate). Rate encoding with optimal value-to-probability mapping achieved the highest accuracy (91.7%). Binary encoding provided a balance between information reconstruction and noise resistance. Multithreshold delta modulation showed the best robustness, with only a 0.7% accuracy drop at a 0.1 noisy spike rate. This work serves researchers in selecting the best encoding scheme for SNN-based ubiquitous sensor signal processing, tailored to specific performance requirements.

Publications 1 - 10 of 18

Sizhen Bian

Last Name

First Name

ORCID

Organisational unit

Filters

Search Results