Mid Sweden University

Structural health monitoring (SHM) of aging civil, aerospace, and energy infrastructure increasingly relies on unmanned aerial vehicles (UAVs) equipped with vision sensors for efficient and large-scale inspections. Among these applications, automated crack classification using deep learning models has emerged as a key use case. However, cloud-based inference for such tasks imposes bandwidth, power, connectivity, and privacy costs that are unacceptable for safety-critical assets. To address these limitations, this study presents a fully self-contained Tiny Machine Learning (TinyML) solution that performs onboard crack classification on a milliwatt-level STM32H7 microcontroller (MCU). Using MobileNetV1x0.25 as a baseline, we systematically evaluate an end-to-end measurement processing pipeline, including image capturing, image preprocessing, and model inference on a low-power embedded system. To identify the optimal pipeline configuration, we compare two image preprocessing strategies consisting of a handcrafted grayscale–contrast–denoise–median–binarization method and a greedy algorithm–based composite approach. We further assess four model compression techniques, including 8-bit post-training quantization (PTQ), quantization-aware training (QAT), pruning, and weight clustering, both individually and in combination. The proposed pipeling achieves an F1-score of 0.938, which outperforms the state-of-the-art by 11.4\%. At the same time, it only requires 2.9 MB of RAM and 309 KB of flash memory. The deployed solution has an end-to-end latency of 461.6 ms and an energy cost of 623.16 mJ per inference. For a DJI Mini 4 Pro UAV, continuous operation is estimated to shorten the flight time by merely 1.31 minutes (i.e., 4\%). In contrast, previously reported deployments based on NVIDIA Jetson NX implementations reduce flight time by 8 minutes (i.e., 24\%). This work thus provides a reproducible benchmark and a practical trade-off of accuracy, resource usage, and energy consumption for on-device crack classification in highly resource-constrained, UAV-based SHM scenarios.

Keyword spotting (KWS) is a key component of smart devices, enabling efficient and intuitive audio interaction. However, standard KWS systems deployed on embedded devices often suffer performance degradation under real-world operating conditions. Resilient KWS systems address this issue by enabling dynamic adaptation, with applications such as adding or replacing keywords, adjusting to specific users, and improving noise robustness. However, deploying resilient, standalone KWS systems with low latency on resource-constrained devices remains challenging due to limited memory and computational resources. This study proposes a low computational approach for continuous noise adaptation of pretrained neural networks used for KWS classification, requiring only 1-shot learning and one epoch. The proposed method was assessed using two pretrained models and three real-world noise sources at signal-to-noise ratios (SNRs) ranging from 24 to -3 dB. The adapted models consistently outperformed the pretrained models across all scenarios, especially at SNR≤18 dB, achieving accuracy improvements of 4.9% to 46.0%. These results highlight the efficacy of the proposed methodology while being lightweight enough for deployment on resource-constrained devices.

The mining sector harnesses advancements in automation, digitalization, and interconnected technologies from Industry 4.0 to enhance efficiency, safety, and sustainability. Conveyor belts play a critical role in mining operations, facilitating the continuous and efficient transport of bulk materials over long distances, directly impacting productivity. While anomaly detection in specific conveyor belt components has been extensively studied, continuous monitoring to identify root causes of failures remains in its early stages. Existing methods for anomaly detection in mining conveyor belt duty cycles rely on supervised machine learning (ML) to classify internal machine modes as an intermediate step. While these approaches offer high explainability, they are constrained by the need for extensive labeled data for internal machine modes. This study proposes a novel pattern recognition approach combining unsupervised and supervised ML models for real-time anomaly detection in conveyor belt operational cycles. By evaluating combinations of TinyML models, the approach achieved average F1-scores of 83.2% for abnormal cycles and 97.0% for normal cycles, surpassing the state-of-the-art by 11.4% and 3.3%, respectively. Deployed on low-power microcontrollers, the proposed methods demonstrated efficient, real-time operation, reducing energy consumption by up to 84.5% (4.1 μJ per inference) and program memory usage by up to 72.1%. These results provide valuable insights for detecting early mechanical failures and enabling targeted preventive maintenance. 

Keyword spotting (KWS) is an essential function that enables interaction with ubiquitous smart devices. However, in resource-limited devices, KWS models are often static and can thus not adapt to new scenarios, such as added keywords. To overcome this problem, we propose a Continual Learning (CL) approach for KWS built on Binary Neural Networks (BNNs). The framework leverages the reduced computation and memory requirements of BNNs while incorporating techniques that enable the seamless integration of new keywords overtime. This study evaluates seven CL techniques on a 16-classuse case, reporting an accuracy exceeding 95% for a single additional keyword and up to 86% for four additional classes. Sensitivity to the amount of training samples in the CL phase, and differences in computational complexities are being evaluated. These evaluations demonstrate that batch-based algorithms are more sensitive to the CL dataset size, and that differences between the computational complexities are insignificant. These findings highlight the potential of developing an effective and computationally efficient technique for continuously integrating new keywords in KWS applications that is compatible with resource-constrained devices.

Conveyor belts are crucial in mining operations by enabling the continuous and efficient movement of bulk materials over long distances, which directly impacts productivity. While detecting anomalies in specific conveyor belt components has been widely studied, identifying the root causes of these failures, such as changing production conditions and operator errors, remains critical. Continuous monitoring of mining conveyor belt work cycles is still at an early stage and requires robust solutions. Recently, an anomaly detection method for duty cycle operations of a mining conveyor belt has been proposed. Based on its limited performance and unevaluated long-term proper operation, this study proposes two novel methods for classifying normal and abnormal duty cycles. The proposed approaches are pattern recognition systems that make use of threshold-based duty-cycle detection mechanisms, manually extracted features, pattern-matching, and supervised tiny machine learning models. The explored low-computational models include decision tree, random forest, extra trees, extreme gradient boosting, Gaussian naive Bayes, and multi-layer perceptron. A comprehensive evaluation of the former and proposed approaches is carried out on two datasets. Both proposed methods outperform the former method in anomaly detection, with the best-performing approach being dataset-dependent. The heuristic rule-based approach achieves the highest F1-score in the same dataset used for algorithm training, with 97.3% for normal cycles and 80.2% for abnormal cycles. The ML-based approach performs better on a dataset including the effects of machine aging, with an F1-score scoring 91.3% for normal cycles and 67.9% for abnormal cycles. Implemented on two low-power microcontrollers, the methods demonstrate efficient, real-time operation with energy consumption of 13.3 and 20.6 J during inference. These results offer valuable insights for detecting mechanical failure sources, supporting targeted preventive maintenance, and optimizing production cycles. 

Crack segmentation can play a critical role in Structural Health Monitoring (SHM) by enabling accurate identification of crack size and location, which allows to monitor structural damages over time. However, deploying deep learning models for crack segmentation on resource-constrained micro controllers presents significant challenges due to limited memory, computational power, and energy resources. To address these challenges, this study explores lightweight U-Net architectures tailored for TinyML applications, focusing on three optimization strategies: filter number reduction, network depth reduction, and the use of Depthwise Separable Convolutions (DWConv2D). Our results demonstrate that reducing convolution kernels and network depth significantly reduces RAM and Flash requirement, and inference times, albeit with some accuracy trade-offs. Specifically, by reducing the filer number to 25%, the network depth to four blocks, and utilizing depthwise convolutions, a good compromise between segmentation performance and resource consumption is achieved. This makes the network particularly suitable for low-power TinyML applications. This study not only advances TinyML-based crack segmentation but also provides the possibility for energy-autonomous edge SHM systems.

Acoustic Emission (AE)-based fault diagnosis in Structural Health Monitoring (SHM) systems faces challenges of data scarcity and model overfitting due to the complexity of AE data acquisition and the high cost of labeling. To address these issues, this study systematically explores various data augmentation techniques for AE signal processing and evaluates their impact on model robustness and accuracy. Furthermore, given the complexity of traditional machine learning (ML) models and their deployment challenges on resource-constrained embedded devices, we investigate lightweight ML algorithms and propose a Tiny Machine Learning (TinyML)-based fault diagnosis approach. Experimental validation on a carbon fiber panel fault diagnosis case demonstrates that the proposed method significantly improves classification performance under data scarce conditions while enabling real-time fault diagnosis on embedded systems. These findings underscore the potential of integrating data augmentation, lightweight ML algorithms, and TinyML to enhance both diagnostic accuracy and real-time performance in SHM applications. 

Structural Health Monitoring (SHM) ensures the safety and longevity of infrastructure by enabling timely damage detection. Vision-based crack detection, combined with UAVs, addresses the limitations of traditional sensor-based SHM methods but requires the deployment of efficient deep learning models on resource-constrained devices. This study evaluates two lightweight convolutional neural network models, MobileNetV1x0.25 and MobileNetV2x0.5, across TensorFlow, PyTorch, and Open Neural Network Exchange platforms using three quantization techniques: dynamic quantization, post-training quantization (PTQ), and quantization-aware training (QAT). Results show that QAT consistently achieves near-floating-point accuracy, such as an F1-score of 0.8376 for MBNV2x0.5 with Torch-QAT, while maintaining efficient resource usage. PTQ significantly reduces memory and energy consumption but suffers from accuracy loss, particularly in TensorFlow. Dynamic quantization preserves accuracy but faces deployment challenges on PyTorch. By leveraging QAT, this work enables real-time, low-power crack detection on UAVs, enhancing safety, scalability, and cost-efficiency in SHM applications, while providing insights into balancing accuracy and efficiency across different platforms for autonomous inspections. 

Energy harvesting has been proposed to support or replace battery-based energy supplies in low-power loT systems. However, manual design and optimization is currently required to implement energy harvesters. This implies significant time and cost requirements, reducing the competitiveness of energy harvesting solutions in many applications. In this paper, we propose and study a method for the automated design of ortho-planar springs in vibration energy harvesters. The proposed method uses a Python tool that has been developed to translate parametric descriptions of the spring into 2D and 3D spring CAD models. The tool is combined with a black-box optimization algorithm in order to identify optimized spring parameters. An evaluation of the proposed method on a case study of an electromagnetic vibration energy harvester demonstrates that the approach successfully results in spring designs that perform well in the energy harvesting application. Consequently, the requirement of expert competence to adjust an energy harvester to new application constraints is significantly reduced, contributing to the availability and competitiveness of energy harvesting solutions in real world applications. 

This paper proposes and investigates a novel electro-magnetic vibration energy harvester using ortho-planar springs. Addressing the limited output power of previously reported designs, the proposed harvester targets milliwatt level power outputs to be relevant for practical sensing and IoT applications. The proposed design employs an optimized pickup unit with Halbach array configuration and an optimized ortho-planar spring, which are integrated in a 3D-printed housing. During experimental evaluations, the harvester demonstrates an output power of up to 5.26mW and a normalized power density of 133.05 μW/(cm3∗g2), exceeding the performance of previously reported harvesters with ortho-planar springs. Moreover, exploiting the versatility of ortho-planar spring designs, it is shown that the harvester's frequency response can easily be adjusted to different application conditions.