Mid Sweden University

Acoustic Emission (AE) technology, as one of the non-destructive Structural Health Monitoring (SHM) methods, is increasingly utilized for the damage prediction, classification, maintenance, and real-time monitoring of infrastructure. Addressing the need for low latency, power consumption and high portability, a novel approach has been adopted where processing algorithms are embedded close to the sensors on these devices. Continuous data monitoring and collection, coupled with data processing and interpretation comparable to human experts, are anticipated from the next generation of the Internet of Things and smart sensing systems. While Machine Learning (ML) and Deep Learning (DL) has been successfully applied in a number of domains including SHM, resource-constrained, low-power devices pose a challenge for computationally complex ML algorithm execution.

To explore the feasibility of deploying ML and DL algorithms on edge devices, this study first proposes a lightweight CNN model based on raw AE signals for concrete damage classification and evaluates its performance on an ultra-low-power microcontroller unit (MCU). Subsequently, to further simplify the algorithm and explore the adaptability across various MCU platforms, a raw AE signal-based Artificial Neural Network (ANN) model is proposed, and its deployment performance on multiple MCUs is assessed. Additionally, the study assesses the impact of feature extraction on ANN performance with raw AE signals on MCUs, finding that using raw data directly is more resource and time-efficient. Lastly, the study investigates the generalization ability of the aforementioned CNN on a carbon fiber panel AE dataset, as well as the performance of 13 traditional ML algorithms on this dataset and their final deployment performance on MCUs. Due to the small size of the dataset, various data augmentation methods were also introduced and their impact on model robustness and accuracy was evaluated.

This thesis demonstrates for the first time that real-time inference on edge devices using AE signals for SHM is feasible. It also effectively demonstrates how to balance the critical trade-offs between accuracy, resource demands, and power consumption. Different MCUs and signal preprocessing methods are evaluated, and the impact of various data augmentation techniques on the accuracy of different ML algorithms and their inference robustness is explored in response to the challenge of collecting AE data, which is crucial for the next generation of SHM devices.

Structural Health Monitoring (SHM) plays a pivotal role in ensuring the safety, longevity, and reliability of critical infrastructure. While machine learning (ML) and deep learning (DL) have shown promise in automating SHM tasks, most existing approaches are tailored for high-resource computing platforms and overlook the unique constraints posed by low-power, resource-constrained embedded systems such as microcontrollers (MCUs) and edge devices. This thesis addresses this critical gap by systematically exploring, developing, and optimizing lightweight ML and DL algorithms for efficient on-device intelligence in SHM applications, particularly focusing on acoustic emission (AE)-based and vision-based modalities under the emerging framework of Tiny Machine Learning (TinyML).

The research is organized around three central objectives. First, both lightweight Artificial Neural Networks and Convolutional Neural Networks (CNNs) models are designed and optimized for real-time damage classification using raw AE signals and crack detection tasks using vision data. These models are tailored for deployment on highly resource-constrained MCUs, with particular attention to memory efficiency, inference latency, and energy consumption. Second, the thesis investigates the impact of model complexity and applies a range of model compression techniques such as post-training quantization (PTQ), quantization-aware training (QAT), pruning, and weight clustering. A comprehensive benchmark across various TinyML-compatible toolchains (TensorFlow Lite, STM32Cube.AI, ONNX) evaluates the performance trade-offs of these techniques in real deployment scenarios. Third, recognizing the scarcity of labeled AE data in real-world applications, the research explores both traditional and deep learning-based data augmentation methods, including jittering, warping, and DCGAN-generated synthetic signals, to improve model robustness and generalization.

The study includes extensive empirical evaluations using multiple publicly available AE and image datasets, covering diverse damage types in concrete and composite materials. On the vision side, UAV-based crack detection and segmentation pipelines are implemented using lightweight CNN backbones such as MobileNetV1x0.25 and U-Net variants. These models are trained and validated with various image preprocessing strategies, including grayscale conversion, contrast adjustment, denoising, and morphological operations. A greedy algorithm is also introduced to explore optimal preprocessing combinations. Performance is assessed through metrics such as F1-score, precision, recall, mIoU, model size, inference latency, memory footprint, and per-inference energy consumption, both before and after deployment to embedded devices like OpenMV and STM32 platforms.

The findings demonstrate that, with careful model design, quantization, and data augmentation, it is feasible to achieve high-performance SHM inference directly on-device. Notably, certain models maintained classification accuracies above 90% while reducing energy consumption by more than 60% through compression. For vision tasks, preprocessing pipelines significantly enhanced crack detection accuracy, and deployment on UAV platforms confirmed the viability of real-time aerial inspections with negligible impact on flight time.

In summary, this thesis makes significant contributions to the field of embedded SHM by bridging the gap between algorithmic advances and real-world deployment. It provides a structured pipeline for transitioning from conventional, cloud-centric SHM workflows to fully embedded, energy efficient, and real-time monitoring systems empowered by TinyML. The insights gained herein are expected to inform future developments in both civil infrastructure diagnostics and low-power AI systems.