Mid Sweden University

miun.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Drill Failure Detection based on Sound using Artificial Intelligence
Mid Sweden University, Faculty of Science, Technology and Media, Department of Electronics Design. (AISound – Akustisk sensoruppsättning för AI-övervakningssystem)ORCID iD: 0000-0002-8262-2414
2021 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

In industry, it is crucial to be able to detect damage or abnormal behavior in machines. A machine's downtime can be minimized by detecting and repairing faulty components of the machine as early as possible. It is, however, economically inefficient and labor-intensive to detect machine fault sounds manual. In comparison with manual machine failure detection, automatic failure detection systems can reduce operating and personnel costs.  Although prior research has identified many methods to detect failures in drill machines using vibration or sound signals, this field still remains many challenges. Most previous research using machine learning techniques has been based on features that are extracted manually from the raw sound signals and classified using conventional classifiers (SVM, Gaussian mixture model, etc.). However, manual extraction and selection of features may be tedious for researchers, and their choices may be biased because it is difficult to identify which features are good and contain an essential description of sounds for classification. Recent studies have used LSTM, end-to-end 1D CNN, and 2D CNN as classifiers for classification, but these have limited accuracy for machine failure detection. Besides, machine failure occurs very rarely in the data. Moreover, the sounds in the real-world dataset have complex waveforms and usually are a combination of noise and sound presented at the same time.

Given that drill failure detection is essential to apply in the industry to detect failures in machines, I felt compelled to propose a system that can detect anomalies in the drill machine effectively, especially for a small dataset. This thesis proposed modern artificial intelligence methods for the detection of drill failures using drill sounds provided by Valmet AB. Instead of using raw sound signals, the image representations of sound signals (Mel spectrograms and log-Mel spectrograms) were used as the input of my proposed models. For feature extraction, I proposed using deep learning 2-D convolutional neural networks (2D-CNN) to extract features from image representations of sound signals. To classify three classes in the dataset from Valmet AB (anomalous sounds, normal sounds, and irrelevant sounds), I proposed either using conventional machine learning classifiers (KNN, SVM, and linear discriminant) or a recurrent neural network (long short-term memory). For using conventional machine learning methods as classifiers, pre-trained VGG19 was used to extract features and neighborhood component analysis (NCA) as the feature selection. For using long short-term memory (LSTM), a small 2D-CNN was proposed to extract features and used an attention layer after LSTM to focus on the anomaly of the sound when the drill changes from normal to the broken state. Thus, my findings will allow readers to detect anomalies in drill machines better and develop a more cost-effective system that can be conducted well on a small dataset.

There is always background noise and acoustic noise in sounds, which affect the accuracy of the classification system. My hypothesis was that noise suppression methods would improve the sound classification application's accuracy. The result of my research is a sound separation method using short-time Fourier transform (STFT) frames with overlapped content. Unlike traditional STFT conversion, in which every sound is converted into one image, a different approach is taken. In contrast, splitting the signal into many STFT frames can improve the accuracy of model prediction by increasing the variability of the data. Images of these frames separated into clean and noisy ones are saved as images, and subsequently fed into a pre-trained CNN for classification. This enables the classifier to become robust to noise. The FSDNoisy18k dataset is chosen in order to demonstrate the efficiency of the proposed method. In experiments using the proposed approach, 94.14 percent of 21 classes were classified successfully, including 20 classes of sound events and a noisy class.

Place, publisher, year, edition, pages
Sundsvall, Sweden: Mid Sweden University , 2021. , p. 46
Series
Mid Sweden University licentiate thesis, ISSN 1652-8948 ; 188
Keywords [en]
Convolutional neural network, machine failure detection, Mel-spectrogram, long short-term memory, sound signal processing
National Category
Other Computer and Information Science Computer Sciences
Identifiers
URN: urn:nbn:se:miun:diva-43841ISBN: 978-91-89341-37-1 (print)OAI: oai:DiVA.org:miun-43841DiVA, id: diva2:1614190
Presentation
2021-12-16, C312, Holmgatan 10, Sundsvall, 13:00 (English)
Opponent
Supervisors
Projects
AISound – Akustisk sensoruppsättning för AI-övervakningssystemMiLo — miljön i kontrolloopen
Note

Vid tidpunkten för disputationen var följande delarbeten opublicerade: delarbete 2 och 3 inskickat.

At the time of the doctoral defence the following papers were unpublished: paper 2 and 3 submitted.

Available from: 2021-11-25 Created: 2021-11-24 Last updated: 2021-11-25Bibliographically approved
List of papers
1. Drill Fault Diagnosis Based on the Scalogram and Mel Spectrogram of Sound Signals Using Artificial Intelligence
Open this publication in new window or tab >>Drill Fault Diagnosis Based on the Scalogram and Mel Spectrogram of Sound Signals Using Artificial Intelligence
2020 (English)In: IEEE Access, E-ISSN 2169-3536, Vol. 8, p. 203655-203666Article in journal (Refereed) Published
Abstract [en]

In industry, the ability to detect damage or abnormal functioning in machinery is very important. However, manual detection of machine fault sound is economically inefficient and labor-intensive. Hence, automatic machine fault detection (MFD) plays an important role in reducing operating and personnel costs compared to manual machine fault detection. This research aims to develop a drill fault detection system using state-of-the-art artificial intelligence techniques. Many researchers have applied the traditional approach design for an MFD system, including handcrafted feature extraction of the raw sound signal, feature selection, and conventional classification. However, drill sound fault detection based on conventional machine learning methods using the raw sound signal in the time domain faces a number of challenges. For example, it can be difficult to extract and select good features to input in a classifier, and the accuracy of fault detection may not be sufficient to meet industrial requirements. Hence, we propose a method that uses deep learning architecture to extract rich features from the image representation of sound signals combined with machine learning classifiers to classify drill fault sounds of drilling machines. The proposed methods are trained and evaluated using the real sound dataset provided by the factory. The experiment results show a good classification accuracy of 80.25 percent when using Mel spectrogram and scalogram images. The results promise significant potential for using in the fault diagnosis support system based on the sounds of drilling machines.

Keywords
Deep learning, machine fault diagnosis, machine learning, sound signal processing
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:miun:diva-40655 (URN)10.1109/ACCESS.2020.3036769 (DOI)000590435900001 ()2-s2.0-85102866677 (Scopus ID)
Available from: 2020-12-03 Created: 2020-12-03 Last updated: 2023-08-31
2.
The record could not be found. The reason may be that the record is no longer available or you may have typed in a wrong id in the address field.
3. Separate Sound into STFT Frames to Eliminate Sound Noise Frames in Sound Classification
Open this publication in new window or tab >>Separate Sound into STFT Frames to Eliminate Sound Noise Frames in Sound Classification
Show others...
2021 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Sounds always contain acoustic noise and background noise that affects the accuracy of the sound classification system. Hence, suppression of noise in the sound can improve the robustness of the sound classification model. This paper investigated a sound separation technique that separates the input sound into many overlapped-content Short-Time Fourier Transform (STFT) frames. Our approach is different from the traditional STFT conversion method, which converts each sound into a single STFT image. Contradictory, separating the sound into many STFT frames improves model prediction accuracy by increasing variability in the data and therefore learning from that variability. These separated frames are saved as images and then labeled manually as clean and noisy frames which are then fed into transfer learning convolutional neural networks (CNNs) for the classification task. The pre-trained CNN architectures that learn from these frames become robust against the noise. The experimental results show that the proposed approach is robust against noise and achieves 94.14% in terms of classifying 21 classes including 20 classes of sound events and a noisy class. An open-source repository of the proposed method and results is available at https://github.com/nhattruongpham/soundSepsound.

Keywords
Sound separation, Sound classification, Short Time Fourier Transform, Transfer learning.
National Category
Computer Sciences Signal Processing
Identifiers
urn:nbn:se:miun:diva-43839 (URN)10.1109/SSCI50451.2021.9660125 (DOI)000824464300302 ()2-s2.0-85125765216 (Scopus ID)
Conference
IEEE Symposium Series on Computational Intelligence (IEEE SSCI 2021), Orlando, USA, [DIGITAL], December 5-7, 2021.
Available from: 2022-03-20 Created: 2021-11-24 Last updated: 2023-08-31Bibliographically approved

Open Access in DiVA

fulltext(7497 kB)663 downloads
File information
File name FULLTEXT02.pdfFile size 7497 kBChecksum SHA-512
d9800e894a038a3d5a42cac92965e033f1cb0f23792b9dbacff84d94cafa7c3062f3c985b24093459dfdbd546af757baf45664b0ec1469fd18744007e594a079
Type fulltextMimetype application/pdf

Authority records

Tran, Thanh

Search in DiVA

By author/editor
Tran, Thanh
By organisation
Department of Electronics Design
Other Computer and Information ScienceComputer Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 667 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 1080 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf