Mid Sweden University

miun.sePublications
Operational message
There are currently operational disruptions. Troubleshooting is in progress.
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Deep Learning Approaches towards Skin Lesion Classification with Dermoscopic Images
Mid Sweden University, Faculty of Science, Technology and Media, Department of Computer and Electrical Engineering (2023-).
2023 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Melanoma is a skin cancer that tends to be deadly. The incidence of melanoma is currently at the highest level ever recorded in Europe, North America and Oceania. The survival rate can be significantly increased if skin lesions are identified in dermoscopic images at an early stage. In the other hand, the classification of skin lesions is incredibly challenging. Skin lesion classification using deep learning approaches has provided better results in classifying skin diseases than those of dermatologist, which is lifesaving in terms of diagnosis.

This thesis presents a review of our research articles on classifying skin lesions using deep learning. Regarding the research, I have four goals concerning research frontier work, small datasets, data imbalance, and improving accuracy. In this thesis, I discuss how deep learning can classify skin diseases, summarizing the problems that remain at this stage and the outlook for the future.

For the above goals, I first studied and summarized more than 200 highguality articles published over five years. I then used three versions of You only look once (Yolo) to detect skin lesions. Although there were only 200 pictures, the test was very effective for detection. I applied the five-fold algorithm to Vgg_16, trained five models, and fused them so solve the small data problem. To improve the accuracy, I also tried to combine the traditional machine learning method, i.e., the seven-point checklist, with three different backbones. Since the learning rate. Then, I also tried to use the hybrid model, combining convolutional neural networks (CNN) and transformer to train the dataset, and applied focal loss to balance the extremely unbalanced weight of the data.

In addition to high-quality data sets and high-performance computers being extremely important in the research and application of deep learning, the optimization of machine learning algorithms for skin lesions can be endless

Abstract [sv]

Melanom är en form av hudcancer som tenderar att vara dödlig. Förekomsten av melanom är för närvarande på den högsta nivån som någonsin registrerats i Europa, Nordamerika och Oceanien. Chansen för överlevnad ökar avsevärt om hudskadorna identifieras i dermatoskopiska bilder i ett tidigare skede, men klassificering av hudskador är otroligt utmanande. Med metoder för djupinlärning har klassificering av hudsjukdomar i vissa fall gett bättre resultat än hudläkares diagnoser, vilket ger större möjligheter att rädda liv.

Denna avhandling presenterar en genomgång av våra forskningsartiklar om klassificering av hudskador med hjälp av djupinlärning. När det gäller vår forskning har jag fyra mål som handlar om forskningens frontlinjearbete, små datamängder, obalans i data och om att förbättra noggrannheten. I detta avhandlingsarbete diskuterar jag hur djupinlärning kan klassificera hudsjukdomar, sammanfattar de problem som kvarstår i detta skede och diskuterar utsikterna för framtiden.

För ovanstående mål studerade och sammanfattade jag först mer än 200 högkvalitativa artiklar publicerade under fem år. Jag använde sedan tre versioner av You only look once (Yolo) för att upptäcka hudskador. Även om det bara fanns 200 bilder var testet mycket effektivt för upptäckt. Jag tillämpade en femdelad algoritm på Vhh-16, tränade fem modeller och sammanfogade dem för att lösa problemet med små datamängder. För att förbättra noggrannheten försökte jag också kombinera en sjupunkts checklista, förstärkt med maskininlärning, med tre olika grundstommar. Eftersom inlärningshastigheten starkt påverkar modellträningen använde jag cosinus-inlärningshastigheten. Sedan försökte jag också använda hybridmodellen, som kombinerade konvolutionella neurala nätverk (CNN) och transformator för att träna dataset, och tillämpade fokalförlust för att balansera den extremt obalanserade vikten av datan.

Förutom att högkvalitativa datamängder och högpresterande datorer är extremt viktiga i forskningen och tillämpningen av djupinlärning, kan optimeringen av maskininlärningsalgoritmer för hudskador vara oändliga.

Place, publisher, year, edition, pages
Sundsvall: Mid Sweden University , 2023. , p. 51
Series
Mid Sweden University doctoral thesis, ISSN 1652-893X ; 383
National Category
Medical Imaging
Identifiers
URN: urn:nbn:se:miun:diva-46957ISBN: 978-91-89341-86-9 (print)OAI: oai:DiVA.org:miun-46957DiVA, id: diva2:1729104
Public defence
2023-02-16, C312, Holmgatan 10, Sundsvall, 09:00 (English)
Supervisors
Available from: 2023-01-20 Created: 2023-01-19 Last updated: 2025-09-25Bibliographically approved
List of papers
1. Automatic Detection of Melanoma with Yolo Deep Convolutional Neural Networks
Open this publication in new window or tab >>Automatic Detection of Melanoma with Yolo Deep Convolutional Neural Networks
Show others...
2019 (English)In: 2019 E-Health and Bioengineering Conference (EHB), IEEE, 2019Conference paper, Published paper (Refereed)
Abstract [en]

In the past three years, deep convolutional neural networks (DCNNs) have achieved promising performance in detecting skin cancer. However, improving the accuracy and efficiency of the automatic detection of melanoma is still urgent due to the visual similarity of benign and malignant dermoscopy. There is also a need for fast and computationally effective systems for mobile applications targeting caregivers and homes. This paper presents the You Only Look Once (Yolo) algorithms, which are based on DCNNs applied to the detection of melanoma. The Yolo algorithms comprise YoloV1, YoloV2, and YoloV3, whose methodology first resets the input image size and then divides the image into several cells. According to the position of the detected object in the cell, the network will try to predict the bounding box of the object and the class confidence score. Our test results indicate that the mean average precision (mAP) of Yolo can exceed 0.82 with a training set of only 200 images, proving that this method has great advantages for detecting melanoma in lightweight system applications.

Place, publisher, year, edition, pages
IEEE, 2019
Keywords
Image processing, Melanoma, Yolo, Object Detection
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:miun:diva-38154 (URN)10.1109/EHB47216.2019.8970033 (DOI)000558648300163 ()2-s2.0-85079350541 (Scopus ID)978-1-7281-2603-6 (ISBN)
Conference
IEEE International Conference on e-Health and Bioengineering 2019, Romania, 21-23 November 2019
Funder
European Regional Development Fund (ERDF)
Available from: 2019-12-19 Created: 2019-12-19 Last updated: 2025-09-25Bibliographically approved
2. Deep Melanoma classification with K-Fold Cross-Validation for Process optimization
Open this publication in new window or tab >>Deep Melanoma classification with K-Fold Cross-Validation for Process optimization
Show others...
2020 (English)In: 2020 IEEE International Symposium on Medical Measurements and Applications (MeMeA), IEEE, 2020Conference paper, Published paper (Refereed)
Abstract [en]

Deep convolution neural networks (DCNNs) enable effective methods to predict the melanoma classes otherwise found with ultrasonic extraction. However, gathering large datasets in local hospitals in Sweden can take years. Small datasets will result in models with poor accuracy and insufficient generalization ability, which has a great impact on the result. This paper proposes to use a K-Fold cross validation approach based on a DCNN algorithm working on a small sample dataset. The performance of the model is verified via a Vgg16 extracting the features. The experimental results reveal that the model built by the approach proposed in this paper can effectively achieve a better prediction and enhance the accuracy of the model, which proves that K-Fold can achieve better performance on a small skin cancer dataset. 

Place, publisher, year, edition, pages
IEEE, 2020
Keywords
classification, DCNNs, K-Fold, melanoma
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:miun:diva-39610 (URN)10.1109/MeMeA49120.2020.9137222 (DOI)000612835700073 ()2-s2.0-85088904068 (Scopus ID)978-1-7281-5386-5 (ISBN)
Conference
2020 IEEE International Symposium on Medical Measurements and Applications (MeMeA)
Available from: 2020-08-17 Created: 2020-08-17 Last updated: 2025-09-25Bibliographically approved
3. Ensembling CNNs for dermoscopic analysis of suspicious skin lesions
Open this publication in new window or tab >>Ensembling CNNs for dermoscopic analysis of suspicious skin lesions
Show others...
2021 (English)In: 2021 IEEE International Symposium on Medical Measurements and Applications (MeMeA), IEEE, 2021Conference paper, Published paper (Refereed)
Abstract [en]

Deep Convolution Neural Networks (CNN) enable advanced methods to predict the skin cancer classes through the automatic analysis of digital dermoscopic images. However, small datasets' availability often allows the models to be characterized by low prediction accuracy and poor generalization ability, which significantly influences clinical decisions. This paper proposes to use an original ensembling of multiple CNNs as feature extractors able to detect and measure skin lesions atypical criteria according to the well-known diagnostic method 7-Point Check List. The experimental results show that the Artificial Intelligence-based model can suitably manage the classification uncertainty of the single CNNs and finally distinguish melanomas from benignant nevi. Diagnostic performance is promising in terms of sensitivity and specificity towards a decision-supporting system used by a dermatologist with low experience during clinical practice.

Place, publisher, year, edition, pages
IEEE, 2021
National Category
Medical Engineering
Identifiers
urn:nbn:se:miun:diva-41757 (URN)10.1109/MeMeA52024.2021.9478760 (DOI)000847048100106 ()2-s2.0-85114127281 (Scopus ID)978-1-6654-1914-7 (ISBN)
Conference
MeMeA 2021, The 16th edition of IEEE International Symposium on Medical Measurements and Applications, [DIGITAL] Neuchâtel, Switzerland, June 25-28, 2021.
Available from: 2021-03-29 Created: 2021-03-29 Last updated: 2025-09-25Bibliographically approved
4. Skin Cancer Classification based on Cosine Cyclical Learning Rate with Deep Learning
Open this publication in new window or tab >>Skin Cancer Classification based on Cosine Cyclical Learning Rate with Deep Learning
Show others...
2022 (English)In: Conference Record - IEEE Instrumentation and Measurement Technology Conference, IEEE, 2022Conference paper, Published paper (Refereed)
Abstract [en]

Since early-stage skin cancer identification can improve melanoma prognosis and significantly reduce treatment costs, AI-based diagnosis systems might greatly benefit patients suffering from suspicious skin lesions. The study proposes a cosine cyclical learning rate with a skin cancer classification model to improve melanoma prediction. The contributions of models involve three critical CNNs, which are standard deep feature extraction modules for the skin cancer classification in this study (Vgg19, ResNet101 and InceptionV3). Each CNN model applies three different learning rates: fixed learning rate(LR), Cosine Annealing LR, and Cosine Annealing with WarmRestarts. HAM10000 is a large collection of publicly available dermoscopic images dataset used for our experiments. The performance of the proposed approach was appraised through comparative experiments. The outcome has indicated that the proposed method has high efficiency in diagnosing skin lesions with a cosine cyclical learning rate. 

Place, publisher, year, edition, pages
IEEE, 2022
Keywords
cosine cyclical learning rate, deep learning, dermoscopic images, HAM10000, skin cancer
National Category
Computer Engineering Cancer and Oncology
Identifiers
urn:nbn:se:miun:diva-45757 (URN)10.1109/I2MTC48687.2022.9806568 (DOI)000844585400099 ()2-s2.0-85134427579 (Scopus ID)9781665483605 (ISBN)
Conference
2022 IEEE International Instrumentation and Measurement Technology Conference, I2MTC 2022, 16 May 2022 through 19 May 2022
Available from: 2022-08-03 Created: 2022-08-03 Last updated: 2025-09-25Bibliographically approved
5. Recent Advances in Diagnosis of Skin Lesions using Dermoscopic Images based on Deep Learning
Open this publication in new window or tab >>Recent Advances in Diagnosis of Skin Lesions using Dermoscopic Images based on Deep Learning
Show others...
2022 (English)In: IEEE Access, E-ISSN 2169-3536, Vol. 10, p. 95716-95747Article in journal (Refereed) Published
Abstract [en]

Skin cancer is one of the most threatening cancers, which spreads to the other parts of the body if not caught and treated early. During the last few years, the integration of deep learning into skin cancer has been a milestone in health care, and dermoscopic images are right at the center of this revolution. This review study focuses on the state-of-the-art automatic diagnosis of skin cancer from dermoscopic images based on deep learning. This work thoroughly explores the existing deep learning and its application in diagnosing dermoscopic images. This study aims to present and summarize the latest methodology in melanoma classification and the techniques to improve this. We discuss advancements in deep learning-based solutions to diagnose skin cancer, along with some challenges and future opportunities to strengthen these automatic systems to support dermatologists and enhance their ability to diagnose skin cancer. Author

Keywords
Biomedical imaging, Cancer, Classification, Convolutional neural networks, Deep learning, Dermatology, Dermoscopy images, Image color analysis, Image recognition, Lesions, Literature review, Melanoma, Skin, Skin cancer
National Category
Radiology, Nuclear Medicine and Medical Imaging Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:miun:diva-45974 (URN)10.1109/ACCESS.2022.3199613 (DOI)000860813300001 ()2-s2.0-85136647170 (Scopus ID)
Available from: 2022-09-06 Created: 2022-09-06 Last updated: 2025-09-25Bibliographically approved
6. A Deep CNN Transformer Hybrid Model for Skin Lesion Classification of Dermoscopic Images Using Focal Loss
Open this publication in new window or tab >>A Deep CNN Transformer Hybrid Model for Skin Lesion Classification of Dermoscopic Images Using Focal Loss
Show others...
2023 (English)In: Diagnostics, ISSN 2075-4418, Vol. 13, no 1, article id 72Article in journal (Refereed) Published
Abstract [en]

Skin cancers are the most cancers diagnosed worldwide, with an estimated > 1.5 million new cases in 2020. Use of computer-aided diagnosis (CAD) systems for early detection and classification of skin lesions helps reduce skin cancer mortality rates. Inspired by the success of the transformer network in natural language processing (NLP) and the deep convolutional neural network (DCNN) in computer vision, we propose an end-to-end CNN transformer hybrid model with a focal loss (FL) function to classify skin lesion images. First, the CNN extracts low-level, local feature maps from the dermoscopic images. In the second stage, the vision transformer (ViT) globally models these features, then extracts abstract and high-level semantic information, and finally sends this to the multi-layer perceptron (MLP) head for classification. Based on an evaluation of three different loss functions, the FL-based algorithm is aimed to improve the extreme class imbalance that exists in the International Skin Imaging Collaboration (ISIC) 2018 dataset. The experimental analysis demonstrates that impressive results of skin lesion classification are achieved by employing the hybrid model and FL strategy, which shows significantly high performance and outperforms the existing work. 

Keywords
deep learning, focal loss, hybrid model, skin lesion
National Category
Computer graphics and computer vision
Identifiers
urn:nbn:se:miun:diva-46860 (URN)10.3390/diagnostics13010072 (DOI)000908965800001 ()2-s2.0-85145859869 (Scopus ID)
Available from: 2023-01-17 Created: 2023-01-17 Last updated: 2025-09-25Bibliographically approved

Open Access in DiVA

skin lesion classification(9171 kB)1944 downloads
File information
File name FULLTEXT01.pdfFile size 9171 kBChecksum SHA-512
14f1ee2037bb7b24d97aba758b904a7e352ae4800ff7340ed718a144ac9941d620608927785b011ac69933580ee3cbbb6dfdf66c5ec4d97c99efc961bf5c03bb
Type fulltextMimetype application/pdf

Authority records

Nie, Yali

Search in DiVA

By author/editor
Nie, Yali
By organisation
Department of Computer and Electrical Engineering (2023-)
Medical Imaging

Search outside of DiVA

GoogleGoogle Scholar
Total: 1944 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 1347 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf