Mid Sweden University

miun.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Semi-Autonomous Navigation of Powered Wheelchairs: 2D/3D Sensing and Positioning Methods
Mid Sweden University, Faculty of Science, Technology and Media, Department of Electronics Design. (IoT-system)ORCID iD: 0000-0002-1167-8322
2021 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Autonomous driving and assistance systems have become a reality for the automotive industry to improve driving safety in the car. Hence, the cars use a variety of sensors, cameras and image processing techniques to measure their surroundings and control their direction, braking and speed for obstacle avoidance or autonomously driving applications.Like the automotive industry, powered wheelchairs also require safety systems to ensure their operation, especially when the user has controlling limitations, but also to develop new applications to improve its usability. One of the applications is focused on developing a new contactless control of a powered wheelchair using the position of a caregiver beside it as a control reference. Contactless control can prevent control errors, but it can also provide better and more equal communication between the wheelchair user and the caregiver

This thesis evaluates the camera requirements for a contactless powered wheelchair control and the 2D/3D image processing techniques for caregiver recognition and position measurement beside the powered wheelchair. The research evaluates the strength and limitations of different depth camera technologies for caregiver feet detection above the ground plane to select the proper camera for the application. Then, a hand-crafted 3D object descriptor is evaluated for caregiver feet recognition and compared with respect to a state-of-the-art deep learning object detector. Results for both methods are good, however, the hand-crafted descriptor suffers from segmentation errors and consequently, their accuracy is lower. After the depth camera and image processing techniques evaluation, results show that it is possible to use only an RGB camera to recognize and measure his or her relative position.

Place, publisher, year, edition, pages
Sundsvall: Mid Sweden University , 2021. , p. 64
Series
Mid Sweden University doctoral thesis, ISSN 1652-893X
Keywords [en]
3D object recognition, YOLO, YOLO-Tiny, 3DHOG, Histogram-of-Oriented-Gradients, ModelNet40, Feature descriptor, Intel RealSense, Depth camera, Wheelchair
National Category
Computer graphics and computer vision
Identifiers
URN: urn:nbn:se:miun:diva-43829ISBN: 978-91-89341-32-6 (print)OAI: oai:DiVA.org:miun-43829DiVA, id: diva2:1613896
Public defence
2021-12-09, O102, Mittuniversitetet, Sundsvall, 16:06 (English)
Opponent
Supervisors
Note

Vid tidpunkten för disputationen var följande delarbeten opublicerade: delarbete 5 inskickat.

At the time of the doctoral defence the following papers were unpublished: paper 5 submitted.

Available from: 2021-11-24 Created: 2021-11-23 Last updated: 2025-02-07Bibliographically approved
List of papers
1. Evaluation of embedded camera systems for autonomous wheelchairs
Open this publication in new window or tab >>Evaluation of embedded camera systems for autonomous wheelchairs
2019 (English)In: VEHITS 2019 - Proceedings of the 5th International Conference on Vehicle Technology and Intelligent Transport Systems, SciTePress , 2019, p. 76-85Conference paper, Published paper (Refereed)
Abstract [en]

Autonomously driving Power Wheelchairs (PWCs) are valuable tools to enhance the life quality of their users. In order to enable truly autonomous PWCs, camera systems are essential. Image processing enables the development of applications for both autonomous driving and obstacle avoidance. This paper explores the challenges that arise when selecting a suitable embedded camera system for these applications. Our analysis is based on a comparison of two well-known camera principles, Stereo-Cameras (STCs) and Time-of-Flight (ToF) cameras, using the standard deviation of the ground plane at various lighting conditions as a key quality measure. In addition, we also consider other metrics related to both the image processing task and the embedded system constraints. We believe that this assessment is valuable when choosing between using STC or ToF cameras for PWCs.

Place, publisher, year, edition, pages
SciTePress, 2019
Keywords
Autonomous Wheelchair, Embedded Camera System, RANSAC, Stereo Camera, Time-of-Flight, Cameras, Embedded systems, Intelligent systems, Intelligent vehicle highway systems, Quality control, Traffic control, Wheelchairs, Camera systems, Stereo cameras, Time of flight, Stereo image processing
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:miun:diva-36685 (URN)000570379100008 ()2-s2.0-85067542836 (Scopus ID)9789897583742 (ISBN)
Conference
5th International Conference on Vehicle Technology and Intelligent Transport Systems, VEHITS 2019, Heraklion, Crete, Greece, 3 May 2019 through 5 May 2019
Projects
SMART (Smarta system och tjänster för ett effektivt och innovativt samhälle)
Available from: 2019-07-09 Created: 2019-07-09 Last updated: 2021-11-23Bibliographically approved
2. Rotational Invariant Object Recognition for Robotic Vision
Open this publication in new window or tab >>Rotational Invariant Object Recognition for Robotic Vision
2019 (English)In: ICACR 2019 Proceedings of the 2019 3rd International Conference on Automation, Control and Robots, ACM Digital Library, 2019, p. 1-6Conference paper, Published paper (Refereed)
Abstract [en]

Depth cameras have enhanced the environment perception for robotic applications significantly. They allow to measure true distances and thus enable a 3D measurement of the robot surroundings. In order to enable robust robot vision, the objects recognition has to handle rotated data because object can be viewed from different dynamic perspectives when the robot is moving. Therefore, the 3D descriptors used of object recognition for robotic applications have to be rotation invariant and implementable on the embedded system, with limited memory and computing resources. With the popularization of the depth cameras, the Histogram of Gradients (HOG) descriptor has been extended to recognize also 3D volumetric objects (3DVHOG). Unfortunately, both version are not rotation invariant. There are different methods to achieve rotation invariance for 3DVHOG, but they increase significantly the computational cost of the overall data processing. Hence, they are unfeasible to be implemented in a low cost processor for real-time operation. In this paper, we propose an object pose normalization method to achieve 3DVHOG rotation invariance while reducing the number of processing operations as much as possible. Our method is based on Principal Component Analysis (PCA) normalization. We tested our method using the Princeton Modelnet10 dataset.

Place, publisher, year, edition, pages
ACM Digital Library, 2019
Keywords
3D Object Recognition, Histogram of Gradients, Princeton Modelnet10, Principal Component Analysis, Pose Normalization, Image Processing, Depth Camera
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:miun:diva-37973 (URN)10.1145/3365265.3365273 (DOI)000651280300001 ()2-s2.0-85117541596 (Scopus ID)978-1-4503-7288-6 (ISBN)
Conference
2019 3rd International Conference on Automation, Control and Robots, Prague, Czech Republic, 11-13 October, 2019
Available from: 2019-12-12 Created: 2019-12-12 Last updated: 2022-06-01Bibliographically approved
3. Processing chain for 3D histogram of gradients based real-time object recognition
Open this publication in new window or tab >>Processing chain for 3D histogram of gradients based real-time object recognition
2021 (English)In: International Journal of Advanced Robotic Systems, ISSN 1729-8806, E-ISSN 1729-8814, Vol. 18, no 1, article id 1729881420978363Article in journal (Refereed) Published
Abstract [en]

3D object recognition has been a cutting-edge research topic since the popularization of depth cameras. These cameras enhance the perception of the environment and so are particularly suitable for autonomous robot navigation applications. Advanced deep learning approaches for 3D object recognition are based on complex algorithms and demand powerful hardware resources. However, autonomous robots and powered wheelchairs have limited resources, which affects the implementation of these algorithms for real-time performance. We propose to use instead a 3D voxel-based extension of the 2D histogram of oriented gradients (3DVHOG) as a handcrafted object descriptor for 3D object recognition in combination with a pose normalization method for rotational invariance and a supervised object classifier. The experimental goal is to reduce the overall complexity and the system hardware requirements, and thus enable a feasible real-time hardware implementation. This article compares the 3DVHOG object recognition rates with those of other 3D recognition approaches, using the ModelNet10 object data set as a reference. We analyze the recognition accuracy for 3DVHOG using a variety of voxel grid selections, different numbers of neurons (N-h ) in the single hidden layer feedforward neural network, and feature dimensionality reduction using principal component analysis. The experimental results show that the 3DVHOG descriptor achieves a recognition accuracy of 84.91% with a total processing time of 21.4 ms. Despite the lower recognition accuracy, this is close to the current state-of-the-art approaches for deep learning while enabling real-time performance.

National Category
Computer graphics and computer vision
Identifiers
urn:nbn:se:miun:diva-41626 (URN)10.1177/1729881420978363 (DOI)000619537100001 ()2-s2.0-85099946553 (Scopus ID)
Available from: 2021-03-15 Created: 2021-03-15 Last updated: 2025-02-07
4. Realworld 3d object recognition using a 3d extension of the hog descriptor and a depth camera
Open this publication in new window or tab >>Realworld 3d object recognition using a 3d extension of the hog descriptor and a depth camera
2021 (English)In: Sensors, E-ISSN 1424-8220, Vol. 21, no 3, article id 910Article in journal (Refereed) Published
Abstract [en]

3D object recognition is an generic task in robotics and autonomous vehicles. In this paper, we propose a 3D object recognition approach using a 3D extension of the histogram-of-gradients object descriptor with data captured with a depth camera. The presented method makes use of synthetic objects for training the object classifier, and classify real objects captured by the depth camera. The preprocessing methods include operations to achieve rotational invariance as well as to maximize the recognition accuracy while reducing the feature dimensionality at the same time. By studying different preprocessing options, we show challenges that need to be addressed when moving from synthetic to real data. The recognition performance was evaluated with a real dataset captured by a depth camera and the results show a maximum recognition accuracy of 81.5%. 

Keywords
3D object recognition, 3DHOG, Depth camera, Feature descriptor, Histogram-of-gradients, Intel RealSense, ModelNet10, ModelNet40, PCA
National Category
Computer graphics and computer vision
Identifiers
urn:nbn:se:miun:diva-41119 (URN)10.3390/s21030910 (DOI)000615514300001 ()2-s2.0-85099956966 (Scopus ID)
Available from: 2021-02-10 Created: 2021-02-10 Last updated: 2025-02-07
5. Evaluation of 2D-/3D-Feet-Detection Methods for Semi-Autonomous Powered Wheelchair Navigation
Open this publication in new window or tab >>Evaluation of 2D-/3D-Feet-Detection Methods for Semi-Autonomous Powered Wheelchair Navigation
2021 (English)In: Journal of Imaging, ISSN 2313-433X, Vol. 7, no 12, article id 255Article in journal (Refereed) Published
Abstract [en]

Powered wheelchairs have enhanced the mobility and quality of life of people with special needs. The next step in the development of powered wheelchairs is to incorporate sensors and electronic systems for new control applications and capabilities to improve their usability and the safety of their operation, such as obstacle avoidance or autonomous driving. However, autonomous powered wheelchairs require safe navigation in different environments and scenarios, making their development complex. In our research, we propose, instead, to develop contactless control for powered wheelchairs where the position of the caregiver is used as a control reference. Hence, we used a depth camera to recognize the caregiver and measure at the same time their relative distance from the powered wheelchair. In this paper, we compared two different approaches for real-time object recognition using a 3DHOG hand-crafted object descriptor based on a 3D extension of the histogram of oriented gradients (HOG) and a convolutional neural network based on YOLOv4-Tiny. To evaluate both approaches, we constructed Miun-Feet—a custom dataset of images of labeled caregiver’s feet in different scenarios, with backgrounds, objects, and lighting conditions. The experimental results showed that the YOLOv4-Tiny approach outperformed 3DHOG in all the analyzed cases. In addition, the results showed that the recognition accuracy was not improved using the depth channel, enabling the use of a monocular RGB camera only instead of a depth camera and reducing the computational cost and heat dissipation limitations. Hence, the paper proposes an additional method to compute the caregiver’s distance and angle from the Powered Wheelchair (PW) using only the RGB data. This work shows that it is feasible to use the location of the caregiver’s feet as a control signal for the control of a powered wheelchair and that it is possible to use a monocular RGB camera to compute their relative positions.

Keywords
3D object recognition, YOLO, YOLO-Tiny, 3DHOG, histogram of oriented gradients, ModelNet40, feature descriptor, Intel RealSense, depth camera, wheelchair
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:miun:diva-43838 (URN)10.3390/jimaging7120255 (DOI)000737580000001 ()2-s2.0-85121390982 (Scopus ID)
Available from: 2021-11-24 Created: 2021-11-24 Last updated: 2022-02-16Bibliographically approved

Open Access in DiVA

fulltext(11980 kB)883 downloads
File information
File name FULLTEXT02.pdfFile size 11980 kBChecksum SHA-512
35a57e83e2dd4898db9bd53c6cfda292109d45368dfe981ac72b4b9b7363dca1a779be9a88bcd0b3401f7289bf40b3a8e9da7b35ed34df6f08bbbaf43c241478
Type fulltextMimetype application/pdf
Errata(1074 kB)104 downloads
File information
File name FULLTEXT03.pdfFile size 1074 kBChecksum SHA-512
eb3188c079d5ee390837fe7f2c4259b4d557fb5e8ed9a4fe3e8953b88da15de8c695ca64cc56c22c58c47051ef36c7914007694a81d2e6f00dd6f89277afd6aa
Type fulltextMimetype application/pdf

Authority records

Vilar, Cristian

Search in DiVA

By author/editor
Vilar, Cristian
By organisation
Department of Electronics Design
Computer graphics and computer vision

Search outside of DiVA

GoogleGoogle Scholar
Total: 987 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 816 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf