Mid Sweden University

miun.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Investigation of human interaction with an augmented remote operating system for scaling in mining applications
Mid Sweden University, Faculty of Science, Technology and Media, Department of Computer and Electrical Engineering (2023-). RISE Research Institutes of Sweden AB, Stockholm/Gothenburg, Sweden. (Realistic3D)ORCID iD: 0000-0002-5913-3145
Mid Sweden University, Faculty of Science, Technology and Media, Department of Computer and Electrical Engineering (2023-). RISE Research Institutes of Sweden AB, Stockholm/Gothenburg, Sweden.ORCID iD: 0000-0001-5060-9402
RISE Research Institutes of Sweden AB, Stockholm/Gothenburg, Sweden.ORCID iD: 0000-0002-6307-1960
Mid Sweden University, Faculty of Science, Technology and Media, Department of Computer and Electrical Engineering (2023-).
2024 (English)In: Quality and User Experience, ISSN 2366-0139, E-ISSN 2366-0147, Vol. 9, no 4Article in journal (Refereed) Published
Abstract [en]

Thanks to the advent of telepresence applications, we can remotely take control and operate industrial machinery. Teleoperation removes operators from hazardous workplaces such asmining and plays an essential role in the safety of workers. In addition, augmented telepresence can introduce information that helps the user understand the remote scene. However, remote operation has challenges when the received information is more limited than what could be perceived on-site–e.g., judging depth. This study investigates how well operators interact with an Augmented Remote Operation Scaling System (AROSS) in a mining context when different computer-generated visual interfaces are provided. The system can achieve five augmented views: Disocclusion Augmentation using selective content removal; Novel Perspective view generation; Lidar view; Right field of view; and Left field of view. We performed two experiments in a mine-like laboratory. The first experiment was a feasibility test to obtain an understanding of what users need to accurately perceive depth. The second experiment was designed to evaluate the user’s experience with the different versions of AROSS. To analyze human interaction with the designed prototype, we employed a mixed research methodology that used interviews, observations, and questionnaires. This mixed methodology consisted of quality of experience methods to discover the users’requirements from a technological standpoint and user experience methods (i.e., user-centricapproaches). We investigated 10 and 11 users’ interactions in the two subjective experiments. The first experiment focused on the effects of in-view augmentations and interface distributions on perceiving wall patterns. The second focused on the effects of augmentations on the depth and understanding the 3D environment. Using these data, we analyzed both thequality of experience and user experience via evaluation criteria consisting of interface helpfulness, task performance, potential improvement, and user satisfaction. The feasibility test results were mainly used to structure the formative investigation. The overall conclusion from the formative testing shows that the remote operators preferred using natural views (Original) as this approach made it easier to understand the environment. Although the augmented computer-generated views do not look natural, they support 3D cues. In addition, the combination of Novel Perspective and Lidar interfaces as additional views in depth perception tasks seemed helpful. There was difficulty performing tasks when the robot arm was obscured during the Disocclusion Augmentation view and low video quality during the Novel Perspective view. However, participants found the Novel Perspective view useful for geometry and depth estimation.

Place, publisher, year, edition, pages
Springer, 2024. Vol. 9, no 4
Keywords [en]
User Experience (UX), Quality of Experience (QoE), Augmented Telepresence (AT), Intelligent Mining, Industrial remote controlling, UX and QoE, Mixed methodology
National Category
Engineering and Technology Computer and Information Sciences
Identifiers
URN: urn:nbn:se:miun:diva-50092DOI: 10.1007/s41233-024-00068-9OAI: oai:DiVA.org:miun-50092DiVA, id: diva2:1818772
Funder
Swedish Foundation for Strategic ResearchAvailable from: 2023-12-12 Created: 2023-12-12 Last updated: 2025-09-25Bibliographically approved
In thesis
1. Remote Controlled 3D Positioning in Augmented Telepresence: User and Quality of Experience Aspects
Open this publication in new window or tab >>Remote Controlled 3D Positioning in Augmented Telepresence: User and Quality of Experience Aspects
2025 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Industrial companies increasingly adopt remote operation technologies (teleoperation) to enhance safety and operational reach, bringing new challenges in understanding how users interact with these systems. In particular, when operators rely on flat, video-based displays without natural depth cues, challenges arise in user performance, spatial depth perception, and the overall quality of user experience when interacting with teleoperation systems. There is a growing need for evaluation approaches that extend beyond technical performance to assess the quality of users’ experience. This dissertation integrates findings from multiple studies conducted throughout the doctoral project. It explores how a mixed-method,user-centred evaluation strategy, combining system performance measurement with analysis of the quality of user experience, improves understanding of interaction quality in remote operation systems. Empirical investigations in mining and construction machinery domains examined how visual configurations such as image augmentation, scene presentation, and video degradation influence performance and perceived experience. Grounded in a pragmatic design philosophy, the research applied a combination of quantitative and qualitative methods. Laboratory experiments were conducted using two custom-built remote operation platforms: one simulating robotic arm control in mining, and another emulating construction vehicle teleoperation. Data collection included system-logged performance measures, users’ rating scales, and open-ended reflections to capture personal experiences. This multi-perspective approach enabled triangulation of results and a more complete understanding of functional and experiential aspects of remote interaction. Findings reveal that visual configurations significantly influence performance outcomes, user perceptions, and interaction strategies. Standard views enabled higher precision, while augmented perspectives improved spatial understanding and confidence. Degraded video quality and latency, reduced user comfort, control experience, and task clarity. Combining measurable performance data with reflective feedback offered deeper insights into factors shaping successful and satisfying interaction. This integrated approach contributes new knowledge to the design and evaluation of remote operation systems, ensuring attention to both system efficiency and the human experience. The methodology also provides practical guidance for evaluating complex human-technology interactions in safety-critical contexts. Beyond the mining and construction use cases, the research introduced a third platform aimed at airport safety monitoring that serves as a ready-to-use testbed for future investigations into situational awareness and human-system coordination in remote environments.

Place, publisher, year, edition, pages
Sundsvall: Mid Sweden University, 2025. p. 63
Series
Mid Sweden University doctoral thesis, ISSN 1652-893X ; 433
National Category
Electrical Engineering, Electronic Engineering, Information Engineering Computer and Information Sciences
Identifiers
urn:nbn:se:miun:diva-54678 (URN)978-91-90017-29-6 (ISBN)
Public defence
2025-09-02, L111, Holmgatan 10, Sundsvall, 09:15 (English)
Opponent
Supervisors
Funder
Vinnova, dnr. 2023-00755Vinnova, dnr. 2022- 02670Vinnova, dnr. 2021-02107Swedish Foundation for Strategic Research, FID18-0030
Note

Vid tidpunkten för disputationen var följande delarbete opublicerat: delarbete 4 manuskript.

At the time of the doctoral defence the following paper was unpublished: paper 4 in manuscript.

Available from: 2025-06-18 Created: 2025-06-18 Last updated: 2025-12-04Bibliographically approved

Open Access in DiVA

fulltext(2846 kB)468 downloads
File information
File name FULLTEXT01.pdfFile size 2846 kBChecksum SHA-512
797506af97d581dc1f40e4a0b22fb724d63931fb5c2809fc8d26a4ef39970933edb6ea1c6fc7523dc550db75dcbfea1420233857d832044a9c546724277c2460
Type fulltextMimetype application/pdf

Other links

Publisher's full text

Authority records

Rafiei, ShirinBrunnström, KjellSjöström, Mårten

Search in DiVA

By author/editor
Rafiei, ShirinBrunnström, KjellAndersson, JonasSjöström, Mårten
By organisation
Department of Computer and Electrical Engineering (2023-)
In the same journal
Quality and User Experience
Engineering and TechnologyComputer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 469 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 545 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf