Mid Sweden University

Remote operation and Augmented Telepresence are fields of interest for novel industrial applications in e.g., construction and mining. In this study, we report on an ongoing investigation of the Quality of Experience aspects of an Augmented Telepresence system for remote operation. The system can achieve view augmentation with selective content removal and Novel Perspective view generation. Two formal subjective studies have been performed with test participants scoring their experience while using the system with different levels of view augmentation. The participants also gave free-form feedback on the system and their experiences. The first experiment focused on the effects of in-view augmentations and interface distributions on wall patterns perception. The second one focused on the effects of augmentations on depth and 3D environment understanding. The participants’ feedback from experiment 1 showed that the majority of participants preferred to use the original camera views and the Disocclusion Augmentation view instead of Novel Perspective views. Moreover, the Disocclusion Augmentation, that was shown in combination with other views seemed beneficial. When the views were isolated in experiment 2, the impact of the Disocclusion Augmentation view was found to be lower than the Novel Perspective views.

Tele-operated driving enables industrial operators to control heavy machinery remotely. By doing so, they could work in improved and safe workplaces. However, some challenges need to be investigated while presenting visual information from on-site scenes for operators sitting at a distance in a remote site. This paper discusses the impact of video quality (spatial resolution), field of view, and latency on users' depth perception, experience, and performance in a lab-based tele-operated application. We performed user experience evaluation experiments to study these impacts. Overall, the user experience and comfort decrease while the users' performance error increases with an increase in the glass-to-glass latency. The user comfort reduces, and the user performance error increases with reduced video quality (spatial resolution). 

Thanks to the advent of telepresence applications, we can remotely take control and operate industrial machinery. Teleoperation removes operators from hazardous workplaces such asmining and plays an essential role in the safety of workers. In addition, augmented telepresence can introduce information that helps the user understand the remote scene. However, remote operation has challenges when the received information is more limited than what could be perceived on-site–e.g., judging depth. This study investigates how well operators interact with an Augmented Remote Operation Scaling System (AROSS) in a mining context when different computer-generated visual interfaces are provided. The system can achieve five augmented views: Disocclusion Augmentation using selective content removal; Novel Perspective view generation; Lidar view; Right field of view; and Left field of view. We performed two experiments in a mine-like laboratory. The first experiment was a feasibility test to obtain an understanding of what users need to accurately perceive depth. The second experiment was designed to evaluate the user’s experience with the different versions of AROSS. To analyze human interaction with the designed prototype, we employed a mixed research methodology that used interviews, observations, and questionnaires. This mixed methodology consisted of quality of experience methods to discover the users’requirements from a technological standpoint and user experience methods (i.e., user-centricapproaches). We investigated 10 and 11 users’ interactions in the two subjective experiments. The first experiment focused on the effects of in-view augmentations and interface distributions on perceiving wall patterns. The second focused on the effects of augmentations on the depth and understanding the 3D environment. Using these data, we analyzed both thequality of experience and user experience via evaluation criteria consisting of interface helpfulness, task performance, potential improvement, and user satisfaction. The feasibility test results were mainly used to structure the formative investigation. The overall conclusion from the formative testing shows that the remote operators preferred using natural views (Original) as this approach made it easier to understand the environment. Although the augmented computer-generated views do not look natural, they support 3D cues. In addition, the combination of Novel Perspective and Lidar interfaces as additional views in depth perception tasks seemed helpful. There was difficulty performing tasks when the robot arm was obscured during the Disocclusion Augmentation view and low video quality during the Novel Perspective view. However, participants found the Novel Perspective view useful for geometry and depth estimation.

This study explores how video quality, field of view, and latency influence users' performance, depth perception, and overall experience in remote-control systems for industrial teleoperated applications. In this controlled laboratory investi-gation, we conducted an experimental study using a test setup that closelyreplicated real-world conditions while ensuring experimental control and safety. Participants completed 18 trials under varying fields of view (Wide vs. Narrow), latency (High, Medium, Low), and video quality (High, Medium, Low)conditions to evaluate their navigation accuracy, depth perception and Qualityof Experience. Using a mixed-methods approach, considering Quality of Experience and User Experience insights, we integrated objective performance metrics, quantitative user evaluations, and qualitative feedback to understand how visual constraints impact control and decision-making. The results show that latency had the most impact on navigation accuracy, followed by field of view. The high latency level significantly influenced performance, caused larger deviations from the intended path, and reduced confidence in estimating depth. The effect was even more pronounced in narrow fields of view, where high latency further amplified navigation difficulties. In contrast, a wider field of view helped mitigate latency effects by improving situational awareness and reducing reliance on secondary visual cues. While video quality had minimal impact on objectivenavigation performance, qualitative feedback indicated that higher video quality improved confidence in depth perception and reduced visual strain, supporting users in making precise judgments. We also collected users' feedback that provided cues to further enhance the teleoperation systems