Mid Sweden University

Light field (LF) imaging captures spatial and angular information, offering a 4D scene representation enabling enhanced visual understanding. However, high dimensionality and redundancy across spatial and angular domains present major challenges for compression, particularly where storage, transmission bandwidth, or processing latency are constrained. We present a novel Variational Autoencoder (VAE)-based framework that explicitly disentangles spatial and angular features using two parallel latent branches. Each branch is coupled with an independent hyperprior model, allowing more precise distribution estimation for entropy coding and finer rate-distortion control. This dual-hyperprior structure enables the network to adaptively compress spatial and angular information based on their unique statistical characteristics, improving coding efficiency. To further enhance latent feature specialization and promote disentanglement, we introduce a mutual information-based regularization term that minimizes redundancy between the two branches while preserving feature diversity. Unlike prior methods relying on covariance-based penalties prone to collapse, our information-theoretic regularizer provides more stable and interpretable latent separation. Experimental results on publicly available LF datasets demonstrate our method achieves strong compression performance, yielding an average BD-PSNR gain of 2.91 dB over HEVC and high compression ratios (e.g., 200:1). Additionally, our design enables fast inference, with a total end-to-end time over 19x faster than the JPEG Pleno standard, making it well-suited for real-time and bandwidth-sensitive applications. By jointly leveraging disentangled representation learning, dual-hyperprior modeling, and information-theoretic regularization, our approach offers a scalable, effective solution for practical light field image compression.

Steered Mixtures-of-Experts (SMoE) is an existing regression framework that has previously been applied for modeling and compression of 2D images and higher-dimensional imagery, including compression of light fields and light-field video. SMoE models are sparse, edge-aware representations that allow rendering of imagery with few Gaussians with excellent quality. In this paper a novel, edge-aware "3D SMoE Splatting" (3DSMoES) framework for 3D rendering is introduced, adopted to fit into the existing "3D Gaussian Splatting" (3DGS) CUDA optimization pipeline. Here, SMoE regression serves as a "plug-and-play" solution that replaces the established 3DGS regression as a novel workhorse. 3DSMoES achieves significant visual quality gains with drastically fewer Gaussian kernels compared to 3DGS. We observe up to approximately 4dB improvement in PSNR on individual scenes with kernel reductions between 20 to 50 percent. The sparse models are significantly faster to train and allow up to 30-50 percent improved rendering speeds.

Immersive imaging applications gained a lot of traction in the last decade with the advances in capture, processing, compression, transmission, and display technologies. Recent works on spherical light fields and new view synthesis bring new challenges with respect to fast rendering of new views and a smooth quality of experience (QoE). The real-time rendering capabilities enable realistic immersive telepresence applications, which extend the traditional telepresence systems with improved visual realism and possible addition of depth cues. The recent efforts on the spherical light fields and new view synthesis approaches show that a combined light field and spherical data visualization will be beneficial for the scientific community. In this paper, we provide a WebGL- and WebXR-based visual QoE toolkit which can be used both on traditional displays and extended reality headsets, presenting various visual modalities. To validate the usefulness of the proposed toolkit, we conducted a pilot test on our publicly available spherical light field database. This toolkit can be used for easier and faster quality assessment and can support the scientific community for subjective visual QoE studies focusing on telecommunication, telepresence, and augmented telepresence applications.

Kernel image regression methods have demonstrated excellent efficiency in various image processing tasks, including image and light-field compression, Gaussian Splatting, denoising and super-resolution. The estimation of parameters for these methods commonly employs gradient descent iterative optimization, which poses a significant computational burden for many applications. In this paper, we introduce a novel adaptive segmentation-based initialization method targeted for optimizing Steered-Mixture-of Experts (SMoE) gating networks and RadialBasis-Function (RBF) networks with steering kernels. The novel initialization method allocates kernels into pre-calculated image segments. The optimal number of kernels, kernel positions, and steering parameters are derived per segment in an iterative optimization and kernel sparsification procedure. The kernel information from local segments is then transferred into a global initialization, ready for use in iterative optimization of SMoE, RBF, and related kernel image regression methods. Results demonstrate significant improvements in both objective and subjective quality compared to regular grid, K-Means, deeplearning-based, and previous segmentation-based initialization methods. The proposed initialization method reduces kernel usage by 70% compared to other initialization methods while maintaining the same reconstruction quality. Furthermore, by generating initial parameters closer to optimized results, convergence time is reduced, achieving overall runtime savings of up to 50% compared to prior methods. Additionally, the method supports parallel computation, with initialization time halved when using four GPUs compared to one

Immersive imaging technologies offer a transformative way to change how we experience interacting with remote environments, i.e., telepresence. By leveraging advancements in light field imaging, omnidirectional cameras, and head-mounted displays, these systems enable realistic, real-time visual experiences that can revolutionize how we interact with the remote scene in fields such as healthcare, education, remote collaboration, and entertainment. However, the field faces significant technical and experiential challenges, including efficient data capture and compression, real-time rendering, and quality of experience (QoE) assessment. Expanding on the findings of the authors’ recent publication and situating them within a broader theoretical framework, this article provides an integrated overview of immersive telepresence technologies, focusing on their technological foundations, applications, and the challenges that must be addressed to advance this field.

Differentiable Architecture Search (DARTS) has shown promising results in automating the design of deep learning models. However, its search process is computationally expensive because it evaluates all candidate operations simultaneously, often leading to an over-parameterized and inefficient search network. To reduce the computational cost, DARTS employs a smaller search network than the final evaluation network, which introduces an architecture optimization gap that limits real-world performance. To overcome this limitation, we introduce CR-DARTS, a multi-stage search framework designed to bridge the architecture optimization gap through an adaptive channel redistribution strategy. CR-DARTS reduces the computational complexity of the search network by compressing the shared input features among candidate operations and restoring the network dimensions via channel-wise feature concatenation. In addition, it progressively eliminates underperforming operations and redistributes the number of channels for more relevant feature extraction, thereby narrowing the gap between the search and evaluation networks. We validated CR-DARTS on two diverse computer vision tasks to assess its generalizability. Experimental results show that the proposed search framework reduces the memory requirement of the DARTS algorithm by up to 4.3×, while addressing the architecture optimization gap. Moreover, in the evaluation phase, the discovered architecture achieves up to 25.3% reductions in computational complexity and 50.6% faster inference time compared to state-of-the-art methods, while maintaining comparable accuracy. It also produces a competitive fire segmentation network that outperforms the state-of-the-art methods while maintaining similar computational efficiency. These results demonstrate that CR-DARTS is a practical solution for neural architecture search. Source code will be made publicly available at https://github.com/Realistic3D-MIUN/CR-DARTS.

Light field (LF) imaging captures both spatial and angular information, which is essential for applications such as depth estimation, view synthesis, and post-capture refocusing. However, the efficient processing of this data, particularly in terms of compression and encoding/decoding time, presents challenges. We propose a Variational Autoencoder (VAE)-based framework to disentangle the spatial and angular features of light field images, focusing on fast and efficient compression. Our method uses two separate sub-encoders-one for spatial and one for angular features-to allow for independent processing in the latent space, which facilitates more streamlined compression workflows. Evaluations on standard light field datasets demonstrate that our approach reduces encoding and decoding time significantly, with a slight trade-off in Rate-Distortion (RD) performance, making it suitable for real-time applications.

Convolutional neural networks are widely used forlight field disparity estimation. However, many state-of-the-artdeep learning models are computationally expensive due to their reliance on standard convolutions with varying kernel sizes. In this paper, we analyze the effect of various advanced convolution operations with different kernel sizes for feature extraction in a state-of-the-art light field disparity estimation network. Based on this investigation, we propose an optimized mixed convolution layer to extract relevant features using multiple kernel sizes in parallel, while maintaining significantly lower computational cost. Experimental results demonstrate that our approach reduces model complexity by up to 4.2× while also improving disparity estimation accuracy. These findings make the proposed convolutional operation more practical for lightfield applications, where efficient spatial and angular feature extraction is essential for improved model performance.

Essentially all academic research of today relies on analysis of data from a wide range of sources. Several underpinning, and rapidly developing, technologies are supporting the analysis of this data. Visualization serves as an interface to this ecosystem of tools and methods and integrates them into environments supporting scientific workflows, effectively sharing cognitive load between computers and humans. There is, however, a gap between the state-of-the-art in visual data analysis and current wide-spread academic practice. Support for the introduction of new, improved and tailored, visual data analysis environments thus has the potential to address challenges involving large and complex data, creating competitive advantages for researchers. To fill the gap and capitalize on this opportunity, the InfraVis initiative has been created in Sweden with the mission to operate an infrastructure consisting of visualization experts, software solutions, and access to high-end visualization laboratories. Users of InfraVis are offered assistance through a national helpdesk with rapid response times as well as more in-depth projects addressing specific data and software challenges. InfraVis provides software solutions based on development within connected research groups, curation of international software and best practice, and user training in the form of courses, seminars and on-line documentation. To build an infrastructure with national coverage, we have pooled together nine visualization environments in Sweden interconnected in a nodal structure. The nodes are hosted in proximity to research environments in visualization, which enables direct access to the research front as well as to state-of-art facilities. The governance structure of InfraVis is based on the leading researchers in visualization in Sweden as well as an international advisory board.

Inverse problems involve reconstructing clean images from degraded observations. Maximum a Posteriori (MAP) estimation reconstructs the most probable source image from noisy measurements. When combined with Plug-and-Play (PnP) priors defined by an image denoising algorithm, MAP estimation yields high-quality reconstructions. In contrast, Diffusion Models (DMs) address inverse problems by sampling from the posterior distribution using score functions trained on images perturbed by Gaussian noise. Prior work reformulated diffusion sampling as Deep Equilibrium (DEQ) models but did not fine-tune DMs for inverse problems. This work introduces MaximUm a PostEriori Training (MUPET), a framework that leverages PnP gradient descent to enable DEQ fine-tuning of DMs on inverse problems. By refining a generative prior at the fixed-point of MAP estimation, MUPET enhances image restoration via posterior sampling while maintaining quality when sampling from the prior.