Mid Sweden University

Future sixth-generation (6G) networks require efficient resource management to support a variety of services. This paper addresses the issue of maximizing user rates in a beyond directional reconfigurable intelligent surface (BD-RIS)-assisted network with non-orthogonal multiple access (NOMA) and integrated sensing and communication (ISAC) users. However, exploiting the gains offered by these frameworks necessitates joint tuning of BD-RIS phases and NOMA power, which is an inherently non-convex problem. We model this coupling as a continuous-action Markov decision process and solve it using twin-delayed deep deterministic policy gradient (TD3) reinforcement learning. The learned policy adaptively selects power-allocation factors and BD-RIS phase shifts, thereby boosting both communication and sensing rates under quality-of-service constraints. Simulation results confirm that the proposed deep reinforcement learning (DRL) scheme significantly outperforms conventional heuristics, demonstrating its potential for real-time resource optimization in 6G networks. 

The rapid growth in mobile communication technologies has turned mobile edge computing (MEC) into a paradigm-shifting technology that extends cloud-like capabilities and storage resources to the edge of the network. This allows computation-intensive and latency-sensitive applications to be performed at close proximity to the end-users, thereby overcoming the bottleneck issues of resource-constrained devices. However, ensuring efficient operations in MEC-empowered systems requires intelligent task execution and resource allocation across MEC servers. To this end, MEC-empowered non-terrestrial wireless networks (MeNT-WiN) systems are one of the applications in which deep reinforcement learning (DRL) is seen as a powerful method to enhance the MEC abilities in edge servers and network entities. This paper presents a thorough overview of the applications of DRL in MeNT-WiNs. In particular, it underlines the main contribution of DRL in enhancing the performance of MeNT-WiNs, including unmanned aerial vehicles (UAV) and satellite communications networks. This paper investigates how DRL can meet the unique requirements of MeNT-WiNs by enhancing system efficiency, scalability, and decision-making processes across MEC architectures. First, the article reviews the fundamentals of DRL, it later goes on to discuss its integration with MeNT-WiNs and demonstrates its relevance for the optimization of satellite communications and management of UAV swarms, as well as enhancing connectivity in remote areas. The survey also identifies key challenges for DRL-driven MeNT-WiN systems, such as computational complexity and real-time adaptability, while being scalable. Finally, it discusses future research possibilities, emphasizing the importance of new solutions that integrate DRL with MEC in order to fully exploit the potential of MeNT-WiNs. 

The rapid evolution of mobile communication technologies has propelled mobile edge computing (MEC) as a pivotal paradigm bringing cloud capabilities and storage resources to the network edges, thereby, enabling the execution of computation-intensive, latency-sensitive applications at the network edge, and addressing limited device resources. However, efficient operation in MEC-assisted systems necessitates proper task executions onto MEC servers. Meanwhile, deep reinforcement learning (DRL) can substantially enhance the performance of MEC-enhanced networks by incorporating decision-making capabilities into individual network entities and edge servers. This paper presents a comprehensive survey of the applications of DRL in MEC ecosystems. More specifically, it explores the applications of DRL in MEC-enabled terrestrial wireless networks (TWNs) including Internet-of-things (IoT) and vehicular networks (VNs). The article provides a comprehensive roadmap for researchers navigating the complexities of intelligent systems and MEC-enabled networks, offering a meticulous understanding of the continuously evolving landscape in this domain. Beginning with foundational DRL principles, the survey scrutinizes the integration of DRL in MEC-enabled TWNs, showcasing its efficacy in optimizing modern TWNs. In the context of MEC-empowered IoT, the paper highlights the role of DRL in enhancing resource allocation, data management, and scalability enhancements. Extending beyond, the paper discusses MEC-enabled VNs, where DRL transforms its role in traffic signal control, and route optimization, ultimately improving efficiency and safety. Additionally, we highlight significant challenges and outline future research directions in applying DRL in terrestrial networks (TWNs) empowered by the MEC paradigm. 

This paper investigates adaptive user pairing (UP) under different non-orthogonal medium access choices in 5G-and-beyond cellular IoT networks to balance the uplink performance of mission-critical (MC) and enhanced mobile broadband (eMBB) services. Our objective is to enhance eMBB rates while ensuring quality of service (QoS) for MC users, assessed through average age of information (AoI) and peak AoI (PAoI) violation probabilities. By deriving a signal-to-noise ratio (SNR) gap threshold between a pair of eMBB and MC users, we identify optimal access scheme—puncturing, non-orthogonal mul tiple access (NOMA), or rate-splitting multiple access (RSMA)—with respect to activation probability (pm) and cellular network radius. By using this derived threshold, we design an adaptive pairing algorithm that achieves near-optimal QoS for MC users and maximizes eMBB data rates. To realize different spatial associations among users in the cell, the proposed pairing strategy for eMBB and MC services is evaluated for three user distributions around the base station: concave (eMBB users concentrated near the BS), uniform (evenly spread eMBB and MC users), and convex (MC users concentrated near the BS). The extensive numerical analysis of the proposed solution demonstrates significant performance gains over random and traditional NOMA-based pairings, especially under concave scenarios. In concave distributions, our strategy reduces MC users’ QoS outage by 85% at pm=0.1, achieving zero outage for pm≥0.3. Uniform and convex distributions confirm method robustness, maintaining low or zero outage probabilities across all pm values. We also analyzed the impact of network radius and MC user activation probabilities on access scheme selection. Results show that RSMA generally outperforms other multiple access schemes in terms of eMBB rate, but NOMA exhibits superior performance compared to RSMA and puncturing in larger networks.

This paper investigates a co-design framework for wireless networked control systems (WNCSs) that integrates multi-hop IEEE 802.15.4-based links under Wi-Fi interference, addressing the challenges of signal-to-interference-plus-noise ratio (SINR) degradation in adverse industrial environments. Multihop configurations are essential for extending the operational range and improving SINR in harsh propagation conditions, but they introduce trade-offs in control stability, latency, and computational complexity. We investigate the impact of multi-hop communication on system performance, comparing Bernoulli and Markovian control strategies. Our results demonstrate that multihop links effectively extend the operational range and mitigate SINR degradation, but at the cost of increased latency and computational cost. We analyze the spectral radius of the system stability verification matrix and control costs for Bernoulli and Markovian control strategies, illustrating that network latency and hop counts can be balanced while maintaining the stability of the multi-hop WNCS. Markovian strategy, although more computationally intensive, outperforms Bernoulli strategy under high interference, offering a robust solution for industrial WNCSs. The proposed framework provides a practical approach for deploying reliable WNCSs in interference-prone environments. 

Through the lens of average and peak age-of-information (AoI), this paper takes a fresh look into the uplink medium access solutions for mission-critical (MC) communication coexisting with enhanced mobile broadband (eMBB) service. Considering the stochastic packet arrivals from an MC user, we study three access schemes: orthogonal multiple access (OMA) with eMBB preemption (puncturing), non-orthogonal multiple access (NOMA), and rate-splitting multiple access (RSMA), the latter two both with concurrent eMBB transmissions. Puncturing is found to reduce both average AoI and peak AoI (PAoI) violation probability but at the expense of decreased eMBB user rates and increased signaling complexity. Conversely, NOMA and RSMA offer higher eMBB rates but may lead to MC packet loss and AoI degradation. The paper systematically investigates the conditions under which NOMA or RSMA can closely match the average AoI and PAoI violation performance of puncturing while maintaining data rate gains. Closed-form expressions for average AoI and PAoI violation probability are derived, and conditions on the eMBB and MC channel gain difference with respect to the base station are analyzed. Additionally, optimal power and rate splitting factors in RSMA are determined through an exhaustive search to minimize MC outage probability. Notably, our results indicate that with a small loss in the average AoI and PAoI violation probability the eMBB rate in NOMA and RSMA can be approximately five times higher than that achieved through puncturing. 

In the emerging field of ambient Internet-of-things (A-IoT), a new frontier driven by the need for sustainable, low-power, and energy-efficient communication solutions is paramount. This evolution is embodied in A-IoT, where innovative approaches are essential to meet the growing demands of interconnected devices. One such transformative technique is the integration of quality-of-service (QoS)-aware non-orthogonal multiple access (NOMA) schemes with deep reinforcement learning (DRL), reconfigurable intelligent surfaces (RIS), and radio-frequency energy harvesting (RF-EH) diversity. In this article, we propose how these combined technologies can maximize throughput for low-power IoT devices. Beginning with an overview of the foundational concepts of NOMA, DRL, RIS, and RF-EH, we demonstrate how their synergy can dynamically optimize resource allocation while maximizing throughput. Through illustrative use cases and extensive simulations, we reveal that our proposed scheme significantly enhances both data rate and EH compared to traditional methods. The article also discusses the potential applications and practical implications of this approach, paving the way for the future of near-zero energy IoT networks. Finally, we identify key research challenges and opportunities for realizing scalable and efficient A-IoT systems that leverage these advanced technologies. 

The localization of multiple signal sources using sensor arrays has been a long-standing research challenge. While numerous solutions have been developed, signal space methods like MUSIC and ESPRIT have gained widespread popularity. As sensor arrays grow in size, sources are frequently located in the near-field region. The standard MUSIC algorithm can be adapted to locate these sources by performing a 3D search over both the distance and the angles of arrival (AoA), including azimuth and elevation, though this comes with significant computational complexity. To address this, a modified version of MUSIC has been developed to decouple the AoA and distance, enabling sequential estimation of these parameters and reducing computational demands. However, this approach suffers from reduced accuracy. To maintain the accuracy of MUSIC while minimizing complexity, this paper proposes a novel method that exploits angular variation across the array aperture, eliminating the need for a grid search over distance. The proposed method divides the large aperture into smaller sections, with each focusing on estimating the angles of arrival. These angles are then triangulated to localize the sources in the near-field of the large aperture. Numerical simulations show that this approach not only surpasses the Modified MUSIC algorithm in terms of mean absolute error but also achieves accuracy comparable to standard MUSIC, all while greatly reducing computational complexity- 370 times in our simulation scenario. 

With the increasing number of Internet-of-things (IoT) devices, the need for energy-efficient and spectrum-efficient networks that can support resource-constrained devices within existing wireless infrastructures becomes critical. This paper investigates the application of deep reinforcement learning (DRL) algorithms to optimize the energy efficiency (EE) of a secondary device (SD) equipped with radio frequency energy harvesting (RF-EH) antennas. The system models a wireless powered communication network (WPCN) where the SD employs a cognitive-radio non-orthogonal multiple access (CR-NOMA) scheme to transmit data during uplink communications of neighboring primary devices (PDs). Among the DRL approaches evaluated, proximal policy optimization (PPO) emerged as the most effective, achieving the highest EE values and demonstrating its suitability for this problem. Additionally, our results show that equal gain combining (EGC) consistently achieves superior EE compared to other diversity-combining techniques, making it a favorable choice for self-sustaining IoT networks. These findings provide valuable insights into the role of diversity-combining techniques and DRL algorithms in enhancing SD performance in dynamic EH environments.

This paper presents a novel approach to maximizing the throughput of self-sustaining mobile IoT devices using a quality-of-service (QoS)-aware non-orthogonal multiple access (NOMA) technique. The proposed method enables transmissions within the timeslots of licensed users in IoT networks through a deep reinforcement learning (DRL)-driven strategy. By integrating hybrid energy harvesting (EH) from radio frequency (RF) and solar sources, the proposed framework is designed to optimize the energy usage and data transmission rates of a mobile sensing node (MSN) operating in a dynamic wireless environment. Our model incorporates non-linear RF and solar EH characteristics and accounts for mobility-induced variations in channel conditions. The throughput maximization problem is decomposed into a two-layer optimization framework, where the first layer utilizes convex optimization for power and time-sharing coefficients, while the second layer employs DRL to adapt to one-dimensional state-action spaces. Our results show that the Prioritized Experience Replay (PER)-DDPG algorithm achieves the best performance among the evaluated DRL approaches by enabling hybrid EH to achieve 7.83% higher data rates compared to RF-only scenarios. The results underscore the effectiveness of the DRL-based approach in enabling continuous operation and enhanced data rates for mobile IoT applications in QoS-aware NOMA IoT networks.