doi: 10.56294/dm2024333

 

ORIGINAL

 

Optimizing Energy Consumption in 5G HetNets: A Coordinated Approach for Multi-Level Picocell Sleep Mode with Q-Learning

 

Optimización del consumo de energía en redes de área extensa 5G: Un enfoque coordinado para el modo de reposo multinivel de picocélulas con Q-Learning

 

Macoumba Fall1  *, Mohammed Fattah1  *, Mohammed Mahfoudi2  *, Younes Balboul1  *, Said Mazer1  *, Moulhime El Bekkali1  *, Ahmed D. Kora3  *

 

1IASSE Laboratory, Sidi Mohamed Ben Abdellah University. Fez, Morocco.

2ISE Laboratory, Abdelmalek Essaadi University. Tetuan, Morocco

3EDMI, Cheikh Anta Diop University. Dakar, Senegal.

 

Cite as: Fall M, Fattah M, Mahfoudi M, Balboul Y, Mazer S, El Bekkali M, Kora AD. Optimizing Energy Consumption in 5G HetNets: A Coordinated Approach for Multi-Level Picocell Sleep Mode with Q-Learning. Data and Metadata. 2024; 3:333. https://doi.org/10.56294/dm2024333

 

Submitted: 01-01-2024                   Revised: 08-03-2024                   Accepted: 19-05-2024                 Published: 20-05-2024

 

Editor: Adrián Alejandro Vitón Castillo

 

ABSTRACT

 

Cell standby, particularly picocell sleep mode (SM), is a prominent strategy for reducing energy consumption in 5G networks. The emergence of multi-state sleep states necessitates new optimization approaches. This paper proposes a novel energy optimization strategy for 5G heterogeneous networks (HetNets) that leverages macrocell-picocell coordination and machine learning. The proposed strategy focuses on managing the four available picocell sleep states. The picocell manages the first three states using the Q-learning algorithm, an efficient reinforcement learning technique. The associated macrocell based on picocell energy efficiency controls the final, deeper sleep state. This hierarchical approach leverages localized and network-wide control strengths for optimal energy savings. By capitalizing on macrocell-picocell coordination and machine learning, this work presents a promising solution for achieving significant energy reduction in 5G HetNets while maintaining network performance.

 

Keywords: 5G; Hetnets; Energy Consumption; Energy Modeling; Sleep Mode; Machine Learning.

 

RESUMEN

 

El modo de espera de la célula, en particular el modo de reposo (SM) de la picocélula, es una estrategia destacada para reducir el consumo de energía en las redes 5G. La aparición de estados de reposo multiestado requiere nuevos enfoques de optimización. Este artículo propone una nueva estrategia de optimización energética para redes heterogéneas 5G (HetNets) que aprovecha la coordinación macrocelda-picocelda y el aprendizaje automático. La estrategia propuesta se centra en la gestión de los cuatro estados de reposo disponibles de la picocélula. La picocélula gestiona los tres primeros estados mediante el algoritmo Q-learning, una eficiente técnica de aprendizaje por refuerzo. La macrocélula asociada, basada en la eficiencia energética de la picocélula, controla el último estado de sueño, más profundo. Este enfoque jerárquico aprovecha las ventajas del control localizado y de la red para conseguir un ahorro energético óptimo. Aprovechando la coordinación macrocélula-picocélula y el aprendizaje automático, este trabajo presenta una solución prometedora para lograr una reducción significativa de la energía en redes 5G HetNets, manteniendo al mismo tiempo el rendimiento de la red.

 

Palabras clave: 5G; Hetnets; Consumo Energético; Modelado Energético; Modo Reposo; Aprendizaje Automático.

 

 

 

Introduction

The combination of 5G and different types of networks has helped meet the increasing need for mobile data and fast data speeds. However, this improvement makes networks more complicated and uses much energy. In the future, mobile networks will become even more diverse because more devices will be used, more data will be transferred, and high-frequency networks with limited reach will be deployed more often.

While HetNets can handle more data, they also make it harder to manage energy. Some methods, like making small cells sleep when they’re not needed (sleep mode), have shown promise in saving energy. But it’s tricky to ensure these methods don’t affect service quality. Moving from simple sleep modes to more advanced ones with different states of sleep needs a better way to determine how much energy each state uses. This is important for deciding when it’s worth putting cells to sleep and when it’s not. We also want to use machine learning to control sleep modes as best as possible.

Current methods for sleep mode have trouble managing energy well when there are different sleep states in HetNets. It's hard to save energy while keeping service quality high. Also, focusing only on small cells might not use energy as efficiently as possible in networks where different types of cells work together.

This study introduces a new way to save energy in 5G HetNets. It looks at how big and small cells can work together to decide when small cells should sleep. It also uses energy efficiency rules to pick the best sleep states. Machine learning techniques adjust sleep modes based on what's happening in the network.

This work contributes to the field of energy-efficient HetNet design by proposing a novel approach that leverages macrocell-picocell coordination and ML for optimal picocell sleep state control. This strategy has the potential to significantly reduce energy consumption while maintaining acceptable QoS levels in 5G networks.

 

Related work

SM is a technique used in mobile networks to reduce base station energy consumption significantly. By temporarily turning off non-essential functions during low-traffic periods, SM allows energy usage to adapt to real-time cell load, minimizing the amount of wasted power.(1) This approach is particularly relevant in today's context, driven by both environmental concerns and economic factors. Studies show that base stations account for a substantial portion (60-80 %) of a mobile network's energy consumption.(2)

However, a key challenge lies in the inherent energy draw of base stations, even during low-load periods. Estimates suggest that a base station can still consume 50-60 % of its maximum power under no-load or low-load conditions.(3) This highlights the need for a consumption model that dynamically adapts to cell load variations. Among various proposed solutions, SM stands out for its effectiveness. It offers a simple and readily implementable method for reducing energy consumption without requiring extensive modifications to the existing network architecture.(4) The concept of cell sleep mode was first introduced by Micallef et al.(1), who combined it with "cell size breathing" - a mechanism for dynamically adjusting cell coverage area. Advancements followed the(5) and(6), where researchers proposed putting specific resources on standby based on traffic load, ensuring minimal impact on quality of service (QoS). The emergence of heterogeneous networks (HetNets) opened new possibilities for SM. With macrocells providing full coverage, cellular monitoring could be applied at the picocell level, as explored in some studies. For instance, Frenger et al.(7) introduced Cell Discontinuous Transmission (DTX), where the radio transmitter is switched off when there's no data or reference signal to transmit.

Beyond the initial binary (on/off) sleep mode, research has focused on more granular control. The concept of Advanced Sleep Mode (ASM) with multiple sleep states emerged, facilitated by the more extended (160ms) discontinuous transmission duration introduced in 5G compared to 4G's 1ms limit.(7) Building on this concept, Debaillie et al.(8) proposed a system with four distinct sleep states, each characterized by unique activation, deactivation times, and minimum sleep durations. While multi-state sleep mode (MSSM) strategies, as proposed Saker et al.(9), offer significant energy savings at low traffic loads, they often come at the cost-of-service quality. Finding the optimal balance between energy consumption and QoS becomes crucial. As MSSM systems become more complex, choosing the ideal sleep level presents a growing challenge. Machine learning emerges as a promising solution for optimizing sleep state selection. Several studies have explored this approach. Pervaiz et al.(10) proposed an MDP-based (Markov Decision Process) control mechanism for heterogeneous networks, considering both cell load and user equipment (UE) location. Similarly, Salem et al.(11) and Masoudi et al.(12) leverage Q-learning algorithms to determine optimal sleep durations for minimizing energy consumption while respecting latency and QoS constraints. Notably, Amine et al.(13) incorporates co-channel interference into their HetNet Q-learning solution.

However, current MSSM algorithms primarily focus on optimizing sleep modes at the individual small-cell level, neglecting the overlapping coverage areas within HetNets. This is where the concept of "energy profitability" introduced by Fall et al.(14) becomes relevant. It defines a threshold below which offloading traffic to the macrocell becomes more energy-efficient for the network, considering resource availability. Ultimately, effective energy optimization strategies require maintaining QoS.  As highlighted throughout these studies, robust coordination between macrocells and picocells remains essential for achieving this goal.

 

methodology

Good coordination between macrocells and picocells is essential for an effective SM system. Thus, we introduced a coordinated multi-state SM mechanism, where the picocell manages the first 3 states of a pico-cell, and the associated macrocell manages the 4th state.

 

Macrocell - Picocell coordination principle

A base station must wake up periodically to send signaling bursts to communicate with the mobile terminals.

The article(8) introduced 4 states of sleep: SM1, SM2, SM3 and SM4 (Table 1):

·     SM1: represents the shortest SM, with a transition time of 71 µ s corresponding to an OFDM symbol. In this mode, only the power amplifier and some processing components are disabled.

·     SM2: corresponds to an intermediate sleep with a transition time of 1 ms equivalent to a subframe. We have more disabled processing components here.

·     SM3 corresponds to a sleep state with a transition time of 10 ms (one frame) in which all essential or processing components and the power amplifier are disabled.

·     SM4: corresponds to BSM where only the wake-up functions are retained. This is the deepest sleep and requires a transition duration of 1s.

 

Table 1. Characteristics relating to sleep states

Sleep state

Activation time

Deactivation time

Minimal sleep time

SM1

35,5 µs

35,5 µs

71 µs​

SM2

0,5ms

0,5ms

1ms

SM3

5ms

5ms

10ms

SM4

0,5s

0,5s

1 sec

 

Salem et al.(15) present a detailed table of the different components disabled at each state (Table 2).

 

Table 2. Components disabled in each mode(15)

Components

Subcomponents

SM1

SM2

SM3

SM4

Power amplifier

 

X

X

X

X

Baseband unit

Prewarp

 

 

X

X

Filtering

X

X

X

X

FFT/IFFT

X

X

X

X

MIMO precoding

X

X

X

X

Synchronization

X

X

X

X

Channel estimation

X

X

X

X

Calculation of the Equalizer

X

X

X

X

Equalization

X

X

X

X

OFDM Mod/ Demod

X

X

X

X

Mapping/ Demapping

X

X

X

X

Channel coding

X

X

X

X

Control unit

Control

 

 

 

 

Backhaul

 

 

 

 

Network

 

 

 

 

Analog receiver

LNA

X

X

X

X

LNA2

X

X

X

X

Frequency synthesis

 

X

X

X

Blender

X

X

X

X

VGA

X

X

X

X

Clock

 

 

 

X

ADC

 

 

X

X

Analog transmitter

Modulator

X

X

X

X

Buffer

X

X

X

X

Frequency synthesis

 

X

X

X

VCO Feedback

 

X

X

X

Blender

X

X

X

X

Clock

 

 

 

X

DAC

 

X

X

X

DAC feedback

 

 

X

X

Energy source

AC DC

 

 

 

 

DC/DC

 

 

 

 

Air conditioner

 

 

 

 

 

In 5G, the CRS reference signal is no longer necessary, and synchronization signals can be transmitted with a periodicity of 5, 10, 20, 40, 80, or 160ms.(16) Based on these values, Amine et al.(17) declared that SM4 could not be used and limited their work to the first 3 states.

SM4, for its part, can be compared to the sleep state in binary mode. In the scheme proposed, this state is controlled by the macrocell, which can put the picocells to sleep (SM4) or wake them up. Amine et al.(17) would thus perfectly complement the previous work presented by Fall et al.(14) to have a complete mechanism for handling a 4-state SM: "coordinated multi-state SM." The picocells control the first 3 states (SM1, SM2, and SM3) and can transit between them autonomously. The SM4 state, on the other hand, is controlled by the macrocell, which decides which picocells to activate or deactivate. Therefore, this state is assimilated into the base station's inactive state. From the macrocell perspective, there are only two states at the picocell level, either active or in SM4 (inactive).

The defined management mechanism is based on coordination between the macrocell and the picocells. The macrocell is based on the energy profitability threshold Sre(14) to activate and put a small cell into SM4. When a small one is active, it can manage internal transitions between the active state and the SMs SM1, SM2, and SM3 through a learning algorithm that decides at each cycle (Table 3).

 

Table 3. State management of a small cell

 

 

Managed States

Transition

Active

SM1

SM2

SM3

SM4

Macrocell

X

 

 

 

X

Safe

Small cell

X

X

X

X

 

Reinforced learning

 

Macrocell management based on energy profitability

The macrocell integrates algorithms for optimizing energy consumption based on energy profitability for deactivating unprofitable picocells or activating picocells in sleep mode (Figure 1).

At each SM4 cycle, corresponding to the minimum sleep duration for the state considered and equal to 1s, the deactivation algorithm is executed for all the picocells. It is based on the profitability threshold of picocells to decide whether to put them on standby (SM4). This evaluation frequency (1s) can be increased if traffic fluctuates slightly. Regarding picocells in SM4, their activation will depend on the charge or saturation level of the macrocell. These different parameters are grouped around the concept of load index Ic presented Fall et al.(14).

 

Figure 1. Picocell management algorithms

 

Integrating the multi-state SM at the SB of picocells can save significant energy savings. Lähdekorpi et al.(18)  highlighted these savings as up to 92 % at low load or 15 % at 40 %. The following figure illustrates this new consumption curve:

 

Figure 2. Picocell energy consumption curve

 

The new consumption curve looks like a logarithmic function, which we have modeled as follows:

 

 

Where Cp represents the power consumed by the base station; Ap represents the coefficient of the logarithm function; bp is a horizontal translation value of the energy curve; Cp0 represents the fixed component of the power; And x represents the percentage of chargeBased on our curvature estimates, we have used the base 2 logarithm:

 

 

We, therefore, need to re-evaluate the break-even point, materialized by the equality between the consumption of the macrocell Cxm (induced by the UEs) and that of the pico-cell Cxp induced by the same number of UEs.

 


Where Np represents the total number of UEs supported by the picocell, so we have the following equation to find Sre:

 


Picocell management based on reinforced learning

Each picocell SB autonomously manages 3 other sleep states through a reinforced learning policy, adapted to decision problems. We have chosen the Q-learning algorithm, based on the evaluation of a state-value function Q, which links a state S to an action A, and which is updated at each iteration.(19)

Our work is thus associated with that presented by Amine et al.(17) who proposed a Q- learning algorithm which is presented as follows:

 

Figure 3. Description of the Q- learning algorithm(17)

 

However, a buffer must be set up in each picocell to keep incoming connection requests that find the picocell in SM.(15,4) We propose the establishment of a buffer that would store the service requests and, depending on the depth of the sleep state and the phase in which the base station is, would decide between waiting for the base station to wake up and offloading requests to the macrocell. This offloading only concerns demand and not the UE as a whole.

 

RESULTS AND DISCUSSION

Simulation scenarios

For our simulations, we use the environment that we defined in our article Fall et al.(14) and which is inspired by the "Urban Dense" architecture of(20), namely:

A surface of dimension 500m x 500m with a Macro BS in the center (0,0) and 25 Pico BS distributed according to a Poisson law. The BS transmits at a frequency of 2GHz. There are 100 UEs for a low load, 500 for a medium load, or 1000 for a high load: 80 % indoors at a speed of 0,3m/s and 20 % outdoors at a speed of 10m/s.

We have the following parameters (Table 4):

 

Table 4. Simulation parameters

Setting

Value

Simulation surface

500m x 500m

Number of MBS

01

Transmission power

43 dBm

Number of PBS

25

Transmission power

23dBm

Bias (CRE)

6dB

MBS frequency

2GHz

Frequency of PBS

3,6GHz

Bandwidth

20 MHz

Loss Model

UMa3D and UMi3D

 

In(14) we presented the table below, which describes the standard energy consumption model of 5G base stations (Table 5):

 

Table 5. Energy consumption of 5G macro and pico BS

Base station type

Max number of UEs

Energy cost (W)

Load factor (W)

Zero load

Sleep state

Macro

800

780

450

564

Pico

32

40,8

25,8

3

 

Evaluation of the break-even point

The break-even point Sre is defined by the following equation:

 


 


Evaluation of Ap, ap, bp and Cp0

 

 

With 3 remarkable points:

 


Where Cp represents the consumption function of a picocell without multi-state SM. We have the following system of equations:

 


We consider Cp0 it is to be the fixed component at zero load. So, we have:

 


The equation becomes:

 


Figure 4. Evaluation of the energy profitability threshold

 

With BSM, the break-even point Sre was 25 UEs, while with ASM, it is 69. This is because ASM is more efficient, so most of the energy consumed by the picocell is associated with the UEs. In contrast, with BSM, the picocell consumed almost all of its maximum energy, even at zero load or in SM. The energy consumption per UE is very low: 0,705W for a microcell UE and 1,369W for a picocell UE.

Therefore, the main reasons for deploying picocells are not energy-related but are instead focused on improving throughput and capacity. For picocells to be energy-efficient, the energy consumption per UE (Cup) must be lower than that for the macro-cell (Cum).

We note this factor k:

 


Table 6. Break-even threshold values as a function of k

k

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

Sre

69

27,87

16,35

11,04

8,06

6,178

4,898

3,979

3,293

2,763

2,345

2,008

1,732

1,502

1,308

1,143

1,001

0,879

0,772

0,678

 

The main motivation for deploying picocells could be reducing energy consumption if we lower the factor k to 18, making Sre equal to 1. This would make picocells profitable regardless of their load. Even below this factor of 18, reducing energy consumption can still be a goal, considering the average load of picocells relative to the break-even threshold. If the average load of picocells is below this threshold, deploying picocells can be seen as aiming to reduce energy consumption.

 

Simulation results

Fall et al.(14) presented our energy consumption optimization algorithm (AOCE), compared to Max-SINR,(21) CRE (Cell Range Expansion)(22) and the energy optimum, which represents the theoretical minimum consumption. With this new coordination, integrating multi-state SM, we have AOCE2 which represents its version 2.

Introducing ASM, along with macrocell-picocell coordination, maximizes energy optimization. At low load, as shown in figure 5, the ECOA (ASM) aligns with the optimal (ASM) because all UEs are connected to the macrocell. This happens because the break-even threshold exceeds the maximum number of UEs a picocell can handle, making the UE-macrocell association more efficient than the UE-picocell association. For the same reason, the MaxSINR association mode is more economical than the CRE mode.


Figure 5. Energy consumption at low load

 

At high load, as shown in figure 6, the difference in energy consumption between BSM and ASM naturally decreases, and the likelihood of using SM is lower. There is a noticeable difference between the ECOA and the optimal scenario in ASM. The picocells become loaded with the number of UEs exceeding the macrocell's maximum capacity. In this situation, achieving an optimal configuration is nearly impossible due to the saturation of active picocells. However, the ECOA remains close to optimal, with only a 7,45 % increase in consumption.

 

Figure 6. Energy consumption at high load

 

Coordination assessment

The new ECOA2 algorithm combines the benefits of ASM, limited to state 3 of SM, with the advantages of coordinated cellular association from our previous article, which extends to state 4. This allows for optimal use of picocell SM (see Table 7).

 

Table 7. Energy savings compared to MaxSINR BSM

 

100 UEs

500 UEs

1000 UEs

ECOA 1

11,24 %

15,28 %

11,26 %

MaxSINR ASM

33,37 %

17,88 %

11,02 %

ECOA 2

44,68 %

41,15 %

21,61 %

 

The algorithm can save up to 45 % energy compared to traditional cellular selection with BSM. Most of the savings come from ASM; a picocell can reduce energy consumption by 92 % at zero load. However, the impact of coordination is less significant due to the low number of UEs. As the load increases, the energy savings from ASM decrease because of the logarithmic nature of the consumption curve. At 40 % load, ASM savings drop from 92 % to 15 %. Conversely, as the load increases, the coordination between the macrocell and picocells becomes more crucial, leading to greater energy savings. The reduction mechanisms near their limits at high load levels, and potential savings become increasingly minimal.

Figure 7 illustrates the energy consumption at low, medium, and high loads. The ECOA ASM effectively optimizes energy consumption across these different loads. However, it's essential also to consider other metrics such as QoS and throughput.

 

Figure 7. Energy consumption at 100, 500, and 1000 UEs

 

Figure 8 shows the average throughput per UE at low, medium, and high loads. ECOA ASM has a smaller average throughput at low load compared to ECOA BSM. Its throughput is notably lower at medium load because the UEs are associated with the macrocell, preventing the picocells from reaching their break-even point. ECOA ASM also has a lower average flow rate at high load, as the macrocell is fully utilized before the picocells are engaged. The high profitability threshold, which exceeds the picocell's maximum capacity, negatively impacts throughput and overall quality of service. Since picocells perform better than macrocells, the break-even point must be reduced to optimize energy and improve QoS. However, ECOA ensures QoS by allowing uplink transfers only when certain metric limits defined by the load index in the article, subject to compliance with certain metric limits defined through the load index present by Salem et al.(15), are met.

 

Figure 8. Average throughput per UE at various loads

 

Conclusion

This paper presented a novel approach to tackle the growing energy consumption challenge in 5G networks. Focusing on heterogeneous networks with macrocells and picocells, we proposed a coordinated multi-state sleep mode (SM) strategy that combines two established techniques: Advanced Sleep Mode (ASM) and macrocell-picocell coordination for cell selection.

We developed a new logarithmic energy consumption model for picocells to achieve this. This model and implementation of Q-learning, a machine learning algorithm, allowed for dynamic and efficient management of ASM states within picocells. Additionally, a new energy profitability threshold was calculated for picocells to guide macrocell coordination decisions. This comprehensive approach, named "coordinated multi-state SM," leverages the full potential of sleep states and macrocell-picocell collaboration, resulting in significant energy savings of up to 45 %.

While this work demonstrates the effectiveness of coordinated multi-state SM, further research holds promise for even more significant energy efficiency gains. One key area of focus lies in optimizing the hardware components within picocells. By exploring more energy-efficient hardware designs, we can further minimize the energy consumption associated with user equipment (UE)-base station links. This optimization would maximize the overall benefits achieved through macrocell-picocell coordination. Additionally, investigating more advanced coordination algorithms between macrocells and picocells has the potential to unlock even more significant energy savings.

 

REFERENCES

1. Cell size breathing and possibilities. Micallef, G., Mogensen, P. and Scheck, H. 2010, 2010 European Wireless Conference (EW), pp. 111-115.

 

2. Base Station ON-OFF Switching in 5G Wireless Networks: Approaches and Challenges. Feng, Mingjie, Mao, Shiwen and Jiang, Tao. 4, Aug. 2017, IEEE Wireless Communications, Vol. 24, pp. 46-54.

 

3. A Survey of Energy-Efficient Techniques for 5G Networks and Challenges Ahead. Buzzi, Stefano, et al. 4, April 2016, IEEE Journal on Selected Areas in Communications, Vol. 34, pp. 697-709.

 

4. Energy Optimization with Multi-Sleeping Control in 5G Heterogeneous Networks using Reinforcement Learning. Amine, Ali El, et al. 4, Dec. 2022, IEEE Transactions on Network and Service Management, Vol. 19, pp. 4310-4322.

 

5. Sleep mode implementation issues in green base stations. Saker, L. and Elayoubi, S. E. 2010, 21st Annual IEEE International Symposium on Personal, Indoor and Mobile Radio Communications, pp. 1683-1688.

 

6. S. Elayoubi, L. Saker and T. Chahed. 2011, 2011 Proceedings IEEE INFOCOM, pp. 106-110.

 

7. Reducing Energy Consumption in LTE with Cell DTX. Frenger, P., et al. Yokohama, : s.n., 2011. 2011 IEEE 73rd Vehicular Technology Conference (VTC Spring). pp. 1-5.

 

8. A Flexible and Future-Proof Power Model for Cellular Base Stations. Debaillie, Bjorn, Desset, Claude and Louagie, Filip. May 2015, IEEE 81st Vehicular Technology Conference (VTC Spring).

 

9. Optimal Control of Wake Up Mechanisms of Femtocells in Heterogeneous Networks. Saker, L., et al. 3, April 2012, IEEE Journal on Selected Areas in Communications, Vol. 30, pp. 664-672.

 

10. Pervaiz, Haris, et al. 4, Dec 2018, EEE Vehicular Technology Magazine, Vol. 13, pp. 51-59.

 

11. Reinforcement learning approach for Advanced Sleep Modes management in 5G networks. Salem, Fatma Ezzahra, et al. Aug 2018, 2018 88th Vehicular Technology Conference (VTC-FALL).

 

12. Reinforcement Learning for Traffic-Adaptive Sleep Mode Management in 5G Networks. Masoudi, Meysam, et al. Oct 2020, 2020 IEEE 31st Annual International Symposium on Personal, Indoor and Mobile Radio Communications.

 

13. Reinforcement Learning for Delay-Constrained Energy-Aware Small Cells with Multi-Sleeping Control. Amine, Ali El, Dini, Paolo and Nuaymi, Loutfi. Jul 2020, 2020 IEEE International Conference on Communications Workshops (ICC Workshops).

 

14. Towards Sustainable 5G Networks: A Proposed Coordination Solution for Macro and Pico Cells to Optimize Energy Efficiency. FALL, MACOUMBA, et al. May 2023, IEEE Access, Vol. 11.

 

15. Management of advanced sleep modes for energy-efficient 5G networks. Salem, Fatma Ezzahra. Dec 2019, Networking and Internet Architecture [cs.NI].

 

16. 3GPP. TR 36.814 Evolved Universal Terrestrial Radio Access (E-UTRA); Further advancements for E-UTRA physical layer aspects (Release 9). Mar. 2017.

 

17. Energy Optimization With Multi-Sleeping Control in 5G Heterogeneous Networks Using Reinforcement Learning. Amine, Ali El, et al. 4, Dec. 2022, EEE Transactions on Network and Service Management, Vol. 19, pp. 4310-4322.

 

18. Energy efficiency of 5G mobile networks with base station sleep modes. Lähdekorpi, Panu, et al. 2017, 2017 IEEE Conference on Standards for Communications and Networking (CSCN), pp. 163-168.

 

19. Sutton, Richard S. and Barton, Andrew G. Reinforcement learning : An introduction. s.l. : Cambridge, MA : The MIT Press, 2018.

 

20. ETSI - ETSI TR 138 913 V14.3.0. 5G - Study on scenarios and requirements for next generation access technologies. 2017.

 

21. Khandekar, A., Bhushan, N. and Tingfang, J. LTE-Advanced: Heterogeneous. 2010.

 

22. 3GPP. 3GPP Release 10 - TR 36.942 V10.2.0. 2010.

 

23. Wu, J., et al. 2, 2015, IEEE Communications Surveys & Tutorials, Vol. 17, pp. 803-826.

 

24. Abdellaoui, Maroua and Fattah, Mohammed. Characterization of Ultra Wide Band indoor propagation. Mediterranean Congress of Telecommunications (CMT). 24-25 October 2019.

 

25. Resource placement strategy optimization for smart grid application using 5G wireless networks. Chafi, Saad-Eddine, et al. 4, 2022, International Journal of Electrical and Computer Engineering (IJECE), Vol. 12, pp. 3932-3942.

 

26. 5G uplink interference simulations, analysis and solutions: The case of pico cells dense deployment. Younes, Balboul, et al. 3, June 2021, International Journal of Electrical and Computer Engineering (IJECE), Vol. 11, pp. 2245-2255.

 

27. The impact of scheduling algorithms for real-time traffic in the 5G femto-cells network. MAMANE, Asmae, et al. Rabat : s.n., 09 May 2019, International Symposium on Signal, Image, Video and Communications (ISIVC).

 

FINANCING

None.

 

CONFLICT OF INTEREST

The authors declare that there are no conflicts of interest.

 

AUTHORSHIP CONTRIBUTION

Conceptualization: Macoumba Fall, Mohammed Fattah, Mohammed Mahfoudi, Younes Balboul, Said Mazer, Moulhime El Bekkali, Ahmed D. Kora.

Data curation: Macoumba Fall, Mohammed Fattah, Mohammed Mahfoudi, Younes Balboul, Said Mazer, Moulhime El Bekkali, Ahmed D. Kora.

Formal analysis: Macoumba Fall, Mohammed Fattah, Mohammed Mahfoudi, Younes Balboul, Said Mazer, Moulhime El Bekkali, Ahmed D. Kora.

Research: Macoumba Fall, Mohammed Fattah, Mohammed Mahfoudi, Younes Balboul, Said Mazer, Moulhime El Bekkali, Ahmed D. Kora.

Methodology: Macoumba Fall, Mohammed Fattah, Mohammed Mahfoudi, Younes Balboul, Said Mazer, Moulhime El Bekkali, Ahmed D. Kora.

Supervision: Macoumba Fall, Mohammed Fattah, Mohammed Mahfoudi, Younes Balboul, Said Mazer, Moulhime El Bekkali, Ahmed D. Kora.

Validation: Macoumba Fall, Mohammed Fattah, Mohammed Mahfoudi, Younes Balboul, Said Mazer, Moulhime El Bekkali, Ahmed D. Kora.

Original drafting and editing: Macoumba Fall, Mohammed Fattah, Mohammed Mahfoudi, Younes Balboul, Said Mazer, Moulhime El Bekkali, Ahmed D. Kora.

Writing - proofreading and editing: Macoumba Fall, Mohammed Fattah, Mohammed Mahfoudi, Younes Balboul, Said Mazer, Moulhime El Bekkali, Ahmed D. Kora.