TIANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY Master’s Thesis in Data Science and Artificial Intelligence An Efficient, On-demand Charging for WRSNs Using Fuzzy Logic and Q-Learning La Van Quan Quon.vn Supervisor: Dr. Nguyen Phi Le Department: Department cf Software engineering Institute: School of Information and Communication ‘lechnology Hanoi, 2022 Declaration of Authorship and Topic Sentences 1. Personal information Full name: La Yan Quan Phone number: 039 721 1659 Email: Quan. vir Mojor: Data Scicuce uid Artificial Intelligence 2.
"Topic An Efficient, On-demand Charging for WRSNs Using urzy Logic and Q-Tearning 3. Contributions © Propose « Fuzzy logic-based algorithm that determines the energy level to be charged to the sensors. © Introduce a new method that optimizes the optimal charging time at each charging location to maximize the number of alive sensors.charging, which uses Q-learning in its charging scheme to guarantce the target coverage and connectivity, 4A. Declaration of Authorship Thercby declare that my thesis, titled ‘An Bificlent, On-demand Charging for WRSNs Using Fuzzy Logic and Q-Learning", is the work of myself and my supervisor Dr.
Nguycn Phi Le. All papers, sources, tables, and so on used in this thesis huve been (horoughly cited,. Supervisor confirmation on Ha Noi, April 2022 Supervisor Dr. Nguyen Phi Le Acknowledgments I would like to thank my supervisor, Dr.
Nguyen Phi Le, for her continued support and guidance throughout the course of my Masters’ studies. She has been a great teacher and mentor for me since iny underyraduate years, and I am proud tu have completed this thesis under her supervision. Twant to thank my family aud my friends, who have given me their unconditional Jove and support to finish my Masters’ studies, Finally, T wonid like to again thank Vingronp and the Vingronp Trnovation Foundation, who have supported my studies through their Domestic Master/Ph. Paxts of this work were published in the paper “(Q-learning based, Optimized On.
demand Charging Algorithm in WRSN” by La Van Quan, Phi Le Nguyen, Thanh. Hung Nguyen, and Kieu Nguyen in the Proceedings of the 19th IEEE International Symposium on Network Computing and Applications, 2020. La Van Quan was fuuded by Vingroup Joint Stock Company and supported by the Domestic: Master/Th. Scholarship Programme of Vingroup Tnnovation Foun- dation (VINIF], Vingronp Tig Data Institute, eade VINTF.1 Problem overview Wireless Sensor Networks (WSNs) have found various applications, such as air quality monitoring, environmental management, ete.
A WSN typically in- cludes many battery-powered sensor nodes, monitoring several target and sending sensed data to a base station for further proe ing. In the WSNs, it is nec« ry to provide sufficient monitoring quality surrounding the targets (i., guarantee- ing target coverage). Moreover, the WSNs need to have adequate capacity for the communication between the sensors and base station (i. The target coverage and connectivity are severely affected by the depletion of the battery on sensor nodes, When a node runs out of battery, it becomes a dead node without s nsing and communication capability, damaging the whole network in consequence.
Wireless Rechargeable Sensor Networks WRSNs) leverages the advan- tages of wireless power transferring technology to solve that critical issue in WSNs. A WRSN uses a mobile charger (MC) to wirelessly compensate for a rechargeable battery's energy consumption on a sensor node, aiming to guarantee both the target coverage and connectivity In a normal operation, the MC moves around the networks and performs charg- ing strategie: h can be classified into the periodic {{)][1][10][11][12] or on-demand charging [1][2][/4][15] [10][L7][1S]. In the former, the MC, with a predefined trajec- tory, stops at charging locations to charge the nearby sensors’ batteries. In the latter.
the MC will move and charge upon receiving requests from the sensors, which have the remaining energy below a threshold. The periodic strategy is limited since it can- not adapt to the sensors’ energy consumption rate dynamic. On the contrary, the on-demand charging approach potentially deals with the uncertainty of the energy consumption rate. Since a sensor with a draining battery triggers the on-demand op- cration, the MC’s charging strategy faces a new time constraint challenge.
The MC needs to handle two crucial issues: deciding the next charging location and staying period at the location. Comparison with existing algorithms 5.1 Impacts of the number of sensors 2 Impacts of the number of targets 5.3 Impacts of the packet generation frequency. 524 Non-monitored targets and dead sensors over time Tiibliography Chapter 1 Introduction 1.1 Problem overview Wireless Sensor Networks (WSNs) have found various applications, such as air quality monitoring, environmental management, ete. A WSN typically in- cludes many battery-powered sensor nodes, monitoring several target and sending sensed data to a base station for further proe ing.
In the WSNs, it is nec« ry to provide sufficient monitoring quality surrounding the targets (i., guarantee- ing target coverage). Moreover, the WSNs need to have adequate capacity for the communication between the sensors and base station (i. The target coverage and connectivity are severely affected by the depletion of the battery on sensor nodes, When a node runs out of battery, it becomes a dead node without s nsing and communication capability, damaging the whole network in consequence. Wireless Rechargeable Sensor Networks WRSNs) leverages the advan- tages of wireless power transferring technology to solve that critical issue in WSNs.
A WRSN uses a mobile charger (MC) to wirelessly compensate for a rechargeable battery's energy consumption on a sensor node, aiming to guarantee both the target coverage and connectivity In a normal operation, the MC moves around the networks and performs charg- ing strategie: h can be classified into the periodic {{)][1][10][11][12] or on-demand charging [1][2][/4][15] [10][L7][1S]. In the former, the MC, with a predefined trajec- tory, stops at charging locations to charge the nearby sensors’ batteries. In the latter. the MC will move and charge upon receiving requests from the sensors, which have the remaining energy below a threshold.
The periodic strategy is limited since it can- not adapt to the sensors’ energy consumption rate dynamic. On the contrary, the on-demand charging approach potentially deals with the uncertainty of the energy consumption rate. Since a sensor with a draining battery triggers the on-demand op- cration, the MC’s charging strategy faces a new time constraint challenge. The MC needs to handle two crucial issues: deciding the next charging location and staying period at the location.
Although many, the <isting on-demand charging schemes in the literature face two seriona problems. The first one is the consideration of the same role for the sen- sor nodes in WRSNs. ia somewhat unrealistic since, intuitively, several sensors, depending on their locations, significantly impact the target coverage and the con- nectivity than others. Hence, the existing charging schemes may enrich unnecessary sensors’ power while letting necessary ones run out of energy, leading to charging algoriUnus" jneflicicney.
Tt is of great importance to tuke inte account the target coverage und connectivity simulluncously. The second problen is about Uie MC's chorging awount, which iy vither a ull capacity (uf seusor buttcry) or a fixed amount of energy. The former case may cause: 1) a. long waiting time of other sensors stay- ing near the charging lacatin; 2) qnick exhanstion of the MC’s energy.
Tn contrast, charging a too small amomnt to a node may tead to its lack of power ta operare until the next charging round. ‘Therefore, the charging strate, should adjust the transferred energy level dynamically following the network condition. Thesis contributions Motivated by the above, this thesis propos a nevel on-demand charging scheme for WRSN that assures the larget coverage aud connectivity and adjusts Lie energy level charged to the sensors dynamically. Jy proposal, named Fuzzy Q-charging, aims to maximize the network lifetime, which is the time until the first target is nov monitored.
First, this work exploit Fuzzy logic in an optimization algorithm that determines the optimal charging time at each charging location, aiming to maximize the numbers of alive sensors and monitoring targets. Fuzzy logic is used to cope with network dynamics by taking various nctwork parameters into account during the determination process of optimal charging time. Socond, this thesis leverage the Qlcamning technique in 9 now algorithm that selects the next charging location to maximize Uhe network lifetiine. The MC naintains a Q-table coulaiuing Uhe charging locations’ Q-values representing the charging locations’ goodness.
The Q-values will be updated in a real-time manner whenever there is a new charging request from a sensor. I design the Q-value to pricritize charging locations at which the MC cen charge a node depending on its critical role. After finishing tasks in one place, the MC chooses the next one, which has the highest Q-value, and determines an optimal charging time. The main contributions of the paper are as follows.
« This thesis propose a Fuzzy logic-based algorithm that determines the energy level to be charged to the sensors. ‘I'he energy level is adjusted dynamically following the nctwork condition. @ Based on the above algorithm, this thesis introduce a new method that opti- mizes the optimal charging time at each charging location. It considers sev- 3.
Comparison with existing algorithms 5.1 Impacts of the number of sensors 2 Impacts of the number of targets 5.3 Impacts of the packet generation frequency. 524 Non-monitored targets and dead sensors over time Tiibliography Contents List. of Figures vi List of Tables 1 Introduction ee 1. 12) Thesis contributions wh 1.
2 Theoretical Basis hee 2.1 Wircluss Reehurgeable Scusor Networks.3 Furry Logic 3 Literature Review 3L Relaved Work. 32 Problem delinition 4 Pussy Qcharging algorithm AL Overview.2 State space, uvtion space and Q tuble 4.4 Charging time determination 4A Fury logic-based safe energy level determination 44. Furzification 443 Luzzy controller 44. Q lable update Experimental Results 24 e 5.1 Impacts of parameters 25 SLL Impacts of & 2 Impacts of + Although many, the <isting on-demand charging schemes in the literature face two seriona problems.
The first one is the consideration of the same role for the sen- sor nodes in WRSNs. ia somewhat unrealistic since, intuitively, several sensors, depending on their locations, significantly impact the target coverage and the con- nectivity than others. Hence, the existing charging schemes may enrich unnecessary sensors’ power while letting necessary ones run out of energy, leading to charging algoriUnus" jneflicicney. Tt is of great importance to tuke inte account the target coverage und connectivity simulluncously.
The second problen is about Uie MC's chorging awount, which iy vither a ull capacity (uf seusor buttcry) or a fixed amount of energy. The former case may cause: 1) a. long waiting time of other sensors stay- ing near the charging lacatin; 2) qnick exhanstion of the MC’s energy. Tn contrast, charging a too small amomnt to a node may tead to its lack of power ta operare until the next charging round.
‘Therefore, the charging strate, should adjust the transferred energy level dynamically following the network condition. Thesis contributions Motivated by the above, this thesis propos a nevel on-demand charging scheme for WRSN that assures the larget coverage aud connectivity and adjusts Lie energy level charged to the sensors dynamically. Jy proposal, named Fuzzy Q-charging, aims to maximize the network lifetime, which is the time until the first target is nov monitored. First, this work exploit Fuzzy logic in an optimization algorithm that determines the optimal charging time at each charging location, aiming to maximize the numbers of alive sensors and monitoring targets.
Fuzzy logic is used to cope with network dynamics by taking various nctwork parameters into account during the determination process of optimal charging time. Socond, this thesis leverage the Qlcamning technique in 9 now algorithm that selects the next charging location to maximize Uhe network lifetiine. The MC naintains a Q-table coulaiuing Uhe charging locations’ Q-values representing the charging locations’ goodness. The Q-values will be updated in a real-time manner whenever there is a new charging request from a sensor.
I design the Q-value to pricritize charging locations at which the MC cen charge a node depending on its critical role.