Resource Allocation in Computation Offloading Based on improved two-layer Q-network Reinforcement Learning

Maryam Thabit Hussein; Ehsan Shoja

Author(s)

Maryam Thabit Hussein Department of electrical and computer engineering, Urmia University,Urmia,Iran

Ehsan Shoja Department of electrical and computer engineering, Urmia University,Urmia,Iran

Abstract

Collaborative edge computing (CEC) and mobile-edge computing (MEC)have recently emerged as a popular approach that allows for the sharing of computational resources among various edge devices. In this model, multiple parties, including mobile users, edge computing servers, and cloud servers, work together to enhance computational capabilities, enabling the efficient handling of computation-heavy tasks at the edge. The paper presents a model where each task and its associated data, whether processed in the cloud or at the edge, is represented as a network of interdependent sub-tasks. A critical issue in CEC is the allocation of resources for offloading tasks, which involves determining the timing and location for executing each subtask. This work establishes a mathematical framework for optimizing resource allocation to reduce the average delay in completing tasks. The MEC network incorporates mobile users, edge computing servers, and cloud servers. Efforts have been made to develop both static and dynamic strategies aimed at minimizing the average computation delay for tasks within the network. The system was modeled as a Queuing Network to derive optimal routing probabilities under a static policy. Additionally, the problem was modeled using a ContinuousTime Markov Chain (CTMC), but due to the vast state space and dynamics of the CTMC, coupled with a reward structure that becomes apparent after several state transitions, Deep Reinforcement Learning algorithms were employed to determine the most effective dynamic policy. While the static policy conserves energy by not requiring system state measurements, the dynamic policy provides superior performance in reducing average delay compared to benchmark algorithms.

Publication Details

Page(s) 471-486

DOI DOI not available

Published Journal: International Journal of Communication Networks and Information Security, Volume: 16, Issue: 5, Year: 2024

Request the full text from PASTIC

Keywords

Cloud Computing, resource allocation, Offloading, Reinforcement Learning, Mobile Edge Computing, Collaborative Edge Computing

References

Mao Y.,Zhang and K. B. Letaief J. .2016 ."Dynamic Computation Offloading for Mobile-Edge Computing With Energy Harvesting Devices,". IEEE Journal on Selected Areas in Communications, 2611964(12) : 3590-3605.

Hong Z.,Chen W.,Huang H.,Guo S.,Zheng Z. .2019 ."Multi-Hop Cooperative Computation Offloading for Industrial IoT-Edge-Cloud Computing Environments,". IEEE Transactions on Parallel and Distributed Systems, 2926979(2774) : 2759.

Funai C.,Tapparello C.,Heinzelman W. .2019 ."Computational Offloading for Energy Constrained Devices in Multi-Hop Cooperative Networks,". IEEE Transactions on Mobile Computing, 2892100(73) : 60.

Bertsekas D. P. .1995 .Dynamic programming and optimal control. Athena Scientific Belmont, 1 : .

Ahmed A.,Ahmed E. .2016 ."A survey on mobile edge computing,". in 2016 10th International Conference on Intelligent Systems and Control (ISCO), 7727082 : 1-8.

Tran T. X.,Pompili D. .2018 ."Joint Task Offloading and Resource Allocation for Multi-Server MobileEdge Computing Networks,". IEEE Transactions on Vehicular Technology, 2881191(1) : 856-868.

Guo S.,Xiao B.,Yang Y.,Yang Y. .2016 ."Energy-efficient dynamic offloading and resource scheduling in mobile cloud computing,". in IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications, 7524497 : 1-9.

Sundar S.,Liang B. .2018 ."Offloading Dependent Tasks with Communication Delay and Deadline Constraint,". in IEEE INFOCOM 2018 - IEEE Conference on Computer Communications, 8486305 : 37-45.

Sun X.,Ansari N. .2017 ."Latency Aware Workload Offloading in the Cloudlet Network,". IEEE Communications Letters, 2690678(7) : 1481-1484.

Ning Z.,Dong P.,Kong X.,Xia F. .2018 ."A Cooperative Partial Computation Offloading Scheme for Mobile Edge Computing Enabled Internet of Things,". IEEE Internet of Things Journal, 2868616(3) : 4804-4814.

Kao Y.,Krishnamachari B.,Ra M.,Bai F. .2017 ."Hermes: Latency Optimal Task Assignment for Resource-constrained Mobile Computing,". IEEE Transactions on Mobile Computing, 2679712(3069) : 3056.

C. -S. Yang R.,Pedarsani A. S.,Avestimehr A. S. .2019 ."Communication-Aware Scheduling of Serial Tasks for Dispersed Computing,". IEEE/ACM Transactions on Networking, 2919553(4) : 1330-1343.

Sahni Y.,Cao J.,Yang L.,Ji Y. .2020 ."Multihop Offloading of Multiple DAG Tasks in Collaborative Edge Computing,". IEEE Internet of Things Journal, 3030926(6) : 15-March15.

Liu Y. .2020 ."Dependency-Aware Task Scheduling in Vehicular Edge Computing,". IEEE Internet of Things Journal, 2972041(6) : 4961-4971.

Shu C.,Zhao Z.,G Z.,Duan H.,Latency-Optimal Approach .2019 .". IEEE Internet of Things Journal, 2943373(3) : 1678-1689.

Dab B.,Aitsaadi N.,Langar R. .2019 ."Q-Learning Algorithm for Joint Computation Offloading and Resource Allocation in Edge Cloud,". in 2019 IFIP/IEEE Symposium on Integrated Network and Service Management (IM), : 45-52.

Huang L.,Bi S.,Zhang Y. -J. A. .2019 ."Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks,". IEEE Transactions on Mobile Computing, 2928811(2593) : 2581.

Wang S.,Zafer and K. K. Leung M. .2017 ."Online Placement of Multi-Component Applications in Edge Computing Environments,". IEEE Access, 2665971 : 2514-2533.

Sun X.,Ansari N. .2017 ."Latency Aware Workload Offloading in the Cloudlet Network,". IEEE Communications Letters, 2690678(7) : 1481-1484.

Zhang Y.,Lan X.,Ren J.,Cai L. .2020 ."Efficient Computing Resource Sharing for Mobile Edge-Cloud Computing Networks,". IEEE/ACM Transactions on Networking, 2979807(3) : 1227-1240.

Hassan M.,Kacem I.,Martin S. .2014 ."Unrelated parallel machines with precedence constraints: application to cloud computing,". in 2014 IEEE 3rd International Conference on Cloud Networking (CloudNet), 6969034 : 438-442.

Sundar S.,Liang Ben .2016 ."Communication Augmented Latest Possible Scheduling for cloud computing with delay constraint and task dependency,". in 2016 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), 7562231 : 1009-1014.

Chao Masakiyo,Miyazawa Michael,Pinedo Michael .1999 .Queueing networks: customers, signals and product form solutions. , 1 : .

Boyd ,Stephen .2007 ."A tutorial on geometric programming. " Optimization and engineering 8, : 67-127.