【作者】 甘钊宇；

【导师】 邹华；

【作者基本信息】 北京邮电大学 ， 计算机科学与技术， 2022， 硕士

【摘要】 随着第五代移动通信技术(5G)的发展,5G的高带宽与低延迟正在为移动用户提供更多的便利并持续改善移动用户的使用体验,并助力着物联网、云游戏、无人驾驶等一系列新应用的研究与落地。为了满足日渐增长的应用的带宽与延迟需求,5G与移动边缘计算的结合正在成为一个新的范式。通过在网络边缘设置节点并与5G核心网连接,移动边缘计算可以承担移动设备的部分计算任务或实现内容缓存,从而释放移动设备的部分性能、加快内容加载速度并节省设备的电量。在未来,移动边缘计算将释放更多的潜能,提供高性能、低成本、按需供应的平台以支撑更多的应用场景。结合5G网络下移动边缘计算环境的特点,本文提出了 5G环境下的移动边缘计算任务调度系统,提供移动用户计算任务的卸载与调度服务,重点关注5G环境下移动边缘计算任务调度中的两个关键调度问题,分别为计算卸载的决策问题以及容器伸缩的优化问题,并使用强化学习方法在已有算法上进行进一步的优化。其一,受到移动边缘计算系统的总容量限制,并非所有的计算任务都适合被发送至移动边缘计算平台执行,因此选择高效的算法进行计算卸载是提升服务质量的重要因素之一。本文针对计算卸载中的计算、传输开销与能源消耗进行定量分析,提出一种基于多智能体强化学习的计算卸载决策算法,在移动边缘计算平台采用Q-Mix网络对多个移动终端的计算卸载决策进行综合评价,以增强移动终端的协作决策能力。其二,在移动边缘计算平台中有大量的计算任务在运行。为了更好地利用边缘计算资源,本文提出了基于强化学习的容器自动伸缩算法,在已有的基于离散动作空间的自动伸缩算法的基础上,建立连续动作空间并设计基于近端策略优化的容器自动伸缩算法,以实现更加精确的容器资源分配控制。在上述关键问题的基础上,本文最后完成了基于强化学习的5G移动边缘计算任务调度系统原型的设计与实现,以测试方案的可行性以及算法在实际使用中的性能。更多还原

【Abstract】 With the development of the fifth-generation of mobile network technology(5G),the abilities of high bandwidth and low latency in 5G are providing more convenience,continuously improving the experience for mobile users,and helping the research and implementation of a series of new applications,such as Internet of Things(IoT),autonomous vehicle,Virtual Reality(VR),Augmented Reality(AR),and immerse gaming,etc.The combination of 5G and Mobile Edge Computing(MEC)is emerging as a new paradigm to meet the bandwidth and latency demands of growing applications.By setting up nodes at the edge of the network and connecting them to the 5G core network,the MEC can undertake part of the computing tasks of mobile devices or implement content caching,thereby releasing part of the performance of mobile devices,speeding up content loading and saving the device power.In the future,mobile edge computing will release more potential and provide high-performance,low-cost,on-demand platforms to support more and more application scenarios.Combined with the characteristics of the mobile edge computing and the 5G network,this thesis proposes a task scheduling system of mobile edge computing in 5G environment,which provides computation tasks offloading for mobile users and services scheduling for MEC platform.There are two problems in this scenario that should be focused,which are the decision problem of computing offloading and the optimization problem of container scaling.This thesis uses the reinforcement learning approach to achieve further optimizations based on the existing algorithms.On the one hand,due to the total capacity of the mobile edge computing system,not all computing tasks are suitable to be sent to the mobile edge computing platform for execution.Therefore,it’s important to design an efficient algorithm for appropriate computing offload to improve service quality.This thesis quantitatively analyzes the computing cost,transmission cost and energy consumption in the computing offloading problem,and proposes a computing offloading decision algorithm based on multi-agent reinforcement learning,which uses a QMix network in the mobile edge computing platform to evaluate the decisions of mobile devices,and enhance the collaborative decisionmaking capability of mobile devices.On the other hand,there are a lot of computing tasks running on the mobile edge computing platform.In order to make a better use of the edge computing resources,this thesis proposes an auto-scaling algorithm for containers based on reinforcement learning,which formulates a continuous action space instead of the discrete action space and uses Proximal Policy Gradient(PPO)approach to achieve more precise control of container resource allocation.Based on the solutions of the above problems,this thesis finally proposes a design and an implementation of a prototype of the task scheduling system of mobile edge computing in 5G network based on reinforcement learning,to test the feasibility of the scheme and the performance of the algorithm in real environments.更多还原