A Q-Learning Approach for Dynamic Resource Management in Three-Tier Vehicular Fog Computing

Document Type : Research Article

Authors

1 Department of Computer Engineering and Information Technology, Razi University, Iran.

2 Faculty of Information Technology, Kermanshah University of Technology, Kermanshah, Iran.

10.22108/jcs.2026.147193.1184

Abstract

In this paper, a method for predicting the resources required for an intelligent vehicle client using a three-layer vehicular computing architecture is proposed. {This method leverages Q-Learning to optimize resource allocation and enhance overall system performance.} This approach employs reinforcement learning capabilities to provide a dynamic and adaptive strategy for resource management in a fog computing environment. The key findings of this study indicate that Q-learning can effectively predict the appropriate allocation of resources by learning from past experiences and making informed decisions. Through continuous training and updating of the Q-learning agent, the system can adapt to changing conditions and make resource allocation decisions based on real-time information. The experimental results demonstrate the effectiveness of the proposed method in optimizing resource allocation. The Q-learning agent predicts the optimal values for memory, bandwidth, and processor. These predictions not only minimize resource consumption but also meet the performance requirements of the fog system. Implementations show that this method improves the average task processing time compared to other methods evaluated in this study.

Keywords

Main Subjects

References

[1] Ahangar, M. Nadeem and Ahmed, Qasim Z. and Khan, Fahd A. and Hafeez, Maryam. A Survey of Autonomous Vehicles: Enabling Communication Technologies and Challenges. Sensors. https://www.mdpi.com/1424-8220/21/3/706. 21(3): 2021. [DOI ]
[2] Sekione Reward Jeremiah and Laurence Tianruo Yang and Jong Hyuk Park. Digital twin-assisted resource allocation framework based on edge collaboration for vehicular edge computing. Future Generation Computer Systems. https://www.sciencedirect.com/science/ article/ pii/S0167739X23003278. 150: 243-254, 2024. [DOI ]
[3] Zhou, Zhenyu and Liu, Pengju and Feng, Junhao and Zhang, Yan and Mumtaz, Shahid and Rodriguez, Jonathan. Computation Resource Allocation and Task Assignment Optimization in Vehicular Fog Computing: A Contract-Matching Approach. IEEE Transactions on Vehicular Technology. 68(4): 3113-3125, 2019. [DOI ]
[4] Rejiba, Zeineb and Masip-Bruin, Xavier and Marín-Tordera, Eva. Computation Task Assignment in Vehicular Fog Computing: A Learning Approach via Neighbor Advice. 2019 IEEE 18th International Symposium on Network Computing and Applications (NCA). (): 1-5, 2019. [DOI ]
[5] Zhu, Chao and Tao, Jin and Pastor, Giancarlo and Xiao, Yu and Ji, Yusheng and Zhou, Quan and Li, Yong and Ylä-Jääski, Antti. Folo: Latency and Quality Optimized Task Allocation in Vehicular Fog Computing. IEEE Internet of Things Journal. 6(3): 4150-4161, 2019. [DOI ]
[6] Shi, Jinming and Du, Jun and Wang, Jian and Yuan, Jian. Deep Reinforcement Learning-Based V2V Partial Computation Offloading in Vehicular Fog Computing. 2021 IEEE Wireless Communications and Networking Conference (WCNC). (): 1-6, 2021. [DOI ]
[7] Vivek Sethi and Sujata Pal. FedDOVe: A Federated Deep Q-learning-based Offloading for Vehicular fog computing. Future Generation Computer Systems. https://www.sciencedirect.com/ science/ article/pii/ S0167739X22003739. 141: 96-105, 2023. [DOI ]
[8] Khayyat, Mashael and Elgendy, Ibrahim A. and Muthanna, Ammar and Alshahrani, Abdullah S. and Alharbi, Soltan and Koucheryavy, Andrey. Advanced Deep Learning-Based Computational Offloading for Multilevel Vehicular Edge-Cloud Computing Networks. IEEE Access. 8(): 137052-137062, 2020. [DOI ]
[9] Lee, Seung-Seob and Lee, Sukyoung. Resource Allocation for Vehicular Fog Computing Using Reinforcement Learning Combined With Heuristic Information. IEEE Internet of Things Journal. 7(10): 10450-10464, 2020. [DOI ]
[10] Zhao, Junhui and Kong, Ming and Li, Qiuping and Sun, Xiaoke. Contract-Based Computing Resource Management via Deep Reinforcement Learning in Vehicular Fog Computing. IEEE Access. 8(): 3319-3329, 2020. [DOI ]
[11] Xianjing Wu and Shengjie Zhao and Hao Deng. Joint task assignment and resource allocation in VFC based on mobility prediction information. Computer Communications. https://www.sciencedirect.com/ science/ article/pii/S0140366423001172. 205: 24-34, 2023. [DOI ]
[12] Birhanie, Habtamu Mohammed and Messous, Mohammed Ayoub and Senouci, Sidi-Mohammed and Aglzim, El-Hassane and Ahmed, Ahmedin Mohammed. MDP-Based Resource Allocation Scheme Towards a Vehicular Fog Computing with Energy Constraints. 2018 IEEE Global Communications Conference (GLOBECOM). (): 1-6, 2018.
[13] Zhang, Yifan and Qin, Xiaoqi and Song, Xianxin. Mobility-Aware Cooperative Task Offloading and Resource Allocation in Vehicular Edge Computing. 2020 IEEE Wireless Communications and Networking Conference Workshops (WCNCW). (): 1-6, 2020. [DOI ]
[14] Liu, Lei and Feng, Jie and Mu, Xuanyu and Pei, Qingqi and Lan, Dapeng and Xiao, Ming. Asynchronous Deep Reinforcement Learning for Collaborative Task Computing and On-Demand Resource Allocation in Vehicular Edge Computing. IEEE Transactions on Intelligent Transportation Systems. 24(12): 15513-15526, 2023. [DOI ]
[15] Lee, Seung-Seob and Lee, Sukyoung. Poster Abstract: Deep Reinforcement Learning-based Resource Allocation in Vehicular Fog Computing. IEEE INFOCOM 2019 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). (): 1029-1030, 2019. [DOI ]
[16] Wu, Xianjing and Zhao, Shengjie and Zhang, Rongqing and Yang, Liuqing. Mobility Prediction-Based Joint Task Assignment and Resource Allocation in Vehicular Fog Computing. 2020 IEEE Wireless Communications and Networking Conference (WCNC). (): 1-6, 2020. [DOI ]
[17] Yan, Liangliang and Zhang, Min and Song, Chuang and Wang, Danshi and Li, Jin and Guan, Luyao. Deep Learning-Based Containerization Resource Management in Vehicular Fog Computing. 2019 Asia Communications and Photonics Conference (ACP). (): 1-3, 2019. [DOI ]
[18] Sutagundar, Ashok and Attar, Ameenabegum H and Patil, Basamma. Resource Allocation for Fog Enhanced Vehicular Services (FEVS). 2018 International Conference on Inventive Research in Computing Applications (ICIRCA). (): 360-365, 2018. [DOI ]
[19] Gu, Lin and Zeng, Deze and Guo, Song and Barnawi, Ahmed and Xiang, Yong. Cost Efficient Resource Management in Fog Computing Supported Medical Cyber-Physical System. IEEE Transactions on Emerging Topics in Computing. 5(1): 108-119, 2017. [DOI ]
[20] Liu, Tingting and Li, Jun and Shu, Feng and Han, Zhu. Optimal Task Allocation in Vehicular Fog Networks Requiring URLLC: An Energy-Aware Perspective. IEEE Transactions on Network Science and Engineering. 7(3): 1879-1890, 2020. [DOI ]
[21] Lin, Fuhong and Zhou, Yutong and Pau, Giovanni and Collotta, Mario. Optimization-Oriented Resource Allocation Management for Vehicular Fog Computing. IEEE Access. 6(): 69294-69303, 2018. [DOI ]
[22] Fatin Hamadah Rahman and S.H. Shah Newaz and Thien-Wan Au and Wida Susanty Suhaili and M.A. Parvez Mahmud and Gyu Myoung Lee. EnTruVe: ENergy and TRUst-aware Virtual Machine allocation in VEhicle fog computing for catering applications in 5G. Future Generation Computer Systems. https://www.sciencedirect.com/ science/ article/pii/S0167739X21002983. 126: 196-210, 2022. [DOI ]
[23] Chinmaya Kumar Dehury and Bharadwaj Veeravalli and Satish Narayana Srirama. HeRAFC: Heuristic resource allocation and optimization in MultiFog-Cloud environment. Journal of Parallel and Distributed Computing. https://www.sciencedirect.com/ science/ article/pii/S0743731523001302. 183: 104760, 2024. [DOI ]
[24] Liu, Chunhui and Liu, Kai and Ren, Hualing and Xu, Xincao and Xie, Ruitao and Cao, Jingjing. RtDS: real-time distributed strategy for multi-period task offloading in vehicular edge computing environment. Neural Comput. Appl.. https://doi.org/10.1007/s00521-021-05766-5. 35(17): 12373–12387, Springer-Verlag. 2021. [DOI ]
[25] Bushra Jamil and Humaira Ijaz and Mohammad Shojafar and Kashif Munir. IRATS: A DRL-based intelligent priority and deadline-aware online resource allocation and task scheduling algorithm in a vehicular fog network. Ad Hoc Networks. https://www.sciencedirect.com/ science/ article/pii/S1570870523000100. 141: 103090, 2023. [DOI ]
[26] Zhu, Hongbiao and Wu, Qiong and Wu, Xiao-Jun and Fan, Qiang and Fan, Pingyi and Wang, Jiangzhou. Decentralized Power Allocation for MIMO-NOMA Vehicular Edge Computing Based on Deep Reinforcement Learning. IEEE Internet of Things Journal. 9(14): 12770-12782, 2022. [DOI ]
[27] Xu, Lianming and Yang, Zexuan and Wu, Huaqing and Zhang, Yanru and Wang, Yanhui and Wang, Li and Han, Zhu. Socially Driven Joint Optimization of Communication, Caching, and Computing Resources in Vehicular Networks. IEEE Transactions on Wireless Communications. 21(1): 461-476, 2022. [DOI ]
[28] Zhou, Zhenyu and Liu, Pengju and Feng, Junhao and Zhang, Yan and Mumtaz, Shahid and Rodriguez, Jonathan. Computation Resource Allocation and Task Assignment Optimization in Vehicular Fog Computing: A Contract-Matching Approach. IEEE Transactions on Vehicular Technology. 68(4): 3113-3125, 2019. [DOI ]
[29] Dr. Amit Agarwal and Saloni Jain. Efficient Optimal Algorithm of Task Scheduling in Cloud Computing Environment. https://arxiv.org/abs/1404.2076. 2014.
Journal of Computing and Security
Volume 12, Issue 2 - Serial Number 2
September 2025
Pages 107-128

Files

History

  • Receive Date: 24 October 2025
  • Revise Date: 01 January 2026
  • Accept Date: 15 February 2026

Share

How to cite

Statistics