Intelligent Cross-Layer Routing Using Trust-Integrated Multi-Agent Actor–Critic Reinforcement Learning for Hybrid IoT Systems
DOI:
https://doi.org/10.31838/NJAP/08.02.13Keywords:
IoT Routing, Multi-Agent Reinforcement Learning (MARL), Actor–Critic, Trust Management, Cross-Layer Optimization, 5G NR-Redcap, Massive IoTAbstract
The Internet of Things (IoT) is undergoing explosive growth, requiring routing mechanisms that are energy-efficient, secure, and scalable. This need is further emphasized by the upcoming massive deployments of heterogeneous networks, such as LoRaWAN and 5G NR-RedCap (5G New Radio Reduced Capability). The existing routing protocols are designed to optimize energy (e.g., LoRaWAN ADR, LEACH), improve security (e.g., Trust-RPL, blockchain-based routing), or achieve adaptive control (e.g., SDN, reinforcement learning); however, they fail to address all three requirements in hybrid networks. This paper presents a Trust-Integrated Actor-Critic Multi-Agent Reinforcement Learning (MARL) framework with Cross-Layer Optimization for a hybrid LoRa/NR-RedCap IoT network. Each IoT device is modelled as an intelligent agent that makes decisions on forwarding, channel, and transmit power based on local information such as residual energy, link quality, trust value, duty-cycle budget, and queue size, as well as cross-layer information including PHY/MAC scheduling and application traffic type. The trust component aims to detect malicious nodes and defend against attacks like blackhole, wormhole, and selective forwarding attacks, while the actor-critic part ensures policy convergence to support incremental learning. The Centralized Training and Decentralized Execution (CTDE) approach is used to support seamless scalability for networks with tens of thousands of nodes.Simulation results on a large-scale show that the proposed scheme provides an improvement of up to 30% and at least 40% improvement in packet delivery ratio and latency, respectively, over LoRaWAN ADR, Trust-RPL/SecRPL, blockchain-based routing, and SDN-based IoT routing. Moreover, the proposed scheme provides 35-40% extended network lifetime and 60-70% faster attack recovery time. These results confirm that MARL with trust mechanisms is a capable approach for next-generation secure, energyefficient IoT routing in large-scale hybrid networks.
References
1. A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, and M. Ayyash, “Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications,” IEEE Commun. Surveys Tuts., vol. 17, no. 4, pp. 2347–2376, 2015.
2. M. Centenaro, L. Vangelista, A. Zanella, and M. Zorzi, “Long-range communications in unlicensed bands: The rising stars in the IoT and smart city scenarios,” IEEE Wireless Commun., vol. 23, no. 5, pp. 60–67, 2016.
3. Y. Liu, L. Yan, and Y. Chen, “Reduced Capability (RedCap) NR Devices in 5G: System Design and Performance,” IEEE Commun. Standards Mag., vol. 6, no. 3, pp. 54–61, 2022.
4. K. Mekki, E. Bajic, F. Chaxel, and F. Meyer, “A comparative study of LPWAN technologies for large-scale IoT deployment,” ICT Express, vol. 5, no. 1, pp. 1–7, 2019.
5. Z. Zhang, Y. Xiao, Z. Ma, and M. Xiao, “6G and Semantic Communications: A New Paradigm for Massive IoT,” IEEE Internet Things J., 2023.
6. C. Perkins, E. Belding-Royer, and S. Das, “Ad hoc On-Demand Distance Vector (AODV) Routing,” RFC 3561, 2003.
7. D. Johnson, Y. Hu, and D. Maltz, “The Dynamic Source Routing Protocol (DSR) for Mobile Ad Hoc Networks,” IETF RFC 4728, 2007.
8. T. Winter, P. Thubert, and R. Kelsey, “RPL: IPv6 Routing Protocol for Low-Power and Lossy Networks,” RFC 6550, 2012.
9. P. Levis, N. Patel, D. Culler, and S. Shenker, “Trickle: A self-regulating algorithm for code propagation and maintenance in wireless sensor networks,” in Proc. NSDI, 2004, pp. 15–28.
10. J. Ko, A. Terzis, and S. Dawson-Haggerty, “Connecting Low-Power and Lossy Networks to the Internet,” IEEE Commun. Mag., vol. 49, no. 4, pp. 96–101, 2011.
11. W. Heinzelman, A. Chandrakasan, and H. Balakrishnan, “Energy-efficient communication protocol for wireless microsensor networks,” in Proc. HICSS, 2000, pp. 8020–8029.
12. S. Lindsey and C. Raghavendra, “PEGASIS: Power-efficient gathering in sensor information systems,” in Proc. IEEE Aerospace Conf., 2002, pp. 1125–1130.
13. A. Augustin, J. Yi, T. Clausen, and W. Townsley, “A study of LoRa: Long range & low power networks for the internet of things,” Sensors, vol. 16, no. 9, p. 1466, 2016.
14. B. Reynders and S. Pollin, “Range and coexistence analysis of long range unlicensed communication,” IEEE Internet Things J., vol. 5, no. 4, pp. 2241–2253, 2018.
15. K. Mekki, A. Rachedi, and M. Al-Debagy, “Efficient adaptive data rate management in LoRaWAN networks,” IEEE Access, vol. 9, pp. 77563–77574, 2021.
16. D. Airehrour, J. Gutierrez, and S. Ray, “Trust-based secure routing in RPL for the Internet of Things,” IEEE Sensors J., vol. 16, no. 8, pp. 5848–5858, 2016.
17. L. Wallgren, S. Raza, and T. Voigt, “SecRPL: A Secure Routing Protocol for the Internet of Things,” in Proc. WiMob, 2013, pp. 604–611.
18. L. Nkenyereye, J. Lee, and S. Pack, “A survey on RPL enhancements: A focus on topology, security, and mobility,” J. Netw. Comput. Appl., vol. 144, pp. 152–170, 2019.
19. I. Butun, P. Osterberg, and H. Song, “Security of the Internet of Things: Vulnerabilities, Attacks, and Countermeasures,” IEEE Commun. Surveys Tuts., vol. 22, no. 1, pp. 616–644, 2020.
20. H. Kim and Y. Ko, “Trust evaluation model for RPL-based IoT networks,” IEEE Access, vol. 7, pp. 18165–18177, 2019.
21. A. Dorri, S. Kanhere, R. Jurdak, and P. Gauravaram, “Blockchain for IoT security and privacy: The case study of a smart home,” IEEE Internet Things J., vol. 4, no. 6, pp. 2292–2303, 2017.
22. M. Ferrag, L. Shu, X. Yang, A. Derhab, and L. Maglaras, “Blockchain technologies for the Internet of Things: Research issues and challenges,” IEEE Internet Things J., vol. 6, no. 2, pp. 2188–2204, 2019.
23. K. Christidis and M. Devetsikiotis, “Blockchains and smart contracts for the Internet of Things,” IEEE Access, vol. 4, pp. 2292–2303, 2016.
24. D. Kreutz, F. Ramos, P. Verissimo, C. Rothenberg, S. Azodolmolky, and S. Uhlig, “Software-defined networking: A comprehensive survey,” Proc. IEEE, vol. 103, no. 1, pp. 14–76, 2015.
25. Z. Qin, G. Denker, C. Giannelli, P. Bellavista, and N. Venkatasubramanian, “Software defined networking for wireless and mobile networks: A survey,” IEEE Commun. Surveys Tuts., vol. 17, no. 3, pp. 1197–1219, 2015.
26. P. Lin, J. Bi, J. Wu, and A. Xu, “Toward scalable and intelligent IoT networks: A survey of SDN-based solutions,” IEEE Commun. Surveys Tuts., vol. 23, no. 1, pp. 134–180, 2021.
27. R. Sutton and A. Barto, Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA: MIT Press, 2018.
28. Y. Li, X. Li, X. Xu, L. Wu, and J. Chen, “Reinforcement learning for routing in wireless IoT: A survey,” IEEE Commun. Surveys Tuts., vol. 23, no. 3, pp. 1577–1612, 2021.
29. D. Ye, G. Chen, and J. Cao, “A survey of multi-agent reinforcement learning with communication,” ACM Comput. Surveys, vol. 55, no. 6, pp. 1–49, 2022.
30. C. Zhang, Y. Yang, H. Xu, and Z. Wang, “Multi-agent reinforcement learning: A selective overview of theories and algorithms,” in Handbook of Reinforcement Learning and Control. Springer, 2021, pp. 321–346.
