Reinforcement Learning based Malware mitigation in balanced Multicontroller Software Defined Networking enabled Internet of Things Networks

Abstract

The rapid growth of Internet of Things (IoT) ecosystems has led to an increase in sophisticated malware threats that require advanced security solutions. This research proposes a novel reinforcement learning framework that combines Q-learning for dynamic load balancing and REINFORCE for malware classification and mitigation in software-defined networking (SDN)-enabled IoT networks and has been applied innovatively. Q-learning optimizes resource allocation between controllers, achieving a high load balance ratio (0.984), low imbalance (0.167), minimal packet loss (1.2%), low latency (0.0125 s), and low jitter (0.0079 s). The REINFORCE algorithm enables probabilistic malware classification with an overall accuracy of 0.93 and assesses device criticality to apply targeted countermeasures. All devices were mitigated through an epidemic modeling approach based on the susceptible-infected-recovered (SIR) model, which ensures comprehensive threat containment through controlled state transitions. Experimental results demonstrate the framework’s ability to maintain network stability and responsiveness while neutralizing malware threats. Malware detection reliability was analyzed using average confidence scores and class-wise concept drift analysis, which enhances robustness by ensuring stable detection of static threats while adapting to evolving malware behaviors. The integration of reinforcement learning with SDN and epidemic modeling allows IoT systems to strengthen their cybersecurity defenses without compromising optimal network performance.