Detecting and guarding against kernel backdoors through packet flow differentials

In this paper, we present a novel technique to detect and defeat kernel backdoors which cannot be identified by conventional security solutions. We focus on the fact that since the packet flows of common network applications go up and down through the whole network subsystem but kernel backdoors utilize only the lower layers of the subsystem, we can detect kernel backdoors by employing two host-based monitoring sensors (one at higher layer and the other at lower layer) and by inspecting the packet flow differentials. We also provide strategies to mitigate false positives and negatives and to defeat kernel backdoors. To evaluate the effectiveness of the proposed technique, we implemented a detection system (KbGuard) and performed experiments in a simulated environment. The evaluation results indicate that our approach can effectively detect and deactivate kernel backdoors with a high detection rate. We also believe that our research can help prevent stealthy threats of kernel backdoors.