Comparative Study of Ensemble and Neural Models for Insider Threat Detection Under Class Imbalance (Published)
Insider threats remain one of the most difficult cybersecurity risks to detect because malicious activities often originate from legitimate users operating within authorised boundaries. Machine learning techniques have increasingly been applied to insider threat detection; however, there is limited empirical evidence comparing the effectiveness of classical machine learning models and deep learning architectures on large-scale behavioural datasets under realistic class imbalance conditions. This study presents a comparative performance evaluation of ensemble machine learning and neural deep learning models for insider threat detection using a large publicly available behavioural risk dataset comprising 299,880 employee activity records. After rigorous preprocessing, feature engineering, and class balancing through controlled undersampling, four models were evaluated: Random Forest, Extreme Gradient Boosting (XGBoost), Multi-Layer Perceptron (MLP), and an Autoencoder-enhanced MLP (AE-MLP). Experimental results show that ensemble tree-based methods outperform deep neural models on tabular behavioural data, with XGBoost achieving the best overall performance (Accuracy 0.894, F1-score 0.895, ROC-AUC 0.969). Deep learning models demonstrated competitive precision but lower recall, indicating reduced sensitivity to malicious behaviour patterns. To validate model behaviour, SHAP-based global feature importance analysis was applied to the best-performing model, confirming that predictions relied on meaningful behavioural indicators, including data transfer activity, printing behaviour, access timing, and employee role characteristics. The findings suggest that for structured insider threat datasets, optimised classical ensemble models remain more effective and computationally efficient than deep neural approaches, while lightweight explainability methods can provide useful behavioural validation without heavy interpretability overhead.
Keywords: Comparative study, Ensemble, insider threat detection under class imbalance, neural models