Hybrid Deep Learning and Local Binary Pattern Model for Masked Face Recognition

Abstract

Robust face recognition under partial occlusion remains one of the most persistent challenges in computer vision. The COVID-19 pandemic exposed the vulnerability of traditional facial recognition systems to occlusions such as masks, but the problem continues to affect modern biometric applications in healthcare, security, and public environments. This paper introduces a hybrid approach that enhances masked face recognition by combining deep learning features with handcrafted texture descriptors. The proposed method extracts deep features using three pre-trained Convolutional Neural Networks (CNNs)—VGG-16, MobileNet-V2, and Xception—and complements them with Local Binary Pattern (LBP) features to capture fine-grained texture information. Both feature types are concatenated and fed into a Multilayer Perceptron (MLP) classifier for final recognition. Image enhancement and extensive data augmentation are applied to improve robustness and generalization. Experimental results on the Simulated Masked Face Dataset demonstrate that our hybrid CNN–LBP–MLP framework achieves superior accuracy compared to state-of-the-art methods, while maintaining computational efficiency. This study highlights the continued importance of developing reliable face recognition systems resilient to real-world occlusions beyond the pandemic era.