No Detail Found!

Deep Transfer Learning for Masked Face Reconstruction and Hybrid DCNN-ELM Framework for Recognition

Author(s)
Chandni Agarwal GLA University,Mathura, Uttar Pradesh , India
Anurag Mishra DeenDayal Upadhayay College, Delhi University,New Delhi ,India
Charul Bhatnagar GLA University,Mathura, Uttar Pradesh ,India
Abstract
Facial reconstruction has always been a pivotal aspect of medical and forensic science. The increasing use of face masks in recent years has posed new challenges, making traditional facial recognition techniques less effective. To address this, our research explored innovative methods for reconstructing faces from images obscured by masks. We focused on post mask face reconstruction and facial recognition using cutting-edge techniques. We assess the effectiveness of three key unmasking algorithms: edgeconnect (EC), gated convolution (GC), and hierarchical variational vector quantized autoencoders (HVQVAE). Using two synthetic face datasets, MaskedFace-CelebA and MaskedFace-CelebAHQ, we rigorously evaluate the quality of the reconstructed faces based on metrics such as the PSNR, SSIM, UIQI, and NCORR. Among these, the Gated Convolution algorithm stands out as the superior choice in terms of image quality. For facial recognition, we employ a novel hybrid framework that combines a deep convolutional neural network and an extreme learning machine (DCNN-ELM). We tested five classifiers (Vgg16, Vgg19, ResNet50, ResNet101, and ResNet152) in combination with ELM and a support vector machine (SVM). Our comprehensive ablation study revealed that ResNet152 combined with ELM achieved the best performance, with a facial recognition accuracy of 60.9%, suggesting that the reconstructed faces were of high quality. Our paper presents a novel approach to image classification utilizing five classifiers within the DCNN+ELM hybrid framework and provides a complete ablation study of these classifiers. This research underscores the importance of face reconstruction in the current field and its potential to enhance facial recognition techniques.
Publication Details
Page(s) 170-192
DOI DOI not available
Published Journal: International Journal of Communication Networks and Information Security, Volume: 16, Issue: 4, Year: 2024

Request the full text from PASTIC

Keywords
Deep learning, face recognition, Image Inpainting, Face Reconstruction, Autoencoders, GAN
References
Estudillo Alejandro,Hoo Keat. Alejandro .2021 .The effect of face masks on forensic face matching: An individual differences study. Journal of Applied Research in Memory and Cognition, 10 : 554-563.
Pathak D.,Krahenbuhl P.,Donahue J.,Darrell T.,Efros A. A. .2016 .Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on computer vision and pattern recognition, : 2536-2544.
Jiang Y.,Xu J.,Yang B.,Xu J.,Zhu J. .2020 ."Image Inpainting Based on Generative Adversarial Networks,". in IEEE Access, 2970169 : 22884-22892.
Pathak Deepak,Alexei A.,Efros A. .2016 .Context encoders: Feature learning by inpainting. In CVPR, : 2536-2544.
Liu Z.,Luo P.,Wang X.,Tang X. .2015 .Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, : 3730-3738.
Thung K.-H.,Raveendran P. .2009 .A survey of image quality measures. doi:10, : .
Zhao Lei,Mo Qihang,Wang Qihang .2020 .Diverse image inpainting based on unsupervised crossspace translation. In CVPR, : 5741-5750.
Liu Hongyu,Yang Chao .2020 .Rethinking image inpainting via a mutual encoderdecoder with feature equalizations. In ECCV, : 725-741.
Bertalmio Marcelo,Ballester Coloma .2000 .Image inpainting. In ACM SIGGRAPH, : 417-424.
Qin Z.,Zeng Q.,Zong Y.,Xu F. .2021 .Image inpainting based on deep learning: A review. Displays, 69 : 102028.
Nazeri K.,Ng E.,Joseph T.,Qureshi F. Z.,Ebrahimi M. .1901 .EdgeConnect: Generative image inpainting with adversarial edge learning. , : .
Anwar A.,Raychowdhury A. .2008 .Masked face recognition for secure authentication. , : .
Huang G. B.,Zhu Q. Y.,Siew C. K. .2004 .Extreme learning machine: a new learning scheme of feedforward neural networks. In 2004 IEEE international joint conference on neural networks (IEEE Cat. No. 04CH37541), 2 : 985-990.
Karras .2018 ."Progressive Growing of GANs for Improved Quality, Stability, and Variation". in Internation Conference on Reoresentation Learning (ICLR), : .
Yu J.,Lin Z.,Yang J.,Shen X.,Lu X.,Huang T. S. .1806 .Free-form image inpainting with gated convolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision, : 4471-4480.
Peng J.,Liu D.,Xu S.,Li H. .2021 .Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAE. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, : 10775-10784.
Aaron van den Oord,Kavukcuoglu Koray .2017 .Neural discrete representation learning. In NIPS, : 6306-6315.
Xi Chen .2018 .PixelSNAIL: An improved autoregressive generative model. In ICML, : 864-872.
Tim Salimans,Kingma Diederik P. .2017 .: Improving the PixelCNN with discretized logistic mixture likelihood and other modifications. arXiv preprint arXiv:1701.05517, : .
Aaron van den Oord,Kavukcuoglu Koray .2016 .Conditional image generation with PixelCNN decoders. In NIPS, : 4790-4798.
Ali Razavi,Vinyals Oriol .2019 .Generating diverse high-fidelity images with VQ-VAE-2. , : 14866-14876.
Yoshua Bengio .2013 .Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, : .
Yang C.,Lu X.,Lin Z.,Shechtman E.,Wang O.,Li H. .2017 .High-resolution image inpainting using multiscale neural patch synthesis. In Proceedings of the IEEE conference on computer vision and pattern recognition, : 6721-6729.
Iizuka S.,Simo-Serra E.,Ishikawa H. .2017 .Globally and locally consistent image completion. ACM Transactions on Graphics (ToG), 36(4) : 1-14.
Zhang K.,Zhang Z.,Li Z.,Qiao Y. .2016 .Joint face detection and alignment using multitask cascaded convolutional networks. IEEE signal processing letters, 23(10) : 1499-1503.
Lugaresi C.,Tang J.,Nash H.,McClanahan C.,Uboweja E.,Hays M.,Grundmann M. .1906 .Mediapipe: A framework for building perception pipelines. , : .
Chuanxia Zheng,Tat-Jen Cham Jianfei,Cai Jianfei .2019 .Pluralistic image completion. In CVPR, : 1438-1447.
Huang G. B.,Zhu Q. Y.,Siew C. K. .2006 .. Extreme Learning Machine: Theory and Applications, 70 : 489-501.
Huang G. B.,Zhu Q. Y.,Siew C. K. .2006 .Real - Time Learning Capability of Neural Networks. IEEE Transactions on Neural Networks, 4 : 863-878.
Agarwal C.,Bhatnagar C. .2023 .Unmasking the potential: evaluating image inpainting techniques for masked face reconstruction. Multimedia Tools and Applications, : 1-26.
Agarwal C.,Itondia P.,Mishra A. .2023 .A novel DCNN -ELM hybrid framework for face mask detection. Intelligent Systems with Applications, 17 : 200175.
Huang G. B. .2004 .The MATLAB code for ELM. , : .
Yu J.,Lin Z.,Yang J.,Shen X.,Lu X.,Huang T. S. .1801 .Generative image inpainting with contextual attention. In Proceedings of the IEEE conference on computer vision and pattern recognition, : 5505-5514.
Wang Z.,Bovik A. C. .2002 .A universal image quality index. IEEE signal processing letters, 9(3) : 81-84.
Ud Nizam,Din Nizam,Yi Juneho .2020 .Effective removal of user-selected foreground object from facial images using a novel GAN -based network. IEEE Access, 8 : 109648-109661.
Ud Nizam,Din Nizam,Yi Juneho .2020 .A novel GAN-based network for unmasking of masked face. IEEE Access, 8 : 44276-44287.
Fadi Boutros,Kuijper Arjan .2021 .Unmasking face embeddings by self-restrained triplet loss for accurate masked face recognition. arXiv preprint arXiv:2103.01716, : .
Takeru Miyato .2018 .Spectral Normalization for Generative Adversarial Networks. ” Proc. International Conference on Learning Representations (ICLR), : .
Phillip Isola,Jun-Yan Zhu,Tinghui Zhou Zhu,Alexei A.,Efros A. .2017 .Image-to-Image Translation with Conditional Adversarial Networks. ” Proc. Computer Vision and Pattern Recognition (CVPR), : 21-26.
Huang G. B.,Zhu Q. Y.,Siew C. K. .2004 .Extreme learning machine: a new learning scheme of feedforward neural networks. In 2004 IEEE international joint conference on neural networks (IEEE Cat. No. 04CH37541), 2 : 985-990.
Agarwal C.,Mishra A.,Sharma A.,Chetty G. .2014 .A novel scene based robust video watermarking scheme in DWT domain using extreme learning machine. Extreme Learning Machines 2013: Algorithms and Applications, 209 : 225.
Cortes C.,Vapnik V. .1995 .Support-vector networks. Machine learning, 20 : 273-297.
0

Citations

0

Downloads

67

Views