Content-Based Image Retrieval Established on Deep and Handcrafted Features

Faiza Imdad; Sana Ul Haq; Imtiaz Rasool; Muhammad Wasimuddin; Mohammad Omer Farooq

Author(s)

Faiza Imdad Department of Electronics, University of Peshawar,Peshawar 25120,Pakistan

Sana Ul Haq Department of Electronics, University of Peshawar,Peshawar 25120,Pakistan

Imtiaz Rasool Department of Electronics, University of Peshawar,Peshawar 25120,Pakistan

Muhammad Wasimuddin Department of Electronics, University of Peshawar,Peshawar 25120,Pakistan

Mohammad Omer Farooq Department of Electronics, University of Peshawar,Peshawar 25120,Pakistan

Abstract

The role of image features is crucial in any system. There is a need to enhance Content-Based Image Retrieval (CBIR) systems, but challenges exist in accurately classifying, retrieving, and browsing or mining images. These challenges can be effectively addressed through the extraction of meaningful visual features. Various handcrafted and Deep Learning (DL) techniques have been developed for this purpose, but there has been limited exploration of combining the two approaches. The proposed technique is based on joint use of handcrafted features, i.e., Histogram of Oriented Gradients (HOG), Speed-Up Robust Features (SURF), Bag of Features (BoF), and Local Binary Pattern (LBP), and deep features extracted through AlexNet plus Spatial Pyramid Pooling (SPP) model. The Support Vector Machine (SVM) classifier was used for the classification. The algorithm was evaluated using the Caltech-256 RGB image dataset, achieving an average accuracy of 86.8%. The outcomes demonstrated the benefits of combining handcrafted and DL features, leading to improved accuracy in specific CBIR scenarios.

Publication Details

Page(s) 1-12

DOI DOI not available

Published Journal: The Sciencetech, Volume: 5, Issue: 2, Year: 2024

Request the full text from PASTIC

Keywords

Image Retrieval, Handcrafted Features, combined features, Classification, Deep Features

References

Ahmed A. .2021 .Pre-trained CNNs models for content based image retrieval. International Journal of Advanced Computer Science and Applications, 12(7) : 200-206.

Anavi Y.,Kogan I.,Gelbart E.,Geva O.,Greenspan H. .2015 .A comparative study for chest radiograph image retrieval using binary texture and deep learning classification. Proceedings of international conference of the IEEE engineering in medicine and biology society (EMBC), : 2940-2943.

Babenko A.,Lempitsky V. .2015 .Aggregating deep convolutional features for image retrieval. arXiv preprint arXiv:1510, : 07493.

Babenko A.,Slesarev A.,Chigorin A.,Lempitsky V. .2014 .Neural codes for image retrieval. Proceedings of European Conference on Computer Vision, : 584-599.

Chicco D.,Warrens M. J.,Jurman G. .2021 .The Matthews Correlation Coefficient (MCC) is More Informative Than Cohen's Kappa and Brier Score in Binary Classification Assessment. IEEE Access, 9(1) : 78368-78381.

Clough P.,Müller H.,Deselaers T.,Grubinger M.,Lehmann T. M.,Jensen J. .2006 .The CLEF 2005 Cross-Language Image Retrieval Track. Proceedings of Workshop of the Cross-Language Evaluation Forum for European Languages, : 535-557.

Dubey S. R. .2021 .A decade survey of content based image retrieval using deep learning. IEEE Transactions on Circuits and Systems for Video Technology, 32(5) : 2687-2704.

Everingham M.,Eslami S. A.,Van Gool L.,Williams C. K.,Winn J.,Zisserman A. .2015 .The pascal visual object classes challenge: A retrospective. International journal of computer vision, 111 : 98-136.

Griffin G.,Holub A.,Perona P. .2022 .. Caltech, 256(1) : .

Hameed I. M.,Abdulhussain S. H.,Mahmmod B. M. .2021 .Contentbased image retrieval: A review of recent trends. Cogent Engineering, 8(1) : 1927469.

Krizhevsky A.,Sutskever I.,Hinton G. E. .2012 .Imagenet classification with deep convolutional neural networks. Communications of ACM, 60 : 84-90.

Kulkarni S.,Manu T. M. .2022 .Content based image retrieval: A literature review. International Conference on Intelligent Computing and Control Systems (ICICCS), : 1580-1587.

Lacoste C.,Lim J.-H.,Chevallet J.-P.,Le D. H. .2007 .Medicalimage retrieval based on knowledge-assisted text and image indexing. IEEE Transactions on Circuits and Systems for Video Technology, 17(7) : 889-900.

Latif A.,Rasheed A.,Sajid U.,Ahmed J.,Ali N.,Ratyal N. I. .2019 .Content-based image retrieval and feature extraction: a comprehensive review. , : 1-21.

Li F.-F.,Andreeto M.,Ranzato M.,Perona P. .2022 .. Caltech, 101(1) : .

Lowe D. G. .2004 .Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60 : 91-110.

Mao J.,Xu W.,Yang Y.,Wang J.,Yuille A. L. .2014 .Explain images with multimodal recurrent neural networks. arXiv preprint arXiv:1410, : 1090.

Opelt A.,Pinz A. .2005 .Object localization with boosting and weak supervision for generic object. Proceedings of Scandinavian Conference on Image Analysis, : 862-871.

Pham ,M.-T. .2018 .Efficient texture retrieval using multiscale local extrema descriptors and covariance embedding. Proceedings of the European Conference on Computer Vision, : 564-579.

Platt J. C. .1999 .Fast training of support vector machines using sequential minimal optimization. Advances in kernel methods, : 185-208.

Qayyum A.,Anwar S. M.,Awais M.,Majid M. .2017 .Medical image retrieval using deep convolutional neural network. Neurocomputing, 266 : 8-20.

Razavian A. S.,Sullivan J.,Carlsson S.,Maki A. .2016 .Visual instance retrieval with deep convolutional networks. ITE Transactions on Media Technology and Applications, 4(3) : 251-258.

Russell B. C.,Torralba A.,Murphy K. P.,Freeman W. T. .2008 .LabelMe: a database and web-based tool for image annotation. International journal of computer vision, 77 : 157-173.

Sarwar A.,Mehmood Z.,Saba T.,Qazi K. A.,Adnan A.,Jamal H. .2019 .A novel method for content-based image retrieval to improve the effectiveness of the bag-of-words model using a support vector machine. Journal of Information Science, 45(1) : 117-135.

Sukanya C. M.,Gokul R.,Paul V. .2016 .A survey on object recognition methods. International Journal of Science, 6(1) : 48-52.

Taheri F.,Rahbar K.,Beheshtifard Z. .2023 .Content-based image retrieval using handcraft feature fusion in semantic pyramid. International Journal of Multimedia Information Retrieval, 12(2) : 21.

Wan J.,Wang D.,Hoi S. C.,Wu P.,Zhu J.,Y. J. .2014 .Deep learning for content-based image retrieval: A comprehensive study. Proceedings of the ACM international conference on Multimedia, : 157-166.

Xie L.,Hong R.,Tian B.,Q. B. .2015 .Image classification and retrieval are one. Proceedings of ACM International Conference on Multimedia Retrieval, : 3-10.

Zhang J.,Xia Y.,Xie Y.,Fulham M.,Feng D. D. .2017 .Classification of medical images in the biomedical literature by jointly using deep and handcrafted visual features. IEEE journal of biomedical and health informatics, 22(5) : 1521-1530.

Zhang R.,Lin L.,Zhang L.,Zuo W.,Zhang L. .2015 .Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Transactions on Image Processing, 24(12) : 4766-4779.

Zhou W.,Li H.,Tian Q. .2017 .Recent advance in content-based image retrieval: A literature survey. arXiv preprint arXiv:1706, : 06064.

Zhou W.,Newsam S.,Li C.,Shao Z. .2018 .PatternNet: A benchmark dataset for performance evaluation of remote sensing image retrieval. ISPRS Journal of Photogrammetry and Remote Sensing, 145 : 197-209.