Classification methods for handwritten digit recognition: A survey

Keywords: handwritten digit recognition, image classification, support vector machine, deep neural networks, convolutional neural networks, hyperparameter optimization, swarm intelligence, MNIST

Abstract


Introduction/purpose: This paper provides a survey of handwritten digit recognition methods tested on the MNIST dataset.

Methods: The paper analyzes, synthesizes and compares the develop- ment of different classifiers applied to the handwritten digit recognition prob- lem, from linear classifiers to convolutional neural networks.

Results: Handwritten digit recognition classification accuracy tested on the MNIST dataset while using training and testing sets is now higher than 99.5% and the most successful method is a convolutional neural network.

Conclusions: Handwritten digit recognition is a problem with numerous real-life applications. Accurate recognition of various handwriting styles, specifically digits is a task studied for decades and this paper summa- rizes the achieved results. The best results have been achieved with convolutional neural networks while the worst methods are linear classifiers. The con- volutional neural networks give better results if the dataset is expended  with data augmentation.

 

References

Aider, M.A., Hammouche, K. & Gaceb, D. 2018. Recognition of handwritten characters based on wavelet transform and SVM classifier. The International Arab Journal of Information Technology, 15(6), pp. 1082–1087 [online]. Available at: https://iajit.org/portal/PDF/November%202018,%20No.%206/10880.pdf [Accessed: 1 March 2022].

Armstrong, S. 2019. Naive-bayesian-mnist. Github. [online]. Available at: https://github.com/sjnarmstrong/naive-bayesian-mnist [Accessed: 10 March 2022].

Babu, U.R., Chintha, A.K. & Venkateswarlu, Y. 2014. Handwritten digit recognition using structural, statistical features and k-nearest neighbor classifier. International Journal of Information Engineering and Electronic Business (IJIEEB), 6(1), pp. 62–68. Available at: https://doi.org/10.5815/ijieeb.2014.01.07

Bacanin, N., Bezdan, T., Tuba, E., Strumberger, I. & Tuba, M. 2020. Monarch Butterfly Optimization Based Convolutional Neural Network Design. Mathematics, 8(6, art.ID:936), pp. 1–32. Available at: https://doi.org/10.3390/math8060936

Bhagya Shree, S. & Sheshadri, H. 2018. Diagnosis of Alzheimer’s disease using Naive Bayesian Classifier. Neural Computing and Applications, 29, pp. 123– 132. Available at: https://doi.org/10.1007/s00521-016-2416-3

Ciresan, D.C., Meier, U., Gambardella, L.M. & Schmidhuber, J. 2010. Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition. Neural Computation, 22(12), pp. 3207–3220. Available at: https://doi.org/10.1162/NECO_a_00052

Cortes, C. & Vapnik, V. 1995. Support-vector networks. Machine Learning, 20(3), pp. 273–297. Available at: https://doi.org/10.1007/BF00994018

Das, N., Reddy, J.M., Sarkar, R., Basu, S., Kundu, M., Nasipuri, M. & Basu, D.K. 2012. A statistical–topological feature combination for recognition of handwritten numerals. Applied Soft Computing, 12(8), pp. 2486–2495. Available at: https://doi.org/10.1016/j.asoc.2012.03.039

Deotte, C. 2018. How to score 97%, 98%, 99%, and 100%. Kaggle. [online]. Available at: https://www.kaggle.com/c/digit-recognizer/discussion/61480 [Accessed: 10 March 2022].

Ebrahimzadeh, R. & Jampour, M. 2014. Efficient Handwritten Digit Recogni- tion based on Histogram of Oriented Gradients and SVM. International Journal of Computer Applications, 104(9), pp. 10–13. Available at: https://doi.org/10.5120/18229-9167

El qacimy, B., Ait kerroum, M. & Hammouch, A. 2014. Handwritten digit recognition based on DCT features and SVM classifier. In: 2014 Second World Conference on Complex Systems (WCCS). Agadir, Morocco, pp. 13–16, November 10-12. Available at: https://doi.org/10.1109/ICoCS.2014.7060935

Esmaili, I., Dabanloo, N.J. & Vali, M. 2016. Automatic classification of speech dysfluencies in continuous speech based on similarity measures and morphological image processing tools. Biomedical Signal Processing and Control, 23, pp. 104–114. Available at: https://doi.org/10.1016/j.bspc.2015.08.006

Gattal, A., Chibani, Y., Djeddi, C. & Siddiqi, I. 2014. Improving Isolated Digit Recognition Using a Combination of Multiple Features. In: 2014 14th International Conference on Frontiers in Handwriting Recognition. Hersonissos, Greece, pp. 446–451, September 01-04. Available at: https://doi.org/10.1109/ICFHR.2014.81

Grover, D. & Toghi, B. 2020. MNIST dataset classification utilizing k-NN classifier with modified sliding-window metric. In: Arai, K. & Kapoor, S. (Eds.) Advances in Computer Vision. CVC 2019. Advances in Intelligent Systems and Computing. 944, pp.583–591. Cham, Switzerland: Springer. Available at: par https://doi.org/10.1007/978-3-030-17798-0_47

Huang, H.Y. & Lin, C.J. 2016. Linear and Kernel Classification: When to Use Which? In: Proceedings of the 2016 SIAM International Conference on Data Mining (SDM). Miami, Florida, USA, pp. 216–224, May 5-7. Available at: https://doi.org/10.1137/1.9781611974348.25

Ilmi, N., Budi, W.T.A. & Nur, R.K. 2016. Handwriting digit recognition using local binary pattern variance and K-Nearest Neighbor classification. In: 2016 4th International Conference on Information and Communication Technology (ICoICT). Bandung, Indonesia, pp. 1–5, May 25-27. Available at: https://doi.org/10.1109/ICoICT.2016.7571937

Kang, C., Huo, Y., Xin, L., Tian, B. & Yu, B. 2019. Feature selection and tumor classification for microarray data using relaxed Lasso and generalized multi-class support vector machine. Journal of Theoretical Biology, 463, pp. 77–91. Available at: https://doi.org/10.1016/j.jtbi.2018.12.010

Lecun, Y., Bottou, L., Bengio, Y. & Haffner, P. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), pp. 2278–2324. Available at: https://doi.org/10.1109/5.726791

Li, Y., Xiao, J., Chen, Y. & Jiao, L. 2019. Evolving deep convolutional neural networks by quantum behaved particle swarm optimization with binary encoding for image classification. Neurocomputing, 362, pp. 156–165. Available at: https://doi.org/10.1016/j.neucom.2019.07.026

Lyu, H.K., Park, C.H., Han, D.H., Kwak, S.W. & Choi, B. 2018. Orchard Free Space and Center Line Estimation Using Naive Bayesian Classifier for Unmanned Ground Self-Driving Vehicle. Symmetry, 10(9, art.ID:355), pp. 1–14. Available at: https://doi.org/10.3390/sym10090355

Meier, U., Ciresan, D.C., Gambardella, L.M. & Schmidhuber, J. 2011. Better Digit Recognition with a Committee of Simple Neural Nets. In: 2011 International Conference on Document Analysis and Recognition. Beijing, China, pp.1250– 1254, September 18-21. Available at: https://doi.org/10.1109/ICDAR.2011.252

Nosseir, A. & Roshdy, R. 2018. Extraction of Egyptian License Plate Numbers and Characters Using SURF and Cross Correlation. In: ICSIE ’18: Proceedings of the 7th International Conference on Software and Information Engineering. Cairo, Egypt, pp.48–55, May 02-04. Available at: https://doi.org/10.1145/3220267.3220276

Patel, A. & Kalyani, T. 2016.   Support Vector Machine with Inverse Fringe as Feature for MNIST Dataset. In: 2016 IEEE 6th International Conference on Advanced Computing (IACC). Bhimavaram, India, pp.123–126, February 27-28. Available at: https://doi.org/10.1109/IACC.2016.32

Sethy, P.K., Barpanda, N.K., Rath, A.K. & Behera, S.K. 2020. Deep feature based rice leaf disease identification using support vector machine. Computers and Electronics in Agriculture, 175, art.number:105527. Available at: https://doi.org/10.1016/j.compag.2020.105527

Sivaram, M., Laxmi, L.E., Pustokhina, I.V., Pustokhin, D.A., Elhoseny, M., Joshi, G.P. & Shankar, K. 2020. An optimal least square support vector machine based earnings prediction of blockchain financial products. IEEE Access, 8, pp. 120321–120330. Available at: https://doi.org/10.1109/ACCESS.2020.3005808

Tuba, E., Bačanin, N., Strumberger, I. & Tuba, M. 2021. Convolutional Neural Networks Hyperparameters Tuning. In: Pap, E. (Eds.) Artificial Intelligence: Theory and Applications. Studies in Computational Intelligence. 973, pp.65–84. Cham, Switzerland: Springer. Available at: par https://doi.org/10.1007/978-3-030-72711-6_4

Tuba, E. & Tuba, I. 2021. Swarm Intelligence Algorithms for Convolutional Neural Networks. In: 2nd Workshop on Evolutionary and Population-based Optimization. Online Event, pp.1–6, November 30. Available at: https://wepo2021.aisylab.com/papers/wepo2021_paper_4.pdf [Accessed: 10 March 2022].

Tuba, E., Tuba, M. & Simian, D. 2016. Handwritten digit recognition by support vector machine optimized by Bat algorithm. In: WSCG ’2016: short commu nications proceedings: The 24th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2016 in cooperation with EUROGRAPHICS. Plzen, Czech Republic: University of West Bohemia, pp.369- 376, May 30-June 03. Available at: http://hdl.handle.net/11025/29725 [Accessed: 10 March 2022].

Wang, H., Zheng, B., Yoon, S.W. & Ko, H.S. 2018. A support vector machine-based ensemble algorithm for breast cancer diagnosis. European Journal of Operational Research, 267(2), pp. 687–699. Available at: https://doi.org/10.1016/j.ejor.2017.12.001

Wang, K. & Zhang, H. 2020. A Novel Naive Bayesian Approach to Inference with Applications to the MNIST Handwritten Digit Classification. In: 2020 International Conference on Computational Science and Computational Intelligence (CSCI). Las Vegas, NV, USA, pp.1354-1358, December 16-18. Available at: https://doi.org/10.1109/CSCI51800.2020.00252

Yuan, G.X., Ho, C.H. & Lin, C.J. 2012. Recent Advances of Large-Scale Linear Classification. Proceedings of the IEEE, 100(9), pp. 2584–2603. Available at: https://doi.org/10.1109/JPROC.2012.2188013

Zhai, Z., Xu, Z., Zhou, X., Wang, L. & Zhang, J. 2015. Recognition of hazard grade for cotton blind stinkbug based on Naive Bayesian classifier. Transactions of the Chinese Society of Agricultural Engineering, 31(1), pp. 204–211 [online]. Available at: http://www.tcsae.org/nygcxben/article/abstract/20150128 [Accessed: 1 March 2022].

Zhu, W., Yeh, W., Chen, J., Chen, D., Li, A. & Lin, Y. 2019. Evolutionary Convolutional Neural Networks Using ABC. In: ICMLC ’19: Proceedings of the 2019 11th International Conference on Machine Learning and Computing. Zhuhai, China, pp.156–162, February 22-24. Available at: https://doi.org/10.1145/3318299.3318301

 

Published
2023/01/30
Section
Review Papers