Information technology for recognition of handwritten ukrainian letters and numbers using synthetic data sets

DOI: 10.31673/2412-9070.2023.013237

Abstract

The paper considers several variants of the architecture of convolutional neural networks for recognizing isolated handwritten digits and Ukrainian letters. The results of recognizing different images containing letters and numbers were compared on models with different architectures. Several variants of rather complex architectures of neural networks were considered. Research was conducted with VGG16 and VGG19 architectures, ResNet or ResNetV2, MobileNet or MobileNetV2, DenseNet. The possibility of learning convolutional neural networks with the help of a synthetic data set built on the basis of handwritten or cursive fonts is shown. The size of the training data set significantly affects the reliability of character recognition. The data sets used in the work correspond in volume to the well-known EMNIST Letters dataset. The lower limit of the sample size, which provides acceptable recognition accuracy, was about 1500 characters per class. Reducing the sample by reducing the number of symbols per class leads to a significant decrease in recognition accuracy (from 90-100% accuracy of recognizing elements of real inscriptions to 40-60% with a 4-fold reduction in the sample size). Thus, when using deep neural networks to recognize letters or numbers, the reliability of recognizing elements of real inscriptions depended primarily on the size of the training data set. The accuracy of recognition of the test data set after training all variants of the models was quite high - 97-98% and higher. However, the generation of training data sets of small size — 300-500 images per class - practically did not provide reliable recognition. In general, when comparing the achieved recognition accuracy of real images and the model training speed, the best performance was provided by the DenseNet or ResNetV2 family models. Experiments with changing the optimization algorithm compared to Adam did not give any improvement in the accuracy and reliability of recognizing real samples. Increasing the number of model training epochs beyond the specified one also did not change the results.

Keywords: handwriting recognition; recognition of Ukrainian characters; convolutional neural networks (CNN); digit recognition; deep learning; image processing.

References
1. Image character recognition using deep convolutional neural network learned from different languages / Bai Jinfeng, Chen Zhineng, Feng, Bailan Xu Bo // 2014 IEEE International Conference on Image Processing, ICIP 2014. 2015. P. 2560–2564. URL: https://doi.org/10.1109/ICIP.2014.7025518.
2. Maitra D. S., Bhattacharya U., Parui S. K. CNN based common approach to handwritten character recognition of multiple scripts // 3th International Conference on Document Analysis and Recognition (ICDAR). 2015. P. 1021–1025. URL: https://doi.org/10.1109/ICDAR.2015.7333916.
3. EMNIST: Extending MNIST to handwritten letters / G. Cohen, S. Afshar, J. Tapson, A. Van Schaik // 2017 international joint conference on neural networks (IJCNN). 2017. P. 2921–2926. URL: https://doi.org/10.48550/arxiv.1702.05373.
4. A repository of images of hand-written Cyrillic and Latin alphabet letters for machine learning applications. URL: https://github.com/GregVial/CoMNIST
5. Bilgin Taşdemir E. F. Online Turkish Handwriting Recognition Using Synthetic Data // Avrupa Bilim ve Teknoloji Dergisi, Ejosat Special Issue 2021 (RDCONF). 2021. P. 649–656. URL: https://doi.org/10.31590/ejosat.1039846.
6. Handwritten Kazakh and Russian (HKR) database for text recognition / D. Nurseitov, K. Bostanbekov, D. Kurmankhojayev [et al.] // Multimed Tools Appl. 2021. v. 80. P. 33075–33097. URL: https://doi.org/10.1007/s11042-021-11399-6
7. Ullah Zahid, Jamjoom Mona. An intelligent approach for Arabic handwritten letter recognition using convolutional neural network // PeerJ Computer Science, 2022. v. 8. P. 995. URL: https://doi.org/10.7717/peerj-cs.995.
8. Baldominos A., Sáez Y., Isasi P. A Survey of Handwritten Character Recognition with MNIST and EMNIST // Applied Sciences. 2019. v. 3169.
URL: https://doi.org/10.3390/app9153169.
9. Handwritten indic character recognition using capsule networks / B. Mandal, S. Dubey, S. Ghosh [et al.] // 2018 IEEE Applied Signal Processing Conference (ASPCON). 2018. P. 304–308. URL:
https://arxiv.org/abs/1901.00166.
10. He K., Girshick R., Dollár P. Rethinking imagenet pre-training // Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019. P. 4918–4927. URL: https://arxiv.org/abs/1811.08883
11. Albattah Waleed, Albahli Saleh. Intelligent Arabic Handwriting Recognition Using Different Standalone and Hybrid CNN Architectures // Applied Sciences. 2022. v. 12. issue 10155. URL: https://doi.org/10.3390/app121910155.
12. Performance Analysis of State of the Art Convolutional Neural Network Architectures in Bangla Handwritten Character Recognition / Tapotosh Ghosh, Abedin, MHZ., Al Banna H. [et al.] // Pattern Recognit. Image Anal. 2021. V. 31. P. 60–71. URL: https://doi.org/10.1134/S1054661821010089.
13. Recognizing Arabic Handwritten Literal Amount Using Convolutional Neural Networks / Aicha Korichi, Slatnia Sihem, Tagougui Najiba [et al.] // 2022. URL: https://doi.org/10.1007/978-3-030-96311-8_15.
14. A new Arabic handwritten character recognition deep learning system (AHCR-DLS) / Balaha Hossam, Sabry Mohamed, Ali Hesham, Badawy Mahmoud // Neural Computing and Applications. 2021. v. 33. URL: https://doi.org/10.1007/s00521-020-05397-2.
15. Abo Samra, Gibrael Al Amin, Oqaibi Hadi. An Optimized Deep Residual Network with a Depth Concatenated Block for Handwritten Characters Classification // Computers, Materials & Continua. 2021. V. 680. P. 1–28. URL: https://doi.org/10.32604/cmc.2021.015318.
16. Classification of handwritten names of cities and handwritten text recognition using various deep learning models / D. Nurseitov, K. Bostanbekov, M. Kanatov [et al.] // 2021. URL: https://doi.org/10.25046%2Faj0505114.
17. Vovchuk O., Kyrychenko M. Recognition of Handwritten Cyrillic Letters using PCA // 2019. URL: https://www.researchgate.net/publication/336987544_Recognition_of_Handwritten_Cyrillic_Letters_using_PCA
18. Economic efficiency of innovative projects of CNN modified architecture application / V. Khavalko, V. Mykhailyshyn, R. Zhelizniak [et al.] // CEUR Workshop Proceedings. 2020. Vol. 2654: Proceedings of the International workshop on cyber hygiene (Cyb-Hyg-2019) co-located with 1st International conference on cyber hygiene and conflict management in global information networks (CyberConf 2019). Kyiv, Ukraine; November 30, 2019. P. 182–193. URL: https://ceur-ws.org/Vol-2654/paper14.pdf.
19. Simonyan K., Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition // CoRR abs/1409.1556. 2014. URL: https://doi.org/10.48550/arXiv.1409.1556.
20. Deep Residual Learning for Image Recognition / K. He, X. Zhang, S. Ren, J. Sun // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015. P. 770–778. URL: https://doi.org/10.1109/cvpr.2016.90.
21. Identity Mappings in Deep Residual Networks / K. He, X. Zhang, Shaoqing Ren, Jian Sun // European Conference on Computer Vision-2016, Springer. 2016. P. 630–645. URL: https://doi.org/10.1007/978-3-319-46493-0_38.
22. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications / Andrew G. Howard, Menglong Zhu, Bo Chen [et al.] // ArXivabs/1704.04861. 2017. URL: https://doi.org/10.48550/arXiv.1704.04861.
23. MobileNetV2: Inverted Residuals and Linear Bottlenecks / Mark Sandler, Andrew G. Howard, Menglong Zhu [et al.] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition/ 2018. P. 4510–4520. URL: https://doi.org/10.1109/CVPR.2018.00474.
24. Densely Connected Convolutional Networks / G. Huang, Z. Liu, L. Van Der Maaten, K. Q. Weinberger // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA. 2017. P. 2261–2269. URL: https://doi.org/10.1109/CVPR.2017.243.
25. Chychkarov Y., Zinchenko O. Handwritten Ukrainian Character Recognition using a Convolutional Neural Networks and Synthetic Dataset // MoMLeT+DS 2023: 5th International Workshop on Modern Machine Learning Technologies and Data Science, June 3, 2023, Lviv, Ukraine, 2023. P. 109–121. URL: https://ceur-ws.org/Vol-3426/paper9.pdf.