Identifying the potential for applying neural network models in the context of natural language processing

DOI: 10.31673/2412-9070.2025.050517

Abstract

Today, mental health remains the number one issue. According to the latest 2024 report on the state of healthcare, 45% of respondents consider mental health to be one of the main healthcare problems facing their country. Cancer ranks second (38%), and stress ranks third (31%) in 31 countries [1].
This article focuses on determining the potential of natural language processing models for analyzing the diagnosis of psychological disorders. Three models were selected for analysis and research: a support vector machine classifier, a logistic regression model, and a DistilBERT transformer model. A dataset was created from open sources Reddit Mental Health Dataset and data was selected from Kaggle for the neutral data class without pronounced markers of psychological disorders.
First, an analysis of the general differences between algorithms and model operating principles was performed. Then, testing was performed on the same volume of pre-processed data. Based on the results of the research and comparisons, conclusions were drawn and it was determined that the transformer model, on a relatively small amount of data, still shows better results than the usual classical models. The classical models also showed good results, but when the amount of data is increased, the results rapidly deteriorate, while the transformer model improves.

Keywords: neural network models; logistic regression; support vector model; transformer models; psychological diagnosis; psychological disorders; natural language processing.