Publication: The effectiveness of a sentence completion test for depression screening using large language models
1
0
Issued Date
2025-09-01
Resource Type
ISSN
00016918
eISSN
18736297
Scopus ID
2-s2.0-105013662119
Journal Title
Acta Psychologica
Volume
259
Rights Holder(s)
SCOPUS
Bibliographic Citation
Acta Psychologica Vol.259 (2025)
Suggested Citation
Porkaew P., Zhu T., Li A., Chuenphitthayavut K. The effectiveness of a sentence completion test for depression screening using large language models. Acta Psychologica Vol.259 (2025). doi:10.1016/j.actpsy.2025.105425 Retrieved from: https://hdl.handle.net/20.500.14740/50354
Author(s)
Corresponding Author(s)
Other Contributor(s)
Abstract
Depressive symptoms pose a significant mental health challenge globally, including in Thailand. The aim of this study was to assess the effectiveness of a sentence completion test for depression using large language models (LLMs). To improve objectivity and reduce bias in assessments, this study detects and classifies the trends in depression, modernizing the screening process with a newly developed depression sentence completion test. This research examines the four key areas: 1) family, 2) society, 3) health, and 4) self-concept among 373 participants, aged 20 to 40. Additionally, four models were applied to test: 1) LLAMA 3.1-8B, 2) Gemma2-9B, 3) Qwen2-7B, and 4) Typhoon1.5-7B. The result revealed that health, self-concept, and DIFF (Difference) were strongly related to sentiment levels with strong positive values of 0.48, 0.49, and 0.54, respectively, which might mean that they are significant indicators of depression risk. Family and society had positive but lower values of 0.27 and 0.19, respectively. Statistical validation confirms model reliability with a 0.78 lower bound accuracy (p ≤ .05). In the evaluation of all Thai-compatible LLMs, random forest models consistently performed better than decision tree classifiers in the classification of depression risk. LLaMA3.1 and Gemma2 produced the highest sensitivity. Ethical problems have to be considered when using LLMs in mental health. Embedding diverse populations and dynamic updating to sample data in future studies will assure greater accuracy and generalization across different demographic groups.