Assessing User Trust in LLM-based Mental Health Applications: Perceptions of Reliability and Effectiveness

Authors

  • Yuxin Zhao New York University
  • Jiawei Wu Illinois Institute of Technology
  • Ping Qu Maharishi International University
  • Beibei Zhang Xi'an Jiaotong University
  • Hao Yan Syracuse University

DOI:

https://doi.org/10.5281/zenodo.12599805

ARK:

https://n2t.net/ark:/40704/JCTAM.v1n2a03

PURL:

https://purl.archive.org/suas/JCTAM.v1n2a03

Keywords:

User Trust, LLM-based Mental Health Applications, Reliability and Effectiveness, Data Privacy, Empathy in AI interactions

Abstract

The advent of Large Language Models (LLMs) in mental health applications has opened new avenues for providing psychological support and interventions. These applications leverage advanced natural language processing capabilities to offer real-time assistance, ranging from emotional support to cognitive behavioral therapy techniques. The success of these applications, however, hinges significantly on user trust in their reliability and effectiveness. This paper investigates the multifaceted factors influencing user trust in LLM-based mental health applications, including transparency of algorithms, data privacy, user interface design, perceived empathy, and the accuracy of the provided interventions. Additionally, it explores user perceptions regarding their reliability and effectiveness through a mixed-methods approach encompassing a comprehensive literature review, user surveys, and expert interviews with psychologists and AI ethicists. This study aims to provide a detailed understanding of the current landscape of LLM-based mental health tools, examining both the potential benefits and limitations. By synthesizing findings from diverse sources, it offers actionable insights into how these tools can be improved to enhance user trust and acceptance, ultimately contributing to better mental health outcomes. The implications of this research extend to developers, mental health professionals, and policymakers, highlighting the importance of ethical considerations and user-centered design in the development and deployment of LLM-based mental health solutions.

Author Biographies

Yuxin Zhao, New York University

Applied Urban science and informatics, New York University, NY, USA.

Jiawei Wu, Illinois Institute of Technology

Engineering in Artificial Intelligence for Computer Vision and Control, Illinois Institute of Technology, Chicago, IL, USA.

Ping Qu, Maharishi International University

Computer Science, Maharishi International University, Fairfield, IA, USA.

Beibei Zhang, Xi'an Jiaotong University

Software Engineering, Xi'an Jiaotong University, Xi'an, China.

Hao Yan, Syracuse University

Engineering and Computer Science, Syracuse University, Syracuse, NY, USA.

References

Hong, B., Zhao, P., Liu, J., Zhu, A., Dai, S., & Li, K. (2024). The application of artificial intelligence technology in assembly techniques within the industrial sector. Journal of Artificial Intelligence General science (JAIGS) ISSN: 3006-4023, 5(1), 1-12.

Liu, T., Xu, C., Qiao, Y., Jiang, C., & Chen, W. (2024). News recommendation with attention mechanism. arXiv preprint arXiv:2402.07422.

Yao, J., Wu, T., & Zhang, X. (2023). Improving depth gradient continuity in transformers: A comparative study on monocular depth estimation with cnn. arXiv preprint arXiv:2308.08333.

Xiong, J., Feng, M., Wang, X., Jiang, C., Zhang, N., & Zhao, Z. (2024). Decoding sentiments: Enhancing covid-19 tweet analysis through bert-rcnn fusion. Journal of Theory and Practice of Engineering Science, 4(01), 86-93.

Zou, Z. B., Song, L. P., & Song, Z. L. (2017, December). Labeled box-particle PHD filter for multi-target tracking. In 2017 3rd IEEE International Conference on Computer and Communications (ICCC) (pp. 1725-1730). IEEE.

Li, K., Xirui, P., Song, J., Hong, B., & Wang, J. (2024). The application of augmented reality (ar) in remote work and education. arXiv preprint arXiv:2404.10579.

Snyder, J., Goldstein, K. M., Gordon, A., Jacobs, M., Nugent, S., Magnante, A. T., ... & Gierisch, J. (2023). Psychiatric Conditions and Symptoms and Toxic Exposures Incurred During Military Service: An Evidence Map.

Su, J., Jiang, C., Jin, X., Qiao, Y., Xiao, T., Ma, H., ... & Lin, J. (2024). Large language models for forecasting and anomaly detection: A systematic literature review. arXiv preprint arXiv:2402.10350.

Zhang, B., Xiao, J., Yan, H., Yang, L., & Qu, P. (2024). Review of NLP Applications in the Field of Text Sentiment Analysis. Journal of Industrial Engineering and Applied Science, 2(3), 28-34.

Dai, S., Li, K., Luo, Z., Zhao, P., Hong, B., Zhu, A., & Liu, J. (2024). AI-based NLP section discusses the application and effect of bag-of-words models and TF-IDF in NLP tasks. Journal of Artificial Intelligence General science (JAIGS) ISSN: 3006-4023, 5(1), 13-21.

Li, K., Zhu, A., Zhou, W., Zhao, P., Song, J., & Liu, J. (2024). Utilizing deep learning to optimize software development processes. arXiv preprint arXiv:2404.13630.

Zhang, N., Xiong, J., Zhao, Z., Feng, M., Wang, X., Qiao, Y., & Jiang, C. (2024). Dose My Opinion Count? A CNN-LSTM Approach for Sentiment Analysis of Indian General Elections. Journal of Theory and Practice of Engineering Science, 4(05), 40-50.

Peng, Q., Zheng, C., & Chen, C. (2023). Source-free domain adaptive human pose estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 4826-4836).

Zhang, Y., Gong, Y., Cui, D., Li, X., & Shen, X. (2024). Deepgi: An automated approach for gastrointestinal tract segmentation in mri scans. arXiv preprint arXiv:2401.15354.

Sun, Y., Cui, Y., Hu, J., & Jia, W. (2018). Relation classification using coarse and fine-grained networks with SDP supervised key words selection. In Knowledge Science, Engineering and Management: 11th International Conference, KSEM 2018, Changchun, China, August 17–19, 2018, Proceedings, Part I 11 (pp. 514-522). Springer International Publishing.

Cao, Y., Yang, L., Wei, C., & Wang, H. (2023, November). Financial Text Sentiment Classification Based on Baichuan2 Instruction Finetuning Model. In 2023 5th International Conference on Frontiers Technology of Information and Computer (ICFTIC) (pp. 403-406). IEEE.

Yi, X., & Qiao, Y. (2024). GPU-Based Parallel Computing Methods for Medical Photoacoustic Image Reconstruction. arXiv preprint arXiv:2404.10928.

Yan, H., Xiao, J., Zhang, B., Yang, L., & Qu, P. (2024). The Application of Natural Language Processing Technology in the Era of Big Data. Journal of Industrial Engineering and Applied Science, 2(3), 20-27.

Peng, Q., Zheng, C., & Chen, C. (2024). A Dual-Augmentor Framework for Domain Generalization in 3D Human Pose Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2240-2249).

Wang, J., Wang, J., Dai, S., Yu, J., & Li, K. (2024). Research on emotionally intelligent dialogue generation based on automatic dialogue system. arXiv preprint arXiv:2404.11447.

Liu, T., Cai, Q., Xu, C., Zhou, Z., Ni, F., Qiao, Y., & Yang, T. (2024). Rumor Detection with a novel graph neural network approach. arXiv preprint arXiv:2403.16206.

Zhu, A., Li, K., Wu, T., Zhao, P., Zhou, W., & Hong, B. (2024). Cross-task multi-branch vision transformer for facial expression and mask wearing classification. arXiv preprint arXiv:2404.14606.

Zhao, Z., Zhang, N., Xiong, J., Feng, M., Jiang, C., & Wang, X. (2024). Enhancing E-commerce Recommendations: Unveiling Insights from Customer Reviews with BERTFusionDNN. Journal of Theory and Practice of Engineering Science, 4(02), 38-44.

Zhang, Y., Gui, K., Zhu, M., Hao, Y., & Sun, H. (2024). Unlocking personalized anime recommendations: Langchain and llm at the forefront. Journal of Industrial Engineering and Applied Science, 2(2), 46-53.

Peng, Q. (2022). Multi-source and Source-Private Cross-Domain Learning for Visual Recognition (Master's thesis, Purdue University).

Li, H., Xu, F., & Lin, Z. (2023). ET-DM: Text to image via diffusion model with efficient Transformer. Displays, 80, 102568.

Zhu, A., Liu, J., Li, K., Dai, S., Hong, B., Zhao, P., & Wei, C. (2024). Exploiting Diffusion Prior for Out-of-Distribution Detection. arXiv preprint arXiv:2406.11105.

Zhao, P., Li, K., Hong, B., Zhu, A., Liu, J., & Dai, S. (2024). Task allocation planning based on hierarchical task network for national economic mobilization. Journal of Artificial Intelligence General science (JAIGS) ISSN: 3006-4023, 5(1), 22-31.

Liu, T., Cai, Q., Xu, C., Zhou, Z., Xiong, J., Qiao, Y., & Yang, T. (2024). Image Captioning in news report scenario. arXiv preprint arXiv:2403.16209.

Wang, X., Qiao, Y., Xiong, J., Zhao, Z., Zhang, N., Feng, M., & Jiang, C. (2024). Advanced network intrusion detection with tabtransformer. Journal of Theory and Practice of Engineering Science, 4(03), 191-198.

Feng, M., Wang, X., Zhao, Z., Jiang, C., Xiong, J., & Zhang, N. (2024). Enhanced Heart Attack Prediction Using eXtreme Gradient Boosting. Journal of Theory and Practice of Engineering Science, 4(04), 9-16.

Zou, Z., Careem, M., Dutta, A., & Thawdar, N. (2023). Joint spatio-temporal precoding for practical non-stationary wireless channels. IEEE Transactions on Communications, 71(4), 2396-2409.

Zou, Z., Careem, M., Dutta, A., & Thawdar, N. (2022, May). Unified characterization and precoding for non-stationary channels. In ICC 2022-IEEE International Conference on Communications (pp. 5140-5146). IEEE.

Zhu, M., Zhang, Y., Gong, Y., Xu, C., & Xiang, Y. (2024). Enhancing Credit Card Fraud Detection A Neural Network and SMOTE Integrated Approach. arXiv preprint arXiv:2405.00026.

Jin, J., Ni, F., Dai, S., Li, K., & Hong, B. (2024). Enhancing federated semi-supervised learning with out-of-distribution filtering amidst class mismatches. Journal of Computer Technology and Applied Mathematics, 1(1), 100-108.

Song, J., Liu, H., Li, K., Tian, J., & Mo, Y. (2024). A comprehensive evaluation and comparison of enhanced learning methods. Academic Journal of Science and Technology, 10(3), 167-171.

Yao, J., Li, C., Sun, K., Cai, Y., Li, H., Ouyang, W., & Li, H. (2023, October). Ndc-scene: Boost monocular 3d semantic scene completion in normalized device coordinates space. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 9421-9431). IEEE Computer Society.

Xu, C., Yu, J., Chen, W., & Xiong, J. (2024, January). Deep learning in photovoltaic power generation forecasting: Cnn-lstm hybrid neural network exploration and research. In The 3rd International Scientific and Practical Conference (Vol. 363, p. 295).

Zhu, M., Zhang, Y., Gong, Y., Xing, K., Yan, X., & Song, J. (2024). Ensemble methodology: Innovations in credit default prediction using lightgbm, xgboost, and localensemble. arXiv preprint arXiv:2402.17979.

Liu, T., Xu, C., Qiao, Y., Jiang, C., & Yu, J. (2024). Particle filter slam for vehicle localization. arXiv preprint arXiv:2402.07429.

Yao, J., Pan, X., Wu, T., & Zhang, X. (2024, April). Building lane-level maps from aerial images. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 3890-3894). IEEE.

Peng, Q., Ding, Z., Lyu, L., Sun, L., & Chen, C. (2022). RAIN: regularization on input and network for black-box domain adaptation. arXiv preprint arXiv:2208.10531.

Zhang, Y., Zhu, M., Gui, K., Yu, J., Hao, Y., & Sun, H. (2024). Development and application of a monte carlo tree search algorithm for simulating da vinci code game strategies. arXiv preprint arXiv:2403.10720.

Zhibin, Z. O. U., Liping, S. O. N. G., & Xuan, C. (2019). Labeled box-particle CPHD filter for multiple extended targets tracking. Journal of Systems Engineering and Electronics, 30(1), 57-67.

Lightning, L. D. Application of DCSK in ACO-OFDM-IM technology of optical wireless communication.

Pinyoanuntapong, E., Ali, A., Jakkala, K., Wang, P., Lee, M., Peng, Q., ... & Sun, Z. (2023, September). Gaitsada: Self-aligned domain adaptation for mmwave gait recognition. In 2023 IEEE 20th International Conference on Mobile Ad Hoc and Smart Systems (MASS) (pp. 218-226). IEEE.

Zhang, Y., Zhu, M., Gong, Y., & Ding, R. (2023). Optimizing science question ranking through model and retrieval-augmented generation. International Journal of Computer Science and Information Technology, 1(1), 124-130.

Ni, F., Zang, H., & Qiao, Y. (2024, January). Smartfix: Leveraging machine learning for proactive equipment maintenance in industry 4.0. In The 2nd International scientific and practical conference “Innovations in education: prospects and challenges of today”(January 16-19, 2024), Sofia, Bulgaria, International Science Group (p. 313).

Downloads

Published

2024-07-01

How to Cite

Zhao, Y., Wu, J., Qu, P., Zhang, B., & Yan, H. (2024). Assessing User Trust in LLM-based Mental Health Applications: Perceptions of Reliability and Effectiveness. Journal of Computer Technology and Applied Mathematics, 1(2), 19–26. https://doi.org/10.5281/zenodo.12599805

Issue

Section

Articles

ARK

PURL