Data Science for Mental Health (DS4MH) @ The Alan Turing Institute

About Us

The vision for this interest group is to kick-start one or more projects using contemporary data science and multi-modal data for mental health to provide insight and benefit for individuals, clinicians, and contribute to fundamental research in mental health (including dementia) as well as the data science methodology. It aims to provide an informal bridge between clinicians, charities, and data owners (like CRIS, UKDP, and Biobank) and data science researchers to stimulate and align cutting edge research in this area.

Events

Meetings

We organise monthly meetings (including half-an-hour long invited talks) at the Turing. Meetings are organised and moderated by Jenny Chim, Yue Wu, and Emilio Ferrucci. Please join our mailing list for more updated information.

As a part of AI UK Fringe, we jointly organised a hybrid event with the NLP interest group on AI for Mental Health Monitoring on 28th March 2024.

See here for our previous talks.

Upcoming Events

Meetings

Date	Time	Presenter	Title
2025.05.15	15:00		Introduction
	15:05	Viet Cuong (Johnny) Nguyen (Georgia Institute of Technology)	Do Large Language Models Align with Core Mental Health Counseling Competencies? The rapid evolution of Large Language Models (LLMs) offers promising potential to alleviate the global scarcity of mental health professionals. However, LLMs' alignment with essential mental health counseling competencies remains understudied. We introduce CounselingBench, a novel NCMHCE-based benchmark evaluating LLMs across five key mental health counseling competencies. Testing 22 general-purpose and medical-finetuned LLMs, we find frontier models exceed minimum thresholds but fall short of expert-level performance, with significant variations: they excel in Intake, Assessment & Diagnosis yet struggle with Core Counseling Attributes and Professional Practice & Ethics. Medical LLMs surprisingly underperform generalist models accuracy-wise, while at the same time producing slightly higher-quality justifications but making more context-related errors. Our findings highlight the complexities of developing AI systems for mental health counseling, particularly for competencies requiring empathy and contextual understanding. We found that frontier LLMs perform at a level exceeding the minimal required level of aptitude for all key mental health counseling competencies, but fall short of expert-level performance, and that current medical LLMs do not significantly improve upon generalist models in mental health counseling competencies. This underscores the critical need for specialized, mental health counseling-specific fine-tuned LLMs that rigorously aligns with core competencies combined with appropriate human supervision before any responsible real-world deployment can be considered.
	15:40	Dr. Yuxiang Zhou (QMUL/KCL)	Modeling Subjectivity in Cognitive Appraisal with Language Models As the utilization of language models in interdisciplinary, human-centered studies grow, the expectation of model capabilities continues to evolve. Beyond excelling at conventional tasks, models are recently expected to perform well on user-centric measurements involving confidence and human (dis)agreement- factors that reflect subjective preferences. While modeling of subjectivity plays an essential role in cognitive science and has been extensively studied, it remains under-explored within the NLP community. In light of this gap, we explore how language models can harness subjectivity by conducting comprehensive experiments and analysis across various scenarios using both fine-tuned models and prompt-based large language models (LLMs). Our quantitative and qualitative experimental results indicate that existing post-hoc calibration approaches often fail to produce satisfactory results. However, our findings reveal that personality traits and demographical information are critical for measuring subjectivity. Furthermore, our in-depth analysis offers valuable insights for future research and development in the interdisciplinary studies of NLP and cognitive science.
	16:20		After talks discussion