Family caregivers of individuals with dementia often experi ence significant emotional and cognitive burden, which may manifest in the language they use on social media. Identify ing such posts is valuable for understanding caregiver needs and advancing mental health research. Task 3 of the SMM4H 2025 shared task focuses on classifying whether a tweet indi cates that the user has a family member with dementia. The task requires sensitivity to both direct and indirect expres sions of caregiving. We addressed this task using a prompt based zero-shot classification system powered by large lan guage models (LLMs). Our method leverages instruction tuned models, including DeepSeek-R1 and Mixtral-8x7B. To further evaluate our results, we developed a LLM-based multi-agent system to analyze cognitive distortions in tweets labeled as caregiver-related. The resulting distortion patterns offer psychological insight into the model’s predictions and highlight the system’s potential for broader applications in mental health monitoring.
We designed a zero-shot prompt for classifying whether a tweet indicates the author has a family member with dementia. The prompt encourages step-by-step reasoning, improving model sensitivity to indirect expressions. The example below illustrates how we structured the system using instruction-tuned models such as DeepSeek-R1 and Mixtral.
DeepSeek-V1.3 achieved top performance on validation set and test set.
After classification, we use a multi-agent system where each agent detects one of 11 cognitive distortion types. A controller agent routes tweets to distortion-specific agents, each leveraging a definition and keyword feature set to decide if the distortion is present. This produces a binary vector for analysis.
To interpret the system’s output, we analyzed the distortion distribution for tweets labeled as “caregiver-related.” The radar chart below compares distortion frequencies between validation and test sets. Emotional Reasoning and Catastrophizing dominate across both, validating the model’s psychological relevance.
Artin Tonekaboni
Vision(Xin) Wang
Sadamori Kojaku
Luis M. Rocha
We thank the reviewers for their constructive feedback on this research.