Empirical Evaluation of Prompt Sensitivity and Optimization in Alzheimer Disease Speech Classification

Large Language Models (LLMs) are increasingly central to Alzheimer’s disease (AD) research, particularly for classifying speech-based transcripts. However, a significant factor affecting LLM performance is the choice of a prompting strategy. Despite the known sensitivity of these models, AD research currently lacks a formal framework for assessing prompt quality or stability. Using the ADReSS Challenge dataset, this study evaluates whether prompts used for AD speech-based classification are sensitive to minor linguistic variations and tests whether a prompt optimization framework can reduce this sensitivity. Our results demonstrate that various models are highly sensitive to small changes in prompts. Notably, optimization led to a significant 16.63% improvement in accuracy for the Mixtral 8x7B model. These findings highlight the necessity of validating and optimizing prompts to ensure reliable diagnostic outputs in clinical research.