Table Info

Table 2.

Model performances for classification of presence of any cognitive impairment among different feature sets (MCI and dementia are grouped together). Thresholds balance out the TPR and FPR

Feature set	Penalty	λ	AUC	Specificity	Sensitivity	PPV	NPV	Number of Features
NPT	L1	10	0.929	0.836	0.927	0.81	0.938	23
MMSE	L1	1	0.870	0.932	0.818	0.900	0.872	18
Baseline Linguistic	L2	10	0.882	0.726	0.855	0.701	0.869	374
Baseline Linguistic + MMSE	L1	1	0.913	0.863	0.873	0.828	0.900	392
Expanded Linguistic	L2	100	0.883	0.767	0.873	0.738	0.889	163
Expanded Linguistic + MMSE	L1	10	0.915	0.863	0.891	0.831	0.913	181
Expanded Syntactic	L2	10	0.821	0.808	0.764	0.75	0.819	31
Expanded Semantic	L2	100	0.870	0.795	0.836	0.754	0.866	126
Expanded Lexical	L1	10	0.844	1.000	0.618	1.000	0.777	6

Declarations

Acknowledgements

We acknowledge the dedication of the Framingham Heart Study participants without whom this research would not be possible. We also thank the FHS study staff for their many years of hard work in the examination of subjects and acquisition of data.

Author contributions

LZ, AN, and RHG contributed to the conception and design of the study. CMP, JAT, and HB contributed to refinement of the design of the study. AN performed the statistical analysis. LZ, AN, and RHG drafted the manuscript. CMP, JAT, HB, and RA critically reviewed the manuscript. All authors approved the final version of the manuscript.

Conflicts of interest

RA has received grant funding support from Biogen. She serves on the scientific advisory boards of Signant Health and Novo Nordisk, and is a scientific consultant to Biogen; none of which have any conflict of interest with the contents of this project. RHG reports personal fees from BrainCheck outside the submitted work and reports receiving stock options from BrainCheck.

Ethical approval

The Framingham Heart Study was approved by the Institutional Review Boards of Boston University Medical Center.

Consent to participate

All participants provided written consent to the study.

Consent to publication

Not applicable.

Availability of data and materials

Given that the text transcripts, demographic, and neuropsychological test data contain personal information, the dataset used in the current study is not publicly available. However, the scripts and tools are available upon request.

Funding

This work was partially supported by the National Library of Medicine Training Grant (T15LM007442; Authors JAT and HAB), National Science Foundation NRT Grant (1735095; Author LZ), Framingham Heart Study’s National Heart, Lung, and Blood Institute contract (N01-HC-25195; HHSN268201500001I), and NIH grants from the National Institute on Aging (AG008122, AG016495, AG033040, AG049810, AG054156, AG062109, AG068753) and Defense Advanced Research Projects Agency FA8750-16-C-0299. The views expressed in this manuscript are those of the authors and do not necessarily represent the views of the National Institutes of Health or the US Department of Health and Human Services. The funding agencies had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright

References

Dementia. WHO; c2021 [cited 2020 Jun 17]. Available from: https://www.who.int/news-room/fact-sheets/detail/dementia

Arrighi HM, Neumann PJ, Lieberburg IM, Townsend RJ. Lethality of Alzheimer disease and its impact on nursing home placement. Alzheimer Dis Assoc Disord. 2010;24:90–5. [DOI] [PubMed]

Alzheimer’s facts and figures report. Alzheimer’s Association; c2021[cited 2020 May 26]. Available from: https://www.alz.org/alzheimers-dementia/facts-figures

de Vugt ME, Verhey FRJ. The impact of early dementia diagnosis and intervention on informal caregivers. Prog Neurobiol. 2013;110:54–62. [DOI] [PubMed]

Research C for DE and. Alzheimer’s disease: developing drugs for treatment guidance for industy. US Food Drug Adm; [cited 2020 Aug 31]. Available from: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/alzheimers-disease-developing-drugs-treatment-guidance-industy [DOI]

Zhang R, Simon G, Yu F. Advancing Alzheimer’s research: a review of big data promises. Int J Med Inform. 2017;106:48–56. [DOI] [PubMed] [PMC]

U. S. Department of Health and Human Services. National plan to address Alzheimer’s disease: 2018 Update [Internet]. 2018. [cited 2020 Aug 31]. Available from: https://aspe.hhs.gov/system/files/pdf/259581/NatPlan2018.pdf

Robin J, Harrison JE, Kaufman LD, Rudzicz F, Simpson W, Yancheva M. Evaluation of speech-based digital biomarkers: review and recommendations. Digit Biomark. 2020;4:99–108. [DOI] [PubMed] [PMC]

Yancheva M, Fraser K, Rudzicz F. Using linguistic features longitudinally to predict clinical scores for Alzheimer’s disease and related dementias. Proceedings of SLPAT 2015: 6th Workshop on Speech and Language Processing for Assistive Technologies; 2015 Sep 11; Dresden, Germany. ACL Anthol; 2015.

10.

Mack WJ, Freed DM, Williams BW, Henderson VW. Boston naming test: shortened versions for use in Alzheimer’s disease. J Gerontol. 1992;47:P154–8. [DOI] [PubMed]

11.

Fraser KC, Meltzer JA, Rudzicz F. Linguistic features identify Alzheimer’s disease in narrative speech. J Alzheimers Dis. 2016;49:407–22. [DOI] [PubMed]

12.

Alhanai T, Au R, Glass J. Spoken language biomarkers for detecting cognitive impairment. arXiv:1710.07551v1 [Preprint]. 2017 [cited 2020 June 15]:[8 p.]. Available from https://arxiv.org/abs/1710.07551. [DOI]

13.

Thomas JA, Burkhardt HA, Chaudhry S, Ngo AD, Sharma S, Zhang L, et al. Assessing the utility of language and voice biomarkers to predict cognitive impairment in the Framingham Heart Study cognitive aging cohort data. J Alzheimers Dis. 2020;76:905–22. [DOI] [PubMed]

14.

Uhlmann RF, Larson EB. Effect of education on the Mini-Mental State Examination as a screening test for dementia. J Am Geriatr Soc. 1991;39:876–80. [DOI] [PubMed]

15.

Matthews F, Marioni R, Brayne C; Medical Research Council Cognitive Function and Ageing Study. Examining the influence of gender, education, social class and birth cohort on MMSE tracking over time: a population-based prospective cohort study. BMC Geriatr. 2012;12:45. [DOI] [PubMed] [PMC]

16.

Orrell M, Sahakian B. Education and dementia. BMJ. 1995;310:951–2. [DOI] [PubMed] [PMC]

17.

Buckwalter JG, Sobel E, Dunn ME, Diz MM, Henderson VW. Gender differences on a brief measure of cognitive functioning in Alzheimer’s disease. Arch Neurol. 1993;50:757–60. [DOI] [PubMed]

18.

Coravos A, Goldsack JC, Karlin DR, Nebeker C, Perakslis E, Zimmerman N, et al. Digital medicine: a primer on measurement. Digit Biomark. 2019;3:31–71. [DOI] [PubMed] [PMC]

19.

Andersson C, Johnson AD, Benjamin EJ, Levy D, Vasan RS. 70-year legacy of the Framingham Heart Study. Nat Rev Cardiol. 2019;16:687–98. [DOI] [PubMed]

20.

Framingham Heart Study-Cohort (FHS-Cohort). BioLINCC; [cited 2020 June 15]. Available from https://biolincc.nhlbi.nih.gov/studies/framcohort/

21.

Framingham Cohort. dbGaP Study Accession: phs000007.v31.p12 [dataset]. March 30, 2020 [cited 2020 June 15]. Available from: https://www.ncbi.nlm.nih.gov/projects/gap/cgibin/study.cgi?study_id=phs000007.v31.p12

22.

Wide range achievement test | fourth edition. Pearson; c1996–2021 [cited 2020 June 16 ]. Available from https://www.pearsonassessments.com/store/usassessments/en/Store/Professional-Assessments/Academic-Learning/Wide-Range-Achievement-Test-%7C-FourthEdition/p/100001722.html

23.

Armitage SG. An analysis of certain psychological tests used for the evaluation of brain injury. Psychol Monogr. 1946;60:i–48. [DOI]

24.

Weintraub S. Boston naming test - second edition. Pearson Clinical Australia & New Zealand. [cited 2020 June 16 ]. Available from https://www.pearsonclinical.com.au/products/view/525

25.

Wechsler D. A standardized memory scale for clinical use. J Psychol. 1945;19:87–95. [DOI]

26.

Hooper E. (VOT^TM) Hooper visual organization test^TM. WPS; c2021 [cited 2020 June 16 ]. Available from https://www.wpspublish.com/vot-hooper-visual-organization-test

27.

Downer B, Fardo DW, Schmitt FA. A summary score for the Framingham Heart Study neuropsychological battery. J Aging Health. 2015;27:1199–222. [DOI] [PubMed] [PMC]

28.

Shimoyama I, Ninchoji T, Uemura K. The finger-tapping test. A quantitative analysis. Arch Neurol 1990;47:681–4. [DOI] [PubMed]

29.

Brain Health Research Lab/neuropsychology group at the Framingham Heart Study [Internet]. Boston: Boston University School of Medicine. [cited 2020 June 16]. Available from https://www.bumc.bu.edu/anatneuro/research/brain-health-research-lab-neuropsychology-group-at-the-framingham-heart-study/

30.

Ash S, Moore P, Antani S, McCawley G, Work M, Grossman M. Trying to tell a tale: discourse impairments in progressive aphasia and frontotemporal dementia. Neurology. 2006;66:1405–13. [DOI] [PubMed]

31.

Kurlowicz L, Wallace M. The mini-mental state examination (MMSE). J Gerontol Nurs. 1999;25:8–9. [DOI]

32.

Lobur M, Romanyuk A, Romanyshyn M. Using NLTK for educational and scientific purposes. 2011 11th Int. Conf. Exp. Des. Appl. CAD Syst. Microelectron. CADSM, 2011, p. 426–8.

33.

Honnibal M, Montani I. SpaCy: industrial-strength natural language processing (NLP) with Python and Cython. Explosion; 2019.

34.

Boschi V, Catricalà E, Consonni M, Chesi C, Moro A, Cappa SF. Connected speech in neurodegenerative language disorders: a review. Front Psychol. 2017;8. [DOI] [PubMed] [PMC]

35.

Kempler D, Goral M. Language and dementia: neuropsychological aspects. Annu Rev Appl Linguist. 2008;28:73–90. [DOI] [PubMed] [PMC]

36.

A speech recognition-based solution for the automatic detection of mild cognitive impairment from spontaneous speech. Curr Alzheimer Res; c2021 (cited 2020 February 6). Available from http://www.eurekaselect.com/157444/article [DOI]

37.

Association AP. Diagnostic and statistical manual of mental disorders (DSM-5®). American Psychiatric Assoc; 2013.

38.

Bachman DL, Wolf PA, Linn R, Knoefel JE, Cobb J, Belanger A, et al. Prevalence of dementia and probable senile dementia of the Alzheimer type in the Framingham Study. Neurology 1992;42:115–9. [DOI] [PubMed]

39.

McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimer’s disease. Neurology 1984;34:939. [DOI] [PubMed]

40.

Satizabal CL, Beiser AS, Chouraki V, Chêne G, Dufouil C, Seshadri S. Incidence of dementia over three decades in the Framingham Heart Study. N Engl J Med 2016;374:523–32. [DOI] [PubMed] [PMC]

41.

Orimaye SO, Wong JS-M, Golden KJ. Learning predictive linguistic features for Alzheimer’s disease and related dementias using verbal utterances. Proc. Workshop Comput. Linguist. Clin. Psychol. Linguist. Signal Clin. Real., Baltimore, Maryland, USA: Association for Computational Linguistics; 2014, p. 78–87.

42.

Taylor A, Marcus M, Santorini B. The Penn Treebank: an overview. In: Abeillé A, , editor. Treebanks, vol. 20, Dordrecht: Springer Netherlands; 2003, pp. 5–22. [DOI]

43.

Hugo J, Ganguli M. Dementia and cognitive impairment: epidemiology, diagnosis, and treatment. Clin Geriatr Med. 2014;30:421–42. [DOI] [PubMed] [PMC]

44.

Jarrold W, Peintner B, Wilkins D, Vergryi D, Richey C, Gorno-Tempini ML, et al. Aided diagnosis of dementia type through computer-based analysis of spontaneous speech. Proc. Workshop Comput. Linguist. Clin. Psychol. Linguist. Signal Clin. Real., Baltimore, Maryland, USA: Association for Computational Linguistics; 2014, pp. 27–37.

45.

Tausczik YR, Pennebaker JW. The psychological meaning of words: LIWC and computerized text analysis methods. J Lang Soc Psychol. 2010;29:24–54. [DOI]

46.

Weiner J, Schultz T. Automatic screening for transition into dementia using speech. Speech Commun. 13th ITG-Symp. 2018, pp. 1–5.

47.

Croisile B, Ska B, Brabant MJ, Duchene A, Lepage Y, Aimard G, et al. Comparative study of oral and written picture description in patients with Alzheimer’s disease. Brain Lang. 1996;53:1–19. [DOI] [PubMed]

48.

Goedert M, Ghetti B, Spillantini MG. Frontotemporal dementia: implications for understanding Alzheimer disease. Cold Spring Harb Perspect Med. 2012;2:a006254. [DOI] [PubMed] [PMC]

49.

Dai T, Davey A, Woodard JL, Miller LS, Gondo Y, Kim S-H, et al. Sources of variation on the Mini-Mental State Examination in a population-based sample of centenarians. J Am Geriatr Soc. 2013;61:1369–76. [DOI] [PubMed] [PMC]

50.

Tangalos EG, Smith GE, Ivnik RJ, Petersen RC, Kokmen E, Kurland LT, et al. The Mini-Mental State Examination in general medical practice: clinical utility and acceptance. Mayo Clin Proc. 1996;71:829–37. [DOI] [PubMed]

51.

Trzepacz PT, Hochstetler H, Wang S, Walker B, Saykin AJ. Relationship between the Montreal Cognitive Assessment and Mini-mental State Examination for assessment of mild cognitive impairment in older adults. BMC Geriatr. 2015;15. [DOI] [PubMed] [PMC]

52.

Lesuis SL, Hoeijmakers L, Korosi A, de Rooij SR, Swaab DF, Kessels HW, et al. Vulnerability and resilience to Alzheimer’s disease: early life conditions modulate neuropathology and determine cognitive reserve. Alzheimers Res Ther. 2018;10:95. [DOI] [PubMed] [PMC]

53.

Zemla JC, Austerweil JL. Analyzing Knowledge Retrieval Impairments Associated with Alzheimer’s disease using network analyses. Complexity. 2019;2019:e4203158. [DOI]

54.

Harwood DG, Ownby RL. Ethnicity and dementia. Curr Psychiatry Rep. 2000;2:40–5. [DOI] [PubMed]

55.

Leveille SG, Guralnik JM, Ferrucci L, Corti MC, Kasper J, Fried LP. Black/white differences in the relationship between MMSE scores and disability: the women’s health and aging study. J Gerontol Ser B. 1998;53B:P201–8. [DOI]

56.

Overton M, Pihlsgård M, Elmståhl S. Diagnostic stability of mild cognitive impairment, and predictors of reversion to normal cognitive functioning. Dement Geriatr Cogn Disord. 2019;48:317–29. [DOI] [PubMed]

57.

Stegmann GM, Hahn S, Liss J, Shefner J, Rutkove SB, Kawabata K, et al. Repeatability of commonly used speech and language features for clinical applications. Digit Biomark. 2020;4:109–22. [DOI] [PubMed] [PMC]

58.

Liu CK, Miller BL, Cummings JL, Mehringer CM, Goldberg MA, Howng SL, et al. A quantitative MRI study of vascular dementia. Neurology. 1992;42:138. [DOI] [PubMed]

59.

Lin H, Karjadi C, Ang TFA, Prajakta J, McManus C, Alhanai TW, et al. Identification of digital voice biomarkers for cognitive health. Explor Med. 2020;1:406–17. [DOI] [PubMed] [PMC]