Our cross-sectional models that sought to predict initial diagnosis from baseline predictors indicates that this task is fairly straightforward if given information about cognitive tests, such as the Mini-Mental State Examination and the Clinical Dementia Rating, which is intuitive since doctors and diagnosticians typically rely on these tests in order to make their diagnoses.
Our investigation with k-Means Clustering indicates that finding a natural clustering within the data is tricky, and lends itself to the conclusion that the diagnoses given to patients may not be indicative of the bigger picture - that there are multiple levels of progression within Mild Cognitive Impairment or Alzheimer’s Disease.
Our longitudinal model, (1) predicting whether a given patient’s diagnosis would either change from Cognitively Normal to either Mild Cognitive Impairment or Alzheimer’s Disease illustrate that Logistic Regression and Support Vector Machines (SVM) have the highest Area Under the Receiver Operating Curve and the highest Average Precision Recall. These specific performance measurements were used to allow us to evaluate the predictive power of our models in a threshold-independent manner. The other advanced models (Gradiant Boosting, XGBoost, MLP and LGBM) showed similar performance, but inferior to logistic regression and and SVM. Our second longitudinal model, (2) whether a given patient’s diagnosis would either change from Mild Cognitive Impairment to Alzheimer’s Disease illustrate that LGBM and XGBoost have the highest AUROC and Average Precision Recall score, followed by random forest, gradient boosting and logistic regression. The performance measures in our longitudinal models for patients evolving from normal to MCI or AD (1) are lower compared to the performance measures for the models predicting if a patient will progress from MCI to Alzheimer’s (2). This might indicate that it is harder to predict who will develop MCI or Alzheimer’s starting from healthy individuals. Yet once a patient suffers from Mild Cognitive Impairment already, our model can more confidently predict if this patient will progress to Alzheimer’s Disease or not at the last visit. This might be explained by the multiple heterogeneous causes that might result in mild cognitive impairment, for example the basic process of normal aging in general results in some degree of the loss of memory when people age. Further conclusions on the performance of the longitudinal models are hard to make concrete since we only have approximately 400 individual patients to train our models on.
Our exploration of genetic data illustrated that the gene with the largest positive coefficient in this model has no identified link with cognitive function, autophagy, protein synthesis, calcium signalling, immune system, inflammation or any other process linked with neurodegenerative disease - and is only moderately expressed in the brain, and the gene with the strongest negative coefficient has actually been previously been positively linked with amyloidogenesis in mice, contrary to what our model might imply, which suggests that there may be limited immediate benefit to using gene expression profile data. Additionally, the samples from which the data is derived were taken from blood, not the central nervous system, which means that the gene expression profiles are not indicative of the cellular environment in the brain, as the gene expression profiles of different tissues varies hugely and the central nervous system is separated from the rest of the body by a near-impervious blood-brain-barrier.
One major limitation of our dataset is that, as our EDA shows, the youngest participant in the study is aged 55. Thus, we cannot really use any of our predictors to provide “early” diagnoses. Thus, extending our methodology to datasets involving younger patients would be extremely valuable. Another limitation of our data is that a large number of participants are male, white and married, which means that our inferences may not be generalizable to a larger population. To this end, we also would like to study whether the model does particularly well for this highly represented demographic group of patients and if so, adopt an adaptive boosting method for being able to predict under-represented groups in the ADNI dataset.
Second, the nature of the classification problem we are dealing with is not simple. Alzheimer’s disease cannot be measured directly, but instead must itself be “predicted” through proxies such as psychometric exams or a variety of physiological factors like amyloid-beta levels and brain volume that together signal the presence of AD. Thus, the response variable to our models is itself a prediction, and therefore introduces error into the model’s accuracy that must be considered. We explored unsupervised clustering methods to identify the degree to which the dependent variable that we chose to model - diagnosis - corresponds with the natural progression of AD. However, our future work entails using other variables, or combinations of variables that match the natural progression of the disease better than the three diagnoses in the dataset.
We might be able to identify predictors that help us move from this classification problem, which is in effect discretization error from imposing a set of three possible diagnoses to describe what is in actuality a continuous progression, to a regression problem. To measure this discretization error, it is necessary to consider the degree to which the diagnoses align with natural discontinuities (if there are any) in the progression of dementia and AD.
We also show that certain cognitive tests, such as the MMSE and the CDR, provide excellent predictive power for the initial diagnosis. However, we would like to generalize our findings here by looking at individual components from these tests and other cognitive tests such as the ADAS, such as spatial and temporal orientation or linguistic markers, to see if any particular feature correlates highly with decline in diagnosis.