Skip to main content

Alzheimer’s Disease Assessment Scale–Cognitive subscale variants in mild cognitive impairment and mild Alzheimer’s disease: change over time and the effect of enrichment strategies

Abstract

Background

Development of new treatments for Alzheimer’s disease (AD) has broadened into early interventions in individuals with modest cognitive impairment and a slow decline. The 11-item version of the Alzheimer’s Disease Assessment Scale–Cognitive subscale (ADAS-Cog) was originally developed to measure cognition in patients with mild to moderate AD. Attempts to improve its properties for early AD by removing items prone to ceiling and/or by adding cognitive measures known to be impaired early have yielded a number of ADAS-Cog variants. Using Alzheimer’s Disease Neuroimaging Initiative data, we compared the performance of the 3-, 5-, 11- and 13-item ADAS-Cog variants in subjects with early AD. Given the interest in enrichment strategies, we also examined this aspect with a focus on cerebrospinal fluid (CSF) markers.

Methods

Subjects with mild cognitive impairment (MCI) and mild AD with available ADAS-Cog 13 and CSF data were analysed. The decline over time was defined by change from baseline. Direct cross-comparison of the ADAS-Cog variants was performed using the signal-to-noise ratio (SNR), with higher values reflecting increased sensitivity to detect change over time.

Results

The decline over time on any of the ADAS-Cog variants was minimal in subjects with MCI. Approximately half of subjects with MCI fulfilled enrichment criteria for positive AD pathology. The impact of enrichment was detectable but subtle in MCI. The annual decline in mild AD was more pronounced but still modest. More than 90 % of subjects with mild AD had positive AD pathology. SNRs were low in MCI but greater in mild AD. The numerically largest SNRs were seen for the ADAS-Cog 5 in MCI and for both the 5- and 13-item ADAS-Cog variants in mild AD, although associated confidence intervals were large.

Conclusions

The possible value of ADAS-Cog expansion or reduction is less than compelling, particularly in MCI. In mild AD, adding items known to be impaired at early stages seems to provide more benefit than removing items on which subjects score close to ceiling.

Background

For the past two decades, the 11-item Alzheimer’s Disease Assessment Scale–Cognitive subscale (ADAS-Cog 11) has been a nearly ubiquitous measure of cognition in clinical trials of putative new therapies for Alzheimer’s disease (AD). However, recent use of this scale has revealed a number of limitations driven by a shift toward assessing and treating patients in earlier stages of the disease. One is the lack of sensitivity to detect change in early stages of AD, as it seems most sensitive when used in patients in the moderate stage of AD [i.e., Mini Mental State Examination (MMSE) score of 12–18] [1]. The second relates to measuring cognitive domains known to be impaired at the early stages, such as executive function [24], that are not captured by ADAS-Cog 11. As a result, subjects with mild cognitive impairment (MCI) tend to score at ceiling (i.e., score of 0 for ADAS-Cog) on eight of the 11 ADAS-Cog subtest items [57].

In response to criticism of the ADAS-Cog’s inability to measure relevant cognitive domains, Mohs et al. [8] suggested the use of additional tests, such as Digit Cancellation, Delayed Word Recall and a Maze test. This has led to the creation of ADAS-Cog variants such as the ADAS-Cog 13. A second approach to improving the scale’s sensitivity was to remove subtests prone to ceiling effects (i.e., ADAS-Cog 3). ADAS-Cog 3 tests solely memory, however. The ADAS-Cog 5 variant combines the ADAS-Cog 3 items with Delayed Recall and Digit Cancellation from ADAS-Cog 13. This tactic could improve detection of cognitive decline over time because it measures more relevant domains impacted in early stages of the disease than the ADAS-Cog 3. However, the sensitivities of these new variants to detect change over time have not been compared in patients with early AD.

In this study, we employed data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database to address this issue, as described in the Methods section. Our aim was to evaluate which of the four variants (ADAS-Cog 3, ADAS-Cog 5, the original ADAS-Cog 11, and ADAS-Cog 13) best detects cognitive decline over time in subjects with MCI or mild AD. We hypothesised that the ‘tailored’ ADAS-Cog 5 variant would provide the best means of assessing decline in the subjects with early AD compared with the other variants. It should be noted that our goal was not to validate a new instrument. In addition, as enrichment should help to identify subjects with MCI with AD pathology [911], we evaluated whether it helps to improve the sensitivity of the ADAS-Cog variants to detect decline over time.

Methods

Data sourceFootnote 1

Data used in the preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). The ADNI was launched in 2003 by the National Institute on Aging (NIA), the National Institute of Biomedical Imaging and Bioengineering (NIBIB), the Food and Drug Administration (FDA), private pharmaceutical companies and non-profit organizations, as a $60 million, 5-year public private partnership. The primary goal of ADNI has been to test whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and early Alzheimer’s disease (AD). Determination of sensitive and specific markers of very early AD progression is intended to aid researchers and clinicians to develop new treatments and monitor their effectiveness, as well as lessen the time and cost of clinical trials.

The Principal Investigator of this initiative is Michael W. Weiner, MD, VA Medical Center and University of California–San Francisco. ADNI is the result of efforts of many coinvestigators from a broad range of academic institutions and private corporations, and subjects have been recruited from over 50 sites across the U.S. and Canada. The initial goal of ADNI was to recruit 800 subjects but ADNI has been followed by ADNI-GO and ADNI-2.

To date these three protocols have recruited over 1500 adults, ages 55 to 90, to participate in the research, consisting of cognitively normal older individuals, people with early or late MCI, and people with early AD. The follow up duration of each group is specified in the protocols for ADNI-1, ADNI-2 and ADNI-GO. Subjects originally recruited for ADNI-1 and ADNI-GO had the option to be followed in ADNI-2. For up-to-date information, see www.adni-info.org .

Trial registration details

The trial is registered at ClinicalTrials.gov under trial registration numbers NCT00106899 (date of registration 31 March 2005), NCT01231971 (date of registration 27 October 2010) and NCT01078636 (date of registration 1 March 2010).

Data from ADNI 1, ADNI Grand Opportunities (ADNI GO) and ADNI 2 were downloaded on 30 September 2014. Subjects with available ADAS-Cog 13 data (i.e., comprising ADAS-Cog 11 items plus Delayed Word Recall and Digit Cancellation) and with available baseline cerebrospinal fluid (CSF) biomarkers, both amyloid-β (Aβ) and total Tau (t-Tau), were included in the main analysis of subjects with mild AD or MCI.

Due to the variable amount of available data at later time points, our focus was on subjects with change from baseline to 12 months in mild AD and to 24 months in MCI. Data beyond 12 months for mild AD and beyond 24 months for MCI were considered for data description over time.

Ethics, consent and permissions

The ADNI study was approved individually by the institutional review boards of all the participating institutions. Written informed consent was obtained from all participants at each site. Please see the Acknowledgements section for a list of institutional review boards that approved the study.

Study participants

The general inclusion and exclusion criteria for the ADNI study and the screening procedures were described by Petersen et al. [12]. The original subject cohorts included age-matched cognitively healthy subjects, subjects with MCI and subjects with mild AD dementia. The subjects with MCI had MMSE scores of 24–30 and a Clinical Dementia Rating (CDR) global score of 0.5 with a mandatory requirement of the memory box score being 0.5 or greater. The subjects with MCI had to be largely intact with regard to functional performance and could not qualify for the diagnosis of dementia [13]. The subjects with mild AD had MMSE scores of 20–26, a CDR score 0.5 or 1 and met the National Institute of Neurological and Communicative Disorders and Stroke/Alzheimer’s Disease and Related Disorders Association criteria for probable AD [14]. Subjects with MCI and subjects with mild AD were also required to meet criteria for memory impairment on the Wechsler Memory Scale–Revised Logical Memory II subscale [12].

Populations in the statistical assessment

The ‘MCI set’ comprised subjects with MCI with available ADAS-Cog 13 at baseline and 24 months as well as available baseline Aβ and t-Tau CSF biomarkers. The ‘AD set’ comprised subjects with mild AD with available ADAS-Cog 13 at baseline and 12 months as well as available baseline Aβ and t-Tau CSF biomarkers. The MCI and AD sets described above are referred to as ‘non-enriched’ because their clinical data were analysed regardless of their biomarker status. The ‘biomarker-positive sets’ (‘enriched’) comprised those subjects with a positive AD pathology as defined by the magnitude of selected baseline CSF biomarkers [15] or with the presence of a genetic risk marker apolipoprotein E ε4 (ApoE4):

  • CSF t-Tau/Aβ ratio >0.39

  • CSF Aβ1–42 < 192 pg/ml

  • CSF phosphorylated Tau (p-Tau) >23 pg/ml

  • CSF t-Tau >93 pg/ml

  • ApoE4 risk (one or two copies of the ApoE4 allele)

For the purposes of this article, a CSF t-Tau/Aβ ratio >0.39 was considered to be the main enrichment strategy because, in the context of MCI, this would fulfil the recent International Working Group 2 research criteria for prodromal AD [16]. These cohorts are labelled as MCI+ and AD+ in this publication.

The ‘biomarker-negative’ sets comprised the complementary groups that did not fulfil the baseline biomarker enrichment criteria for AD based on t-Tau/Aβ and thus were not classified as having prodromal AD. Only biomarker-negative subjects with MCI are presented (MCI−). There were very few biomarker-negative subjects with mild AD.

Statistical methods

In this study, we compared the ADAS-Cog 3 [17], ADAS-Cog 13 [8], the traditional ADAS-Cog 11 [18, 19] and the ADAS-Cog 5 variants (Additional file 1) to see which of them had the best ability to demonstrate a change in subjects with mild AD or with MCI.

Values for ADAS-Cog total scores were calculated on the basis of individual items and were recorded as missing if at least one ADAS-Cog item was not available (affecting six subjects with MCI and seven with mild AD). CSF measurements were obtained from biomarker datasets, where the latest available recorded result of the baseline sample was selected in case of duplicates.

Change from baseline was calculated for each individual. The mean change from baseline was calculated by cohort for the MCI and mild AD sets, as well as for the individual enriched sets, to describe the decline over time.

Demographics were obtained at the baseline visit and described by their mean value and standard deviation for continuous variables or by percentage for categorical variables.

Sensitivity of the ADAS-Cog variants to show a change over time was assessed using the signal-to-noise ratio (SNR). The SNR is calculated as the estimated mean change from baseline divided by the corresponding standard deviation. A positive or negative number indicates the direction of change towards increase (worsening) or reduction (improvement) from the observed ADAS-Cog score at baseline. The SNR reflects changes in the outcome relative to its variability, thereby allowing for direct comparison of different ADAS-Cog variants with respect to their sensitivity to detect a change from baseline. An increased SNR, representing a higher sensitivity, might be expected after removal of items that affect the mean change from baseline minimally (e.g., consistently scoring at the ceiling) through an anticipated decrease in variability that should be associated with a total score containing fewer individual items.

The mean change in ADAS-Cog score and the standard deviation were estimated from an analysis of covariance model correcting for baseline ADAS-Cog and MMSE scores, age, sex and ApoE4 risk category. Variability of SNR estimates was captured using 95 % bootstrap confidence intervals based on 1000 samples for each set.

The four different ADAS-Cog variants were compared using a hypothetical patient, which was an ApoE4-positive 75-year-old woman with baseline MMSE and ADAS-Cog scores in line with the diagnostic group. Specifically, for subjects with MCI, the estimated baseline MMSE of 28 and ADAS-Cog 11 score of 10 were used. For subjects with mild AD, a baseline MMSE of 23 and ADAS-Cog 11 score of 18 were assumed. The baseline ADAS-Cog score for the remaining ADAS-Cog variants was estimated from a linear regression of baseline values of the respective ADAS-Cog variant vs. baseline ADAS-Cog 11.

Differences in SNR between pairs of ADAS-Cog variants were assessed by means of p values obtained from a paired t test accounting for correlated data, making use of covariance estimates obtained from the aforementioned bootstrapping samples. The reported p values are not corrected for multiplicity and are considered exploratory.

The analyses described above were performed on ADAS-Cog variants for the group of enriched subjects and separately for the group of non-enriched subjects.

No imputations for missing data or for discontinued subjects were performed; subjects with missing information were excluded from the analysis.

Results

Datasets analysed

There were 634 subjects with MCI for whom all baseline and month 24 ADAS-Cog 13 values were available. Of these, 382 (60 %) provided a CSF sample at baseline. Of the 242 subjects with mild AD for whom all baseline and month 12 ADAS-Cog 13 values were available, 97 (40 %) provided a CSF sample at baseline. The 229 subjects with MCI and 73 with mild AD who did not have ADAS-Cog 13 values at 24 or 12 months, respectively, were not included in the analyses. The number of subjects attending follow-up visits diminished over time. To maintain a sufficient number of subjects in the analyses, the focus was on change from baseline to 12 months in mild AD, while the change to 24 months was analysed for subjects diagnosed with MCI.

Baseline characteristics

The baseline characteristics in the MCI and mild AD cohorts were similar, except that subjects with mild AD were older and more cognitively impaired (Table 1).

Table 1 Demographic and clinical data at baseline

The 206 MCI+ (t-Tau/Aβ >0.39) set had baseline demographic characteristics similar to those of the non-enriched set, with the exception that 69 % of MCI+ subjects were ApoE4-positive vs. only 20 % of MCI− subjects. The baseline MMSE and CDR scores were also similar between the groups. The MCI+ set, however, was more impaired on those ADAS-Cog variants which include the additional measures of Delayed Word Recall and Digit Cancellation (e.g., ADAS-Cog 13 and the ‘tailored’ ADAS-Cog 5). The data in mild AD and mild AD+ sets are almost identical, as the majority of subjects with mild AD enrolled in ADNI studies had positive AD pathology at baseline (>90 %).

Performance of ADAS-Cog variants in MCI and mild AD populations with and without enrichment

As expected, the magnitude of the absolute change on ADAS-Cog in mild AD is larger than that in MCI. As shown in Table 2, there was practically no decline in mean ADAS-Cog 11 score in subjects with MCI (worsening of 0.9 on a 70-point scale over 24 months and 1.9 points over 36 months). The change from baseline was modest in patients with mild AD (worsening of 3.5 points over 12 months and of 8.3 points over 24 months) (Additional file 2). This was also reflected on the MMSE (Additional file 3).

Table 2 ADAS-Cog 11 scores at each visit and change from baseline for MCI patients

On ADAS-Cog 11, enrichment minimally increased the decline in subjects with MCI over 24 months (Table 2). Even after 36 months in MCI+ subjects, the decline on ADAS-Cog 11 was less than 4 points. As shown in Table 3, there was a minimal decline over 24 months and a minimal impact of enrichment on the other ADAS-Cog variants (3-, 5- and 13-item) in subjects with MCI. As expected, there was no change in any of the ADAS-Cog variants in the subjects with MCI without AD pathology (MCI−) up to 36 months.

Table 3 ADAS-Cog scores and change from baseline up to 36 months for patients with MCI

Changes over time on other ADAS-Cog variants were modest (4.35 points over 12 months on ADAS-Cog 13) in subjects with mild AD (Table 4). Even though the number of subjects with mild AD attending a visit at 24 months was low, there was a trend toward a larger decline over 24 months with almost a 10-point decline on ADAS-Cog 13.

Table 4 ADAS-Cog score and change from baseline up to 24 months for subjects with mild AD

As the majority of subjects with mild AD had positive AD pathology at baseline (>90 %), no benefit of enrichment could be detected with respect to the increased magnitude of change over time (Additional file 2). Analysis of the contribution of individual items showed that Word Recall (Q1), Delayed Word Recall (Q4) and Word Recognition (Q8) largely contributed to the overall ADAS-Cog score (Fig. 1). Delayed Word Recall was impaired early and contributed largely to the overall score of the ADAS-Cog variants containing this item (i.e., the 13- and 5-item variants). Thus, somewhat counterintuitively, the score of the ADAS-Cog 5 exceeded that of the ADAS-Cog 11, presumably due to the contribution of Delayed Word Recall.

Fig. 1
figure 1

Relative percentage contribution of individual items to total Alzheimer’s Disease Assessment Scale–Cognitive subscale (ADAS-Cog) score at baseline for mild cognitive impairment population. a ADAS-Cog 3. b ADAS-Cog 5. c. ADAS-Cog 11. d ADAS-Cog 13

Comparison of ADAS-Cog variants in MCI and mild AD populations with and without enrichment using SNR

To account for differences in possible maximum range and the variability in change from baseline, and hence to allow for cross-comparison of the various ADAS-Cog variants, SNRs were estimated on the basis of an analysis of covariance approach, correcting for baseline age, baseline MMSE, sex and ApoE4 risk status. Specific comparisons of estimated SNRs across ADAS-Cog variants were made for a hypothetical patient as described in the Methods section. Table 5 shows the estimated SNRs for subjects with MCI, jointly presented with the corresponding bootstrap 95 % confidence intervals. Similarly, Table 6 shows results for subjects diagnosed with mild AD.

Table 5 Signal-to-noise ratio for change in ADAS-Cog variants over 24 months in MCI cohorts
Table 6 Signal-to-noise ratio for change in ADAS-Cog variants over 12 months in mild AD cohorts

Higher SNR values reflect increased sensitivity to detect a change. In subjects with MCI, SNRs based on estimated change from baseline to 24 months ranged between 0.37 and 0.61 (Table 5). The numerically largest SNRs were seen for the enriched set with ApoE4 across all variants, and when we compared across subject sets, for the ADAS-Cog 5. However, the 95 % confidence intervals were broad (e.g., 0.33–0.88 for ADAS-Cog 5 on ApoE4+ set) and almost identical to ADAS-Cog 13 and thus did not suggest that SNRs differed substantially among the variants. Overall, estimated SNRs were relatively similar, when looking at both different variants and different enrichment strategies (Fig. 2). A more formal comparison based on paired t tests was done (Additional file 4).

Fig. 2
figure 2

Signal-to-noise ratios (SNRs) for reference patients with mild cognitive impairment (MCI) and subjects with mild Alzheimer’s disease (AD). SNRs were corrected for change from baseline (CFB) for reference patients with a MCI at 24 months and b mild AD at 12 months. Shaded area represents bootstrap 95 % confidence intervals (CIs) of SNR for 11-item Alzheimer’s Disease Assessment Scale–Cognitive subscale (ADAS-Cog11) for each enrichment group. Bounds of CIs are connected solely for display purposes. Abeta amyloid-β, ApoE4 apolipoprotein E ε4, CSF cerebrospinal fluid, p-Tau phosphorylated Tau, t-Tau total Tau, MMSE Mini Mental State Examination

For subjects with mild AD at the 12-month time point, the estimated SNR values ranged from 0.81 to 1.16 (Table 6). Similarly to MCI, the numerically highest SNRs were seen for the ApoE4-enriched set across all variants. Across all subject sets, the highest SNRs were seen for both the 5-item and 13-item variants. The highest SNR (1.16) was observed in the ApoE4-enriched set on ADAS-Cog 13. The 95 % confidence intervals were also broad and overlapping in mild AD.

SNRs are graphically displayed in Fig. 2. SNRs in mild AD (Fig. 2, right panel) were larger than in subjects with MCI (Fig. 2, left panel), indicating that all ADAS-Cog variants are more sensitive to detect changes in subjects with mild AD than in subjects with MCI. SNRs of the variants fall within the 95 % bootstrap confidence intervals of the ADAS-Cog 11 (Fig. 2, shaded area).

A more formal comparison of pairs of variants was done based on t statistics for the difference in SNRs. Resulting p values (corrected not for multiplicity, but for correlation between variants) are reported in Additional file 4: Table S4. As usual, small p values indicate a higher chance of a true difference between the particular variants.

Among the various MCI subsets, there is no evidence indicating a difference in SNR between any of the ADAS-Cog variants based on the available ADNI data. All p values were larger than 0.10 for comparisons to the original ADAS-Cog 11.

In mild AD, the strongest evidence for a difference based on p values (<0.10) was found when we compared the 13- vs. 11-item and the 5- vs. 3-item variants for the t-Tau- and p-Tau-enriched sets (Additional file 4). This indicates that the difference between variants may be driven by the two additional items (Delayed Word Recall and Digit Cancellation) that are not included in the 3- and 11-item variants. When the accepted enrichment strategy of t-Tau/Aβ ratio was used, the p values were above 0.10.

Discussion

ADAS-Cog has been the standard measure of cognition in AD clinical trials [20], but recently conducted trials with therapies aimed at slowing down disease progression in subjects at early stages of the disease revealed the limited sensitivity of the original ADAS-Cog 11 to detect change. Sensitivity to change may be compromised by the lack of tests to assess cognitive domains known to be impaired early in the disease, such as attention and executive functions [4], which has been addressed by adding tests to the original instrument [8] (e.g., the ADAS-Cog 13). A second approach to improve sensitivity of the instrument for early AD was to remove items prone to ceiling effects (e.g., ADAS-Cog 3). Data from subjects that score at ceiling inflate variance with negligible benefits in measuring change over time or treatment difference, particularly the possibility to detect improvement in cognition. The ADAS-Cog 3 extracted the items focused on memory (Word Recall, Orientation and Word Recognition) that are among the earliest manifestations of AD [4] and detect impairment in subjects with MCI approximately midway between normal healthy and patients with mild AD [6]. However, this approach is focused solely on a memory measure without considering other important areas of cognition affected at early stages of disease and therefore may not be optimal. The ADAS-Cog 5 combines both approaches and could theoretically be more sensitive than other variants to detect a change over time. It should be noted that our goal was not to validate a new instrument.

In the context of this study, we also sought to determine whether enrichment strategies could improve the sensitivity to change over time of these ADAS-Cog variants. Since the introduction of enrichment as part of inclusion criteria for early stages of AD in 2007 [21], enrichment biomarker strategies have been used in clinical trials to include subjects with prodromal AD (also called MCI due to AD). The enrichment strategy is aimed at detecting the presence of amyloidosis in the brain as a hallmark of AD [22, 23], for example by using CSF biomarkers [24] such as low Aβ and high Tau. As 60 % of subjects with MCI and 40 % of subjects with AD provided CSF, we used the available CSF baseline data to select enriched populations (i.e., subjects most resembling those to be recruited into clinical trials). This gave us the opportunity to also compare the performance of ADAS-Cog variants in enriched vs. non-enriched groups. For our analyses, t-Tau/Aβ ratio in the CSF was used as the primary enrichment strategy to determine populations with AD pathology (enriched). In addition, we looked at the impact of single-biomarker modalities such as Aβ alone, the marker of neurodegeneration Tau and the prominent genetic risk factor for sporadic AD, the ApoE4 allele [25]. It should be noted that these are not defined as enrichment biomarkers in the research criteria for prodromal AD [16].

There is an increasing body of evidence showing that AD starts decades before the clinical symptoms become apparent [26]. With new putative therapies aimed at slowing disease progression by early intervention, there is an increasing interest in identifying subjects at early stages of AD [27]. Advances in the biomarker field allow us to identify subjects with AD pathology before the clinical diagnosis is made [28]. It was hypothesised that using enriched populations in clinical trials reduces variability and increases the magnitude of cognitive decline over time, thus reducing the sample size necessary to detect a drug effect in clinical trials [29].

We analysed cognitive decline on the four ADAS-Cog variants (3-, 5-, 11- and 13-item scales) in ADNI subjects with MCI or mild AD who provided a CSF sample at baseline. The results showed that, in the MCI population, there was a minimal decline over 24 months on all ADAS-Cog variants. The impact of enrichment was detectable but subtle. The largest decline was less than 3 points over 24 months on ADAS-Cog 13 in enriched MCI. This provides some support for use of enrichment as a tool to identify subjects who are more likely to demonstrate a cognitive decline, although, at least in the case of the MCI, the impact upon such changes measured by ADAS-Cog variants is minimal and of questionable clinical relevance to be of any practical use for clinical trials with pharmacological intervention aimed at slowing decline of disease progression.

As expected, decline in mild AD was more pronounced than in MCI but still modest. ADAS-Cog 11 scores of subjects with mild AD declined 3.5 points over 12 months in our analysis. This seems to correspond to the rate of 2.4 points over 6 months reported by Doraiswamy et al. [1]. It should be noted that the rate of decline could be influenced by many factors, such as education, background medication and potentially attrition. The fact that ADNI participants were highly educated, with, on average, 15–16 years of education, has been reported previously [12]. In addition, many ADNI participants, particularly those with mild AD, may be receiving symptomatic treatment [30]. Even though these factors would be of interest, attempting evaluation of all these factors runs the risk that the ultimate sample size would be too small to provide any meaningful results. The impact of enrichment could not be evaluated, as over 90 % of subjects with mild AD enrolled in ADNI studies had positive AD pathology at baseline.

These results were also reflected on the MMSE scale: no worsening in the MCI set and modest worsening in the mild AD set over time. Meta-analysis of MMSE change in patients with mild to moderate AD has previously shown an annual rate of decline of 3.3 points [31]. Similar rates have been reported by other groups [32]. The rate of decline seen in our analysis is more modest, probably due to the relatively earlier stage of disease than in previously published studies. It should be noted, however, that the researchers in the above-mentioned studies analysed data of non-treated patients, while many ADNI participants, especially those with mild AD, were on background medication [30].

The SNR reflects the ratio between mean change and variability, thus allowing direct comparison of the sensitivity of the different ADAS-Cog variants. High SNR values reflect increased sensitivity to detect a change over time, and thus the ADAS-Cog variant with the highest SNR value should theoretically lead to an optimal instrument.

In MCI, the SNRs for each ADAS-Cog variant and for each enrichment strategy were relatively similar but low. The numerically largest SNRs were seen for the ADAS-Cog 5 and ApoE4 enrichment. However, as the 95 % confidence intervals around each of the SNRs were broad, the results did not support a meaningful difference among groups.

We confirmed that the ADAS-Cog instrument is not sensitive to detect change over time at pre-dementia stages of the disease such as MCI (as shown by small SNRs), and therefore the impact of enrichment strategies was subtle. The numerically largest SNR was seen for the enriched set according to ApoE4 risk. ApoE genotype is a well-described genetic risk factor for AD associated with early deficits in episodic recall, higher rates of cognitive decline before the diagnosis of MCI or AD, and age-related memory decline earlier in life (for review, see [33]). The presence of an ApoE4 allele doubles the risk of progression from normal cognition to onset of clinical symptoms [34]. In addition, the ApoE4 allele has been associated with a faster cognitive deterioration in several, but not all, studies of patients with AD. Therefore, ApoE4 genotype is often used as a stratification factor in clinical trials testing novel therapies and, interestingly, is being used as an enrichment strategy in the ongoing TOMMORROW trial [35]. However, as the 95 % confidence interval was broad, it can be concluded that the benefit of such enrichment in MCI seems negligible for this particular instrument. We do not believe that enrichment would allow for significant reduction of the required sample size in MCI if subjects were to be tested using ADAS-Cog.

The SNRs were larger in subjects with mild AD than in subjects with MCI. Similarly to MCI, the numerically highest SNRs were generally seen for the ApoE4-enriched set across all variants. Interestingly, the highest SNRs were seen for both the 5-item and 13-item ADAS-Cog variants. This was supported by the p values with the strongest evidence for a difference seen when comparing the 11- vs. 13-item variants and the 3- vs. 5-item variants. One could speculate that the difference between variants is driven more by the two additional items (Delayed Word Recall and Digit Cancellation) than by removing the items at ceiling in early stages of AD, suggesting the increase in magnitude of the change outweighs the increase in variability.

Limitations of this analysis

In addition to the high education level of ADNI participants [12, 36] discussed earlier, it is also acknowledged that the participating sites in the ADNI study are experienced. This is reflected by the observation that there is less variability in the scores and a more reliable diagnosis (>90 % subjects with mild AD had positive AD pathology). This is contrary to recent phase III clinical trials with potential anti-amyloid therapies, which have shown that enrolling participants without evidence of amyloid deposition is common [37].

In mild AD, the number of subjects attending follow-up visits beyond the first year dropped significantly and did not provide a sufficient number of subjects for meaningful analyses. Therefore, changes over time beyond 12 months need to be interpreted with caution. Per regulatory requirement, a minimum of 18 months of treatment is required in clinical trials aimed at disease modification.

It was the purpose of this investigation to analyse subjects with follow-up data. The impact of the clinical characteristics of subjects who withdrew prematurely on change over time was not evaluated, as this would have required assumptions regarding the follow-up data. Such assumptions are the subject of an ongoing scientific debate.

We recognize that the value of the ADAS-Cog variants in our dataset is being judged solely on the basis of a longitudinal change over time, regardless of the treatment status of included subjects. Therefore, the outcome may not necessarily directly translate into a variant with the highest sensitivity to detect differences between treatments.

It must also be noted that the ADAS-Cog variants analysed in this study were collected in the context of ADAS-Cog 11, so the items are likely prone to specific order effects. Moreover, the 13- and 5-item variants contained additional two items that were tested outside the original ADAS-Cog 11. The order effect, where exposure to earlier tests influences performance on later tests, is a well-known phenomenon in cognitive testing [38].

Conclusions

Whilst the cognitive domain expansion or reduction of the ADAS-Cog is in principle a worthwhile endeavour, in practice the use of these measures has been less successful. The outcome of our analyses showed that the possible utility of these novel ADAS-Cog variants is less than compelling, particularly in MCI. The impact of enrichment on cognitive decline in MCI using the latest research criteria [16] was subtle. In mild AD, adding items known to be impaired at early stages of the disease seems to provide more benefit than removing items on which subjects score close to ceiling.

Notes

  1. The text in italics in the Data source section is required by the ADNI consortium.

Abbreviations

Aβ:

amyloid-β

AD:

Alzheimer’s disease

ADAS-Cog:

Alzheimer’s Disease Assessment Scale–Cognitive subscale

ADNI:

Alzheimer’s Disease Neuroimaging Initiative

ApoE4 :

apolipoprotein E ε4

CDR-SB:

Clinical Dementia Rating Sum of Boxes

CFB:

change from baseline

CI:

confidence interval

CSF:

cerebrospinal fluid

IWG:

International Working Group

MCI:

mild cognitive impairment

MMSE:

Mini Mental State Examination

p-Tau:

phosphorylated Tau

SD:

standard deviation

SNR:

signal-to-noise ratio

t-Tau:

total Tau

References

  1. Doraiswamy PM, Kaiser L, Bieber F, Garman RL. The Alzheimer’s Disease Assessment Scale: evaluation of psychometric properties and patterns of cognitive decline in multicenter clinical trials of mild to moderate Alzheimer’s disease. Alzheimer Dis Assoc Disord. 2001;15:174–83.

    Article  CAS  PubMed  Google Scholar 

  2. Wesnes KA, Harrison JE. The evaluation of cognitive function in the dementias: methodological and regulatory considerations. Dialogues Clin Neurosci. 2003;5:77–88.

    PubMed Central  PubMed  Google Scholar 

  3. Vellas B, Andrieu S, Sampaio C, Coley N, Wilcock G. Endpoints for trials in Alzheimer’s disease: a European task force consensus. Lancet Neurol. 2008;7:436–50.

    Article  CAS  PubMed  Google Scholar 

  4. Weintraub S, Wicklund AH, Salmon DP. The neuropsychological profile of Alzheimer disease. Cold Spring Harb Perspect Med. 2012;2:a006171.

    Article  PubMed Central  PubMed  Google Scholar 

  5. Winblad B, Gauthier S, Scinto L, Feldman H, Wilcock GK, Truyen L, et al. Safety and efficacy of galantamine in subjects with mild cognitive impairment. Neurology. 2008;70:2024–35.

    Article  CAS  PubMed  Google Scholar 

  6. Grundman M, Petersen RC, Ferris SH, Thomas RG, Aisen PS, Bennett DA, et al. Mild cognitive impairment can be distinguished from Alzheimer disease and normal aging for clinical trials. Arch Neurol. 2004;61:59–66.

    Article  PubMed  Google Scholar 

  7. Hobart J, Cano S, Posner H, Selnes O, Stern Y, Thomas R, et al. Putting the Alzheimer’s cognitive test to the test I: traditional psychometric methods. Alzheimers Dement. 2013;9:S4–9.

    Article  PubMed  Google Scholar 

  8. Mohs RC, Knopman D, Petersen RC, Ferris SH, Ernesto C, Grundman M, et al. Development of cognitive instruments for use in clinical trials of antidementia drugs: additions to the Alzheimer’s Disease Assessment Scale that broaden its scope. The Alzheimer's Disease Cooperative Study. Alzheimer Dis Assoc Disord. 1997;11 Supp. 2:S13–21.

    Article  Google Scholar 

  9. Hansson O, Zetterberg H, Buchhave P, Andreasson U, Londos E, Minthon L, et al. Prediction of Alzheimer’s disease using the CSF Aβ42/Aβ40 ratio in patients with mild cognitive impairment. Dement Geriatr Cogn Disord. 2007;23:316–20.

    Article  CAS  PubMed  Google Scholar 

  10. Visser PJ, Verhey F, Knol DL, Scheltens P, Wahlund LO, Freund-Levi Y, et al. Prevalence and prognostic value of CSF markers of Alzheimer’s disease pathology in patients with subjective cognitive impairment or mild cognitive impairment in the DESCRIPA study: a prospective cohort study. Lancet Neurol. 2009;8:619–27.

    Article  PubMed  Google Scholar 

  11. Sperling RA, Aisen PS, Beckett LA, Bennett DA, Craft S, Fagan AM, et al. Toward defining the preclinical stages of Alzheimer’s disease: recommendations from the National Institute on Aging and the Alzheimer’s Association workgroup. Alzheimers Dement. 2011;7:280–92.

    Article  PubMed Central  PubMed  Google Scholar 

  12. Petersen RC, Aisen PS, Beckett LA, Donohue MC, Gamst AC, Harvey DJ, et al. Alzheimer’s Disease Neuroimaging Initiative (ADNI): clinical characterization. Neurology. 2010;74:201–9.

    Article  PubMed Central  PubMed  Google Scholar 

  13. Petersen RC, Smith GE, Waring SC, Ivnik RJ, Tangalos EG, Kokmen E. Mild cognitive impairment: clinical characterization and outcome. Arch Neurol. 1999;56:303–8.

    Article  CAS  PubMed  Google Scholar 

  14. McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease. Neurology. 1984;34:939–44.

    Article  CAS  PubMed  Google Scholar 

  15. Shaw LM, Vanderstichele H, Knapik-Czajka M, Clark CM, Aisen PS, Petersen RC, et al. Cerebrospinal fluid biomarker signature in Alzheimer’s Disease Neuroimaging Initiative subjects. Ann Neurol. 2009;65:403–13.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  16. Dubois B, Feldman HH, Jacova C, Hampel H, Molinuevo JL, Blennow K, et al. Advancing research diagnostic criteria for Alzheimer’s disease: the IWG-2 criteria. Lancet Neurol. 2014;13:614–29.

    Article  PubMed  Google Scholar 

  17. Raghavan N, Samtani MN, Farnum M, Yang E, Novak G, Grundman M, et al. The ADAS-Cog revisited: novel composite scales based on ADAS-Cog to improve efficiency in MCI and early AD trials. Alzheimers Dement. 2012;9:S21–31.

    Article  PubMed Central  PubMed  Google Scholar 

  18. Rosen WG, Mohs RC, Davis KL. A new rating scale for Alzheimer’s disease. Am J Psychiatry. 1984;141:1356–64.

    Article  CAS  PubMed  Google Scholar 

  19. Mohs RC, Cohen L. Alzheimer’s Disease Assessment Scale (ADAS). Psychopharmacol Bull. 1988;24:627–8.

    CAS  PubMed  Google Scholar 

  20. Mohs RC, Rosen WG, Davis KL. The Alzheimer’s disease assessment scale: an instrument for assessing treatment efficacy. Psychopharmacol Bull. 1983;19:448–50.

    CAS  PubMed  Google Scholar 

  21. Dubois B, Feldman HH, Jacova C, Dekosky ST, Barberger-Gateau P, Cummings J, et al. Research criteria for the diagnosis of Alzheimer’s disease: revising the NINCDS-ADRDA criteria. Lancet Neurol. 2007;6:734–46.

    Article  PubMed  Google Scholar 

  22. Hampel H, Shen Y, Walsh DM, Aisen P, Shaw LM, Zetterberg H, et al. Biological markers of amyloid-β-related mechanisms in Alzheimer’s disease. Exp Neurol. 2010;223:334–46.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  23. Blennow K, de Leon MJ, Zetterberg H. Alzheimer’s disease. Lancet. 2006;368:387–403.

    Article  CAS  PubMed  Google Scholar 

  24. Blennow K, Zetterberg H, Fagan AM. Fluid biomarkers in Alzheimer disease. Cold Spring Harb Perspect Med. 2012;2:a006221.

    Article  PubMed Central  PubMed  Google Scholar 

  25. Liu CC, Kanekiyo T, Xu H, Bu G. Apolipoprotein E and Alzheimer disease: risk, mechanisms and therapy. Nat Rev Neurol. 2013;9:106–18.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  26. Jack Jr CR, Knopman DS, Jagust WJ, Shaw LM, Aisen PS, Weiner MW, et al. Hypothetical model of dynamic biomarkers of the Alzheimer’s pathological cascade. Lancet Neurol. 2010;9:119–28.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  27. Sperling RA, Jack Jr CR, Aisen PS. Testing the right target and right drug at the right stage. Sci Transl Med. 2011;3:111cm33.

    Article  PubMed Central  PubMed  Google Scholar 

  28. Sperling R, Johnson K. Biomarkers of Alzheimer disease: current and future applications to diagnostic criteria. Continuum (Minneap Minn). 2013;19:325–38. A published erratum appears in Continuum (Minneap Minn). 2013;19:892.

    Google Scholar 

  29. Donohue MC, Gamst AC, Thomas RG, Xu R, Beckett L, Petersen RC, et al. The relative efficiency of time-to-threshold and rate of change in longitudinal data. Contemp Clin Trials. 2011;32:685–93.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  30. Schneider LS, Insel PS, Weiner MW. Treatment with cholinesterase inhibitors and memantine of patients in the Alzheimer’s Disease Neuroimaging Initiative. Arch Neurol. 2011;68:58–66.

    Article  PubMed Central  PubMed  Google Scholar 

  31. Han L, Cole M, Bellavance F, McCusker J, Primeau F. Tracking cognitive decline in Alzheimer’s disease using the Mini-Mental State Examination: a meta-analysis. Int Psychogeriatr. 2000;12:231–47.

    Article  CAS  PubMed  Google Scholar 

  32. Mendiondo MS, Ashford JW, Kryscio RJ, Schmitt FA. Modelling Mini Mental State Examination changes in Alzheimer’s disease. Stat Med. 2000;19:1607–16.

    Article  CAS  PubMed  Google Scholar 

  33. Bookheimer S, Burggren A. APOE-4 genotype and neurophysiological vulnerability to Alzheimer’s and cognitive aging. Annu Rev Clin Psychol. 2009;5:343–62.

    Article  PubMed Central  PubMed  Google Scholar 

  34. Albert M, Soldan A, Gottesman R, McKhann G, Sacktor N, Farrington L, et al. Cognitive changes preceding clinical symptom onset of mild cognitive impairment and relationship to ApoE genotype. Curr Alzheimer Res. 2014;11:773–84.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  35. Roses AD, Saunders AM, Lutz MW, Zhang N, Hariri AR, Asin KE, et al. New applications of disease genetics and pharmacogenetics to drug development. Curr Opin Pharmacol. 2014;14:81–9.

    Article  CAS  PubMed  Google Scholar 

  36. Grill JD, Di L, Lu PH, Lee C, Ringman J, Apostolova LG, et al. Estimating sample sizes for predementia Alzheimer’s trials based on the Alzheimer’s Disease Neuroimaging Initiative. Neurobiol Aging. 2013;34:62–72.

    Article  PubMed Central  PubMed  Google Scholar 

  37. Vellas B, Carrillo MC, Sampaio C, Brashear HR, Siemers E, Hampel H, et al. Designing drug trials for Alzheimer’s disease: what we have learned from the release of the phase III antibody trials: a report from the EU/US/CTAD Task Force. Alzheimers Dement. 2013;9:438–44.

    Article  PubMed  Google Scholar 

  38. Cozby PC. Methods in behavioral research. 10th ed. New York: McGraw-Hill; 2009.

    Google Scholar 

Download references

Acknowledgements

Editorial support and formatting assistance were provided by CACTUS Communications, which was contracted and compensated by Boehringer Ingelheim Pharma GmbH & Co. KG for these services.

The authors meet criteria for authorship as recommended by the International Committee of Medical Journal Editors (ICMJE), were fully responsible for all content and editorial decisions and were involved at all stages of manuscript development. Data used in the preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in the analysis or writing of this report. A complete listing of ADNI investigators can be found at http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf.

Sources of funding

Data collection and sharing for this project was funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen Idec Inc.; Bristol-Myers Squibb Company; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC; Johnson & Johnson Pharmaceutical Research & Development LLC; Medpace, Inc.; Merck & Co., Inc.; Meso Scale Diagnostics, LLC; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Synarc Inc.; and Takeda Pharmaceutical Company. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health ( www.fnih.org ). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Disease Cooperative Study at the University of California, San Diego. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.

Ethical approval

Ethical approval was obtained from the institutional review boards of each institution involved: Oregon Health and Science University; University of Southern California; University of California, San Diego; University of Michigan; Mayo Clinic, Rochester, MN, USA; Baylor College of Medicine; Columbia University; Washington University in St. Louis; University of Alabama–Birmingham; Mount Sinai School of Medicine; Rush University Medical Center; Wien Center; The Johns Hopkins University; University of South Florida Health Byrd Alzheimer’s Institute; New York University; Duke University Medical Center; University of Pennsylvania; University of Kentucky; University of Pittsburgh; University of Rochester Medical Center; University of California, Irvine; University of Texas Southwestern Medical Center; Emory University; University of Kansas; University of California, Los Angeles; Mayo Clinic, Jacksonville, FL, USA; Indiana University; Yale University School of Medicine; Jewish General Hospital/McGill University; Sunnybrook Health Sciences Centre; University of British Columbia; St. Joseph’s Hospital, Ontario, Canada; Northwestern University; Nathan S. Kline Institute for Psychiatric Research; Premiere Research Institute; University of California, San Francisco; Georgetown University; Brigham and Women’s Hospital; Stanford University; Banner Sun Health Research Institute; Boston University School of Medicine; Howard University; Case Western Reserve University; University of California, Davis; DENT Neurologic Institute; Parkwood Hospital; University of Wisconsin; University of California, Irvine Brain Imaging Center; Banner Alzheimer’s Institute; The Ohio State University; Albany Medical College; University of Iowa; Dartmouth-Hitchcock Medical Center; Wake Forest University Health Sciences Center; Rhode Island Hospital; Cornell Medical Center; Cleveland Clinic Lou Ruvo Center for Brain Health (CCLRBC); Roper St. Francis Hospital; and Butler Hospital Memory and Aging Program. The information on ethical approval and the centres involved in the ADNI study as listed above was obtained from the ADNI Data and Publications Committee. The authors of this article used the publicly available ADNI data but were not involved in the conduct of the study.

Author information

Authors and Affiliations

Authors

Consortia

Corresponding author

Correspondence to Jana Podhorna.

Additional information

Competing interests

JP and MS are employees of Boehringer Ingelheim Pharma GmbH & Co. KG. TK undertakes work for Boehringer Ingelheim Pharma GmbH & Co. KG as an external contractor. JH has previously received honoraria and consultancy payments from Boehringer Ingelheim but not for work on this manuscript. Additionally, in the past 3 years, JH has received consultancy payments from the following companies and organisations: AbbVie, Amgen, Anavex Life Sciences, AstraZeneca, Avraham Pharma, AXON Neuroscience, Axovant Sciences, Biogen Idec, Bracket, Catenion, CRF Health, DeNDRoN, FORUM (formerly EnVivo) Pharmaceuticals, ePharmaSolutions, Eisai Co., Eli Lilly and Company, Games for Health Europe, Heptares Therapeutics, Janssen Alzheimer Immunotherapy, Johnson & Johnson, Kaasa health, Kyowa Hakko Kirin Co., H. Lundbeck A/S, MedAvante, Merck & Co., MyCognition, Mind Agilis, NeuroCog Trials, Neurim Pharmaceuticals, Neurotrack, Novartis, Nutricia, Orion Pharma, PharmaNet/i3, Pfizer, Prana Biotech, PriceSpective, Probiodrug, ProPhase, ProStrakan, Reviva Pharmaceuticals, Roche, Servier, Shire, Takeda Pharmaceutical Co., TCG Lifesciences, TransTech Pharma and Velacor Therapeutics. JH has also received payment for continuing medical education activities from Medscape and H. Lundbeck, and holds options and a patent in Neurotrack and MyCognition, respectively. JH has received royalties from Oxford University Press and Blackwell Publishing, and research funding from Memorabel. The authors declare that they have no non-financial competing interests.

Authors’ contributions

TK and MS acquired and analysed the patient data, having selected patients according to set criteria. JP and JH guided the project and drafted the manuscript on the basis of the data produced. All four authors worked together to interpret the results and revise the manuscript. All authors read and approved the final manuscript.

Additional files

Additional file 1:

Items of ADAS-Cog variants. Supplementary table providing details about items of ADAS-Cog variants. (DOCX 15 kb)

Additional file 2:

ADAS-Cog 11 scores at each visit and change from baseline for subjects with mild AD. Supplementary table providing details of ADAS-Cog 11 scores at each visit and change from baseline for subjects with mild AD. (DOCX 15 kb)

Additional file 3:

MMSE score up to 24 months for subjects with MCI and mild AD. Supplementary table providing details of MMSE score up to 24 months for subjects with MCI and mild AD. (DOCX 14 kb)

Additional file 4:

P values for comparison of signal-to-noise ratios of ADAS-Cog 11 and ADAS-Cog 5 vs. other variants. Supplementary table providing p values for comparison of signal-to-noise ratios of ADAS-Cog 11 and ADAS-Cog 5 vs. other variants. (DOCX 16 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Podhorna, J., Krahnke, T., Shear, M. et al. Alzheimer’s Disease Assessment Scale–Cognitive subscale variants in mild cognitive impairment and mild Alzheimer’s disease: change over time and the effect of enrichment strategies. Alz Res Therapy 8, 8 (2016). https://0-doi-org.brum.beds.ac.uk/10.1186/s13195-016-0170-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13195-016-0170-5

Keywords