Skip to main content

Table 3 Classifier performance using only morphometric features for classifiers using ADNI subjects with IR-SPGR or MPRAGE scans

From: Early diagnosis of Alzheimer’s disease using machine learning: a multi-diagnostic, generalizable approach

Experiment

Training set

Testing set

Classification task

MCC

[CI: 95%]

BAC

[CI: 95%]

ROC AUC

[CI: 95%]

Sens

[CI: 95%]

Spec

[CI: 95%]

PPV (prevalence)

[CI: 95%]

NPV

(prevalence)

[CI: 95%]

PPV

(standard)

[CI: 95%]

NPV

(standard)

[CI: 95%]

TN

FP

FN

TP

MCC

p-value

(vs. w/ GT)

A1

ADNI MPRAGE

(N=295)

ADNI MPRAGE

(N=128)

HC vs. MCI

0.056

[−0.159;0.259]

52.8%

[42.1%; 62.9%]

58.2%

[46.5%; 70.1%]

50.0%

[35.0%; 64.6%]

55.6%

[41.5%; 69.6%]

33.3%

[21.0%; 48.5%]

71.5%

[59.0%; 81.6%]

53.0%

[37.4%; 68.0%]

52.7%

[39.0%; 66.3%]

25

20

22

22

0.060

HC vs. AD

0.814

[0.679; 0.927]

90.8%

[84.1%; 96.2%]

97.3%

[93.6%; 99.6%]

94.9%

[87.2%; 100.0%]

86.7%

[76.2%; 95.5%]

81.7%

[69.6%; 93.3%]

96.4%

[90.5%; 100.0%]

87.7%

[78.6%; 95.7%]

94.4%

[85.6%; 100.0%]

39

6

2

37

0.974

MCI vs. AD

0.588

[0.424; 0.751]

79.0%

[70.7%; 87.5%]

87.9%

[79.9%; 95.3%]

89.7%

[78.6%; 97.7%]

68.2%

[54.3%; 82.4%]

80.0%

[70.9%; 88.7%]

82.4%

[64.2%; 96.2%]

73.8%

[63.2%; 84.7%]

86.9%

[71.7%; 97.3%]

30

14

4

35

0.627

HC vs. MCI vs. AD

0.350

[0.227; 0.480]

57.7%

[50.2%; 65.2%]

76.6%

[70.8%; 82.3%]

n/a

n/a

n/a

n/a

n/a

n/a

n/a

n/a

n/a

n/a

0.674

A2

ADNI IR-SPGR

(N=106)

ADNI IR-SPGR

(N=46)

HC vs. MCI

0.087

[−0.267; 0.457]

53.8%

[38.7%; 70.4%]

69.2%

[51.2%; 86.0%]

28.6%

[7.1%; 54.5%]

78.9%

[61.1%; 95.0%]

37.5%

[7.5%; 82.8%]

71.4%

[59.8%; 82.5%]

57.5%

[15.4%; 91.6%]

52.5%

[39.7%; 67.6%]

15

4

10

4

0.860

HC vs. AD

0.607

[0.313; 0.867]

79.4%

[65.0%; 92.9%]

93.5%

[82.7%; 100.0%]

69.2%

[44.4%; 92.9%]

89.5%

[75.0%; 100.0%]

80.5%

[52.6%; 100.0%]

82.3%

[68.3%; 95.7%]

86.8%

[64.0%; 100.0%]

74.4%

[57.4%; 93.4%]

17

2

4

9

0.384

MCI vs. AD

0.408

[−0.028; 0.742]

70.1%

[51.5%; 86.7%]

77.5%

[55.1%; 94.0%]

61.5%

[33.3%; 86.7%]

78.6%

[50.0%; 100.0%]

80.3%

[48.5%; 100.0%]

59.1%

[34.6%; 84.2%]

74.2%

[40.0%; 100.0%]

67.1%

[42.8%; 88.3%]

11

3

5

8

0.737

HC vs. MCI vs. AD

0.263

[0.054; 0.491]

49.8%

[37.1%; 62.9%]

77.1%

[66.4%; 87.5%]

n/a

n/a

n/a

n/a

n/a

n/a

n/a

n/a

n/a

n/a

0.996

A3

ADNI MPRAGE

(N=423)

ADNI IR-SPGR

(N=147)

HC vs. MCI

0.268

[0.093; 0.423]

61.9%

[54.2%; 69.4%]

73.5%

[62.9%; 82.7%]

88.4%

[77.8%; 97.4%]

35.5%

[23.9%; 48.3%]

37.8%

[31.2%; 45.5%]

87.4%

[70.8%; 97.7%]

57.8%

[50.6%; 65.3%]

75.4%

[51.8%; 94.9%]

22

40

5

38

0.816

HC vs. AD

0.745

[0.612; 0.867]

87.6%

[80.7%; 94.0%]

96.3%

[92.7%; 98.9%]

88.1%

[77.8%; 97.2%]

87.1%

[77.9%; 95.1%]

81.0%

[68.8%; 92.5%]

92.1%

[84.9%; 98.2%]

87.2%

[77.9%; 95.2%]

88.0%

[77.8%; 97.1%]

54

8

5

37

0.367

MCI vs. AD

0.647

[0.480; 0.805]

82.4%

[74.1%; 90.1%]

87.4%

[79.8%; 90.1%]

83.3%

[71.4%; 93.8%]

81.4%

[68.9%; 92.9%]

86.4%

[76.5%; 94.9%]

77.5%

[63.0%; 91.3%]

81.7%

[69.7%; 92.9%]

83.0%

[70.7%; 93.7%]

35

8

7

35

0.496

HC vs. MCI vs. AD

0.424

[0.312; 0.533]

61.7%

[54.6%; 68.8%]

81.6%

[75.6%; 87.3%]

n/a

n/a

n/a

n/a

n/a

n/a

n/a

n/a

n/a

n/a

0.618

A4

ADNI IR-SPGR

(N=147)

ADNI MPRAGE

(N=423)

HC vs. MCI

0.178

[0.074; 0.287]

57.1%

[53.0%; 61.7%]

65.6%

[59.5%; 72.0%]

26.9%

[20.4%; 34.1%]

87.2%

[81.9%;92.3%]

48.2%

[33.3%; 66.2%]

72.9%

[69.9%; 76.0%]

67.8%

[53.0%; 81.6%]

54.4%

[50.7%; 58.3%]

##

19

##

39

0.462

HC vs. AD

0.711

[0.631; 0.792]

85.4%

[81.2%; 89.4%]

93.5%

[90.7%; 95.9%]

82.2%

[75.0%; 88.1%]

88.6%

[83.3%; 93.3%]

81.9%

[73.8%;89.2%]

88.8%

[84.2%; 92.6%]

87.8%

[81.8%; 92.9%]

83.3%

[76.9%; 88.7%]

##

17

23

##

0.893

MCI vs. AD

0.454

[0.353; 0.551]

71.7%

[66.6%; 76.5%]

81.4%

[76.2%; 86.3%]

87.6%

[81.4%; 92.9%]

68.3%

[56.5%; 70.9%]

73.7%

[68.9%; 78.5%]

76.16%

[64.6%; 86.4%]

66.5%

[61.0%; 72.0%]

81.8%

[72.1%; 90.0%]

81

64

16

113

0.114

HC vs. MCI vs. AD

0.372

[0.307; 0.431]

57.0%

[53.4%; 60.3%]

75.5%

[72.2%; 78.8%]

n/a

n/a

n/a

n/a

n/a

n/a

n/a

n/a

n/a

n/a

0.492

A5

ADNI IR-SPGR

and

ADNI MPRAGE

(N=379)

ADNI IR-SPGR

and

ADNI MPRAGE

(N=164)

HC vs. MCI

0.265

[0.085; 0.441]

62.6%

[54.0%; 71.0%]

64.4%

[53.7%; 74.1%]

47.2%

[34.5%; 60.4%]

78.0%

[66.7%; 87.7%]

48.7%

[31.5%; 68.5%]

76.9%

[69.7%; 83.3%]

68.2%

[50.9%; 83.1%]

59.6%

[50.5%; 68.9%]

46

13

28

25

0.404

HC vs. AD

0.789

[0.669; 0.892]

89.5%

[83.7%; 94.7%]

96.0%

[92.3%; 98.8%]

94.2%

[87.0%; 100.0%]

84.7%

[75.4%; 93.8%]

79.4%

[68.9%; 91.0%]

95.9%

[90.3%; 100.0%]

86.0%

[78.0%; 94.2%]

93.6%

[85.3%; 100.08%]

50

9

3

49

0.883

MCI vs. AD

0.592

[0.433; 0.734]

79.1%

[71.0%; 86.2%]

88.1%

[80.6%; 93.6%]

88.5%

[79.2%; 96.2%]

69.8%

[55.9%; 81.1%]

80.6%

[71.8%; 87.8%]

81.1%

[65.5%; 93.8%]

74.6%

[64.2%; 83.6%]

85.9%

[72.9%; 95.5%]

37

16

6

46

0.774

HC vs. MCI vs. AD

0.438

[0.330;0.541]

62.1%

[55.5%; 68.2%]

77.7%

[72.2%; 82.6%]

n/a

n/a

n/a

n/a

n/a

n/a

n/a

n/a

n/a

n/a

0.505

  1. MCC p-value refers to the p-value for the MCC metric for the comparison with the equivalent classifier (i.e., same training and test sets) with morphometric and GT features used as input. PPV/NPV “prevalence” are calculated with an MCI prevalence of 30.7% for the “HC vs. MCI” classifiers; an AD prevalence of 38.5% for the “HC vs. AD” classifiers; and an AD prevalence of 58.6% for the MCI vs. AD classifier (these correspond to the relative prevalence of the positive class based on prevalence estimates from the first visit in the clinical setting of 42.0% for HC, 18.6% for MCI, and 26.3% for AD) [48]. PPV/NPV “standard” are calculated with a prevalence of 50% to allow comparison with other studies
  2. Legend: CI confidence interval, MCC Matthew’s correlation coefficient, ROC AUC area under the receiver operating characteristic curve, BAC balanced accuracy, Sens sensitivity, Spec specificity, PPV positive predict value, NPV negative predictive value, TN true negatives, FP false positives, FN false negatives, TP true positives