Criteria | Salarian et al[42] | Motoi et al[39] | Zwartjes et al[45] | Lau et al[34] | Barth et al[31] | Yang et al[43] | Cancela et al[32] | Moore et al[38] | Dobkin et al[33] |
---|---|---|---|---|---|---|---|---|---|
Customized scale items | |||||||||
Internal validity method | Â | Â | Â | Â | Â | Â | Â | Â | Â |
1. Baseline characteristics described | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 1 | 1 |
2. System and devices are clearly described | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 |
3. Measurement protocol is clearly described for each group to allow replication | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 1 | 1 |
4. Methods of analysis clearly described | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
5. Classifier(s) are evaluated | 1 | n/a | 0 | 1 | 1 | n/a | 1 | n/a | 0 |
6. Statistical methods used to test reproducibility | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 1 |
7. Reported accuracy metrics | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 |
8. Reported confidence intervals for classifier performance | 0 | n/a | 0 | 0 | 0 | n/a | 0 | n/a | 0 |
9. Study limitations described | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
10. Interpretation of the results | 0 | 0 | 1 | 0 | 1 | 1 | 0 | 1 | 1 |
Construct validity method | Â | Â | Â | Â | Â | Â | Â | Â | Â |
11. Content validity | 1 | 0 | 148 | 0 | 0 | 0 | 1 | 0 | 0 |
12. Criterion-related validity is obtained | 1 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 1 |
13. Cross-validation (i.e. test and training set) | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 |
Total score | 9 | 4 | 8 | 9 | 8 | 6 | 5 | 6 | 7 |