indi > ADHD-200 Global Competition Scoring Policy
Apr 5, 2011  03:04 AM | Michael Milham
ADHD-200 Global Competition Scoring Policy
ADHD-200 Global Competition: Classification Scoring Policy            [/b]

 

On July 1st, 2011, an unlabeled test set will be release for evaluation of entries to the classification portion of the ADHD-200 Global Competition. Given that the test set is constructed from imaging studies, it is not a community representative sample, and as such, the prevalence of ADHD in the test set should be anticipated to be notably higher than the commonly cited prevalence of 8% in the community (anticipate that at least 30% of the test sample will be ADHD cases).

 
Among individuals with ADHD, sufficient numbers of combined type and inattentive type datasets will be included to allow determination of a classifier’s ability to distinguish between the two ADHD subtypes. The test set will be released in identical format to that provided in the training set released on March 1, 2010, with the same accompanying phenotypic information (age, sex, imaging site and FS-IQ), except no diagnostic labels or ADHD symptoms severity measures will be provided. Data from imaging sites included in the training set will be labeled with the imaging site at which they were obtained – importantly, approximately 20% of the test dataset will come from imaging sites not included in the initial release. 

The winning entry will be selected based upon classification accuracy, using the following point values:
Correct TDC: 1 point

Correct ADHD classification, correct subtype: 1 point

Correct ADHD classification, incorrect subtype: 0.5 points

There will be no penalty for inaccurate classifications, and total point value will be used to identify the winner.  Participants are welcomed to build classifiers using the resting state fMRI data, anatomical data, available phenotypic data, or any combination. R-fMRI datasets will be quality controlled such that motion is limited to a maximum Euclidean displacement of 4mm.

The primary goal of the competition is to determine the maximum potential utility of brain imaging data. However, in the case of a tie-break situation, computational complexity will be considered. As such, all entries should be accompanied by a brief calculation of computational complexity using Big O notation (http://en.wikipedia.org/wiki/Big_O_notation).

Any questions should be addressed to the competition coordinators: Damien Fair and Mike Milham

Threaded View

TitleAuthorDate
ADHD-200 Global Competition Scoring Policy
Michael Milham Apr 5, 2011
Paolo Avesani Apr 6, 2011
Michael Milham Apr 6, 2011