open-discussion
open-discussion > Frequently asked questions
Dec 11, 2014 09:12 AM | Matthew Kempton
Frequently asked questions
I thought it would be useful if I copied the FAQ from the website
here:
Frequently asked Questions
Q) What is the Brain Segmentation Testing Protocol?
A) The brain segmentation testing protocol is a freely available collection of 3D MRI data. The images could be used for a variety of purposes but we think they will be especially useful for testing algorithms which determine the volume or shape of brain structures.
Q) Why do I get a message saying "Error This accounts public links are generating too much traffic and have been disabled!"
A) Try using the mirror site links. We use a combination of Dropbox and SugarSync to store the MRI files and there is a download bandwidth limit on each service, so if too many people download the files this error occurs. If you see this error, try the mirror link and if this still doesn't work try again in a day or two and you should be able to access the files. Note that the longitudinal images are kept on a different account than the other images so if the longitudinal images don't work the other might and vice versa. If you are still having problems contact me and I should be able to send the files to you.
Q) I have developed a brain segmentation algorithm, I've downloaded the data, now how do I test my algorithm?
A) In our paper published in Neuroimage we demonstrate an example of how the data could be used to test an algorithm. Firstly you could check how often your algorithm fails – if your algorithm can process most of the data it is likely it will be able to deal with most structural MRI images. You can check that the algorithm works in adults, patients with Alzheimer's disease and infants. We would like to provide manual segmentations for the validity dataset, and currently manually segmented lateral ventricle volume is available.
Using the reliability data you can check that your algorithm gives consistent results. For example using the 'same scanner and same sequence' dataset, you would expect a segmentation program to produce the same or very similar results. You can also check that your segmentation algorithm gives comparable data when different types of images are used. Using the sensitivity dataset it is possible to check that the algorithm is able to detect the effect of aging in a longitudinal dataset of subjects, each scanned over a 2 year period. In our paper we did this in two ways. We checked that volume changes were apparent between the first and second scan (paired t-test) and also examined whether there was a correlation between the volume change and the inter-scan interval. Using this data you could also compare the performance of your segmentation algorithm against more established packages such as Freesurfer.
Q) Why do some images have their faces removed and others don't?
We have removed faces from our data using MRI defacer to preserve subject anonymity. The infant dataset and OASIS longitudinal dataset have already been made publically available with faces intact so we have included these.
Q) Why are the 'Young adults' and 'Longitudinal images' more noisier than the other images?
A) The Young adult dataset was acquired in the early 2000s so is of lower quality but was chosen because we had lateral ventricle volumes on this dataset. The longitudinal images were taken from the OASIS longitudinal dataset which includes 3 images taken at each time point. We could have combined these 3 images to create an image with a higher signal to noise ratio (SNR), but we had concerns about registering the images correctly. One advantage of having noisier data in the test set is that it mirrors the fact that segmentation algorithms often face noisy data in the real world.
Q) Two sets of images in the reproducibility dataset (different MRI sequences) set A4 and B4 seem to have reduced intensity near the back and front of the brain.
A) We used 8, T1 weighted sequences used or being developed at our centre. The A4 and B4 sequences were routinely used coronal sequences which had been developed with the temporal lobe in mind. They should be ok for assessing most segmentation algorithms but you may want to experiment with leaving these two out if you are interested in segmentations including the frontal pole or occipital lobe.
Q) How do I cite this work?
A) If you have validated your algorithm using this test set or used it for something else please cite our Neuroimage paper:
Kempton MJ, Underwood TS, Brunton S, Stylios F, Schmechtig A, Ettinger U, Smith MS, Lovestone S, Crum WR, Frangou S, Williams SC, Simmons A (2011) A comprehensive testing protocol for MRI neuroanatomical segmentation techniques: Evaluation of a novel lateral ventricle segmentation method. Neuroimage. 2011 Oct 15;58(4):1051-9.
If you use the infant images and OASIS images please make sure you cite their relevant paper(s) and grant numbers as outlined in their conditions
Q) Do you have any other neuroimaging websites?
Yes I have been involved in creating:
-The Bipolar Disorder Neuroimaging Database which is an online database and meta-analysis of brain structure in Bipolar Disorder
-The Major Depressive Disorder Neuroimaging Database which is an online database and meta-analysis of brain structure in Major Depression
-ALVIN lateral ventricle segmentation, an SPM8 extension for segmenting the lateral ventricles which has been tested using the brain segmentation testing protocol
Matthew Kempton
Frequently asked Questions
Q) What is the Brain Segmentation Testing Protocol?
A) The brain segmentation testing protocol is a freely available collection of 3D MRI data. The images could be used for a variety of purposes but we think they will be especially useful for testing algorithms which determine the volume or shape of brain structures.
Q) Why do I get a message saying "Error This accounts public links are generating too much traffic and have been disabled!"
A) Try using the mirror site links. We use a combination of Dropbox and SugarSync to store the MRI files and there is a download bandwidth limit on each service, so if too many people download the files this error occurs. If you see this error, try the mirror link and if this still doesn't work try again in a day or two and you should be able to access the files. Note that the longitudinal images are kept on a different account than the other images so if the longitudinal images don't work the other might and vice versa. If you are still having problems contact me and I should be able to send the files to you.
Q) I have developed a brain segmentation algorithm, I've downloaded the data, now how do I test my algorithm?
A) In our paper published in Neuroimage we demonstrate an example of how the data could be used to test an algorithm. Firstly you could check how often your algorithm fails – if your algorithm can process most of the data it is likely it will be able to deal with most structural MRI images. You can check that the algorithm works in adults, patients with Alzheimer's disease and infants. We would like to provide manual segmentations for the validity dataset, and currently manually segmented lateral ventricle volume is available.
Using the reliability data you can check that your algorithm gives consistent results. For example using the 'same scanner and same sequence' dataset, you would expect a segmentation program to produce the same or very similar results. You can also check that your segmentation algorithm gives comparable data when different types of images are used. Using the sensitivity dataset it is possible to check that the algorithm is able to detect the effect of aging in a longitudinal dataset of subjects, each scanned over a 2 year period. In our paper we did this in two ways. We checked that volume changes were apparent between the first and second scan (paired t-test) and also examined whether there was a correlation between the volume change and the inter-scan interval. Using this data you could also compare the performance of your segmentation algorithm against more established packages such as Freesurfer.
Q) Why do some images have their faces removed and others don't?
We have removed faces from our data using MRI defacer to preserve subject anonymity. The infant dataset and OASIS longitudinal dataset have already been made publically available with faces intact so we have included these.
Q) Why are the 'Young adults' and 'Longitudinal images' more noisier than the other images?
A) The Young adult dataset was acquired in the early 2000s so is of lower quality but was chosen because we had lateral ventricle volumes on this dataset. The longitudinal images were taken from the OASIS longitudinal dataset which includes 3 images taken at each time point. We could have combined these 3 images to create an image with a higher signal to noise ratio (SNR), but we had concerns about registering the images correctly. One advantage of having noisier data in the test set is that it mirrors the fact that segmentation algorithms often face noisy data in the real world.
Q) Two sets of images in the reproducibility dataset (different MRI sequences) set A4 and B4 seem to have reduced intensity near the back and front of the brain.
A) We used 8, T1 weighted sequences used or being developed at our centre. The A4 and B4 sequences were routinely used coronal sequences which had been developed with the temporal lobe in mind. They should be ok for assessing most segmentation algorithms but you may want to experiment with leaving these two out if you are interested in segmentations including the frontal pole or occipital lobe.
Q) How do I cite this work?
A) If you have validated your algorithm using this test set or used it for something else please cite our Neuroimage paper:
Kempton MJ, Underwood TS, Brunton S, Stylios F, Schmechtig A, Ettinger U, Smith MS, Lovestone S, Crum WR, Frangou S, Williams SC, Simmons A (2011) A comprehensive testing protocol for MRI neuroanatomical segmentation techniques: Evaluation of a novel lateral ventricle segmentation method. Neuroimage. 2011 Oct 15;58(4):1051-9.
If you use the infant images and OASIS images please make sure you cite their relevant paper(s) and grant numbers as outlined in their conditions
Q) Do you have any other neuroimaging websites?
Yes I have been involved in creating:
-The Bipolar Disorder Neuroimaging Database which is an online database and meta-analysis of brain structure in Bipolar Disorder
-The Major Depressive Disorder Neuroimaging Database which is an online database and meta-analysis of brain structure in Major Depression
-ALVIN lateral ventricle segmentation, an SPM8 extension for segmenting the lateral ventricles which has been tested using the brain segmentation testing protocol
Matthew Kempton