help
help > RE: false positives cluster level of inference
Jul 13, 2016 01:07 AM | Alfonso Nieto-Castanon - Boston University
RE: false positives cluster level of inference
Hi Jeff&Mike,
My reading from the Eklund et al. and Flanding&Friston papers is that if you are using a voxel-wise height threshold p<.001 (this is the default threshold both in SPM and in CONN) then using parametric statistics is perfectly fine, while nonparametric statistics are recommended when you want to use higher (i.e. more liberal) voxel-wise height thresholds (e.g. p<.01, in order to focus on perhaps weaker but large/distributed responses). So yes, either one of the following two approaches should be perfectly fine (the former being more sensitive for strong localized effects, while the latter being more sensitive for weak distributed effects):
height threshold p-unc<.001, cluster-size threshold p-FDR<.05, parametric statistics
height threshold p-unc<.01, cluster-mass threshold p-FDR<.05, non-parametric statistics
Regarding how to use non-parametric statistics in CONN, simply select 'non-parametric statistics' in the explorer window top-right corner menu. Everything works as in the 'parametric statistics' case, only now the choice of statistics being displayed and the associated thresholds that you can use are slightly different. In particular, the information displayed for each cluster when selecting parametric statistics is:
cluster position: MNI coordinates of largest peak within this cluster
cluster size: number of voxels in this cluster
cluster p-FWE: family-wise error corrected p-value (probability under the null hypothesis of observing one or more clusters of at least this size across the entire brain)
cluster p-FDR: false discovery rate corrected p-value (expected proportion under the null hypothesis of false discoveries among clusters of at least this size)
cluster p-unc: uncorrected p-value (probability under the null hypothesis of a randomly-selected cluster having at least this size)
peak p-FWE: family-wise error corrected p-value (probability under the null hypothesis of observing one or more peaks of at least this height across the entire brain)
peak p-unc: uncorrected p-value (probability under the null hypothesis of a randomly-selected peak having at least this height)
all of the above p-values are obtained using random-field-theory (RFT) assumptions.
When selecting instead non-parametric statistics (and after the corresponding permutation/randomization tests are run) you will then get the following information for each cluster:
cluster position: MNI coordinates of largest peak within this cluster
cluster size: number of voxels in this cluster
cluster p-FWE: family-wise error corrected p-value (probability under the null hypothesis of observing one or more clusters of at least this size across the entire brain)
cluster p-FDR: false discovery rate corrected p-value (expected proportion under the null hypothesis of false discoveries among clusters of at least this size)
cluster p-unc: uncorrected p-value (probability under the null hypothesis of a randomly-selected cluster having at least this size)
cluster mass: sum of statistics (F-values or T^2 values) across all voxels within this cluster
cluster p-FWE: family-wise error corrected p-value (probability under the null hypothesis of observing one or more clusters of at least this mass across the entire brain)
cluster p-FDR: false discovery rate corrected p-value (expected proportion under the null hypothesis of false discoveries among clusters of at least this mass)
cluster p-unc: uncorrected p-value (probability under the null hypothesis of a randomly-selected cluster having at least this mass)
and all of the above p-values are obtained using non-parametric assumptions (permutation/randomization analyses). Cluster mass statistics combine information about each cluster size as well as each cluster height/strength, so they are generally considered more sensitive than either cluster-size or peak-level statistics. Typically for non-parametric statistics I would recommend using a cluster-mass p-FDR<.05 threshold by default, since that should typically be one of the most sensitive tests, but your preferences might vary.
Hope this helps
Alfonso
Originally posted by Jeff Browndyke:
My reading from the Eklund et al. and Flanding&Friston papers is that if you are using a voxel-wise height threshold p<.001 (this is the default threshold both in SPM and in CONN) then using parametric statistics is perfectly fine, while nonparametric statistics are recommended when you want to use higher (i.e. more liberal) voxel-wise height thresholds (e.g. p<.01, in order to focus on perhaps weaker but large/distributed responses). So yes, either one of the following two approaches should be perfectly fine (the former being more sensitive for strong localized effects, while the latter being more sensitive for weak distributed effects):
height threshold p-unc<.001, cluster-size threshold p-FDR<.05, parametric statistics
height threshold p-unc<.01, cluster-mass threshold p-FDR<.05, non-parametric statistics
Regarding how to use non-parametric statistics in CONN, simply select 'non-parametric statistics' in the explorer window top-right corner menu. Everything works as in the 'parametric statistics' case, only now the choice of statistics being displayed and the associated thresholds that you can use are slightly different. In particular, the information displayed for each cluster when selecting parametric statistics is:
cluster position: MNI coordinates of largest peak within this cluster
cluster size: number of voxels in this cluster
cluster p-FWE: family-wise error corrected p-value (probability under the null hypothesis of observing one or more clusters of at least this size across the entire brain)
cluster p-FDR: false discovery rate corrected p-value (expected proportion under the null hypothesis of false discoveries among clusters of at least this size)
cluster p-unc: uncorrected p-value (probability under the null hypothesis of a randomly-selected cluster having at least this size)
peak p-FWE: family-wise error corrected p-value (probability under the null hypothesis of observing one or more peaks of at least this height across the entire brain)
peak p-unc: uncorrected p-value (probability under the null hypothesis of a randomly-selected peak having at least this height)
all of the above p-values are obtained using random-field-theory (RFT) assumptions.
When selecting instead non-parametric statistics (and after the corresponding permutation/randomization tests are run) you will then get the following information for each cluster:
cluster position: MNI coordinates of largest peak within this cluster
cluster size: number of voxels in this cluster
cluster p-FWE: family-wise error corrected p-value (probability under the null hypothesis of observing one or more clusters of at least this size across the entire brain)
cluster p-FDR: false discovery rate corrected p-value (expected proportion under the null hypothesis of false discoveries among clusters of at least this size)
cluster p-unc: uncorrected p-value (probability under the null hypothesis of a randomly-selected cluster having at least this size)
cluster mass: sum of statistics (F-values or T^2 values) across all voxels within this cluster
cluster p-FWE: family-wise error corrected p-value (probability under the null hypothesis of observing one or more clusters of at least this mass across the entire brain)
cluster p-FDR: false discovery rate corrected p-value (expected proportion under the null hypothesis of false discoveries among clusters of at least this mass)
cluster p-unc: uncorrected p-value (probability under the null hypothesis of a randomly-selected cluster having at least this mass)
and all of the above p-values are obtained using non-parametric assumptions (permutation/randomization analyses). Cluster mass statistics combine information about each cluster size as well as each cluster height/strength, so they are generally considered more sensitive than either cluster-size or peak-level statistics. Typically for non-parametric statistics I would recommend using a cluster-mass p-FDR<.05 threshold by default, since that should typically be one of the most sensitive tests, but your preferences might vary.
Hope this helps
Alfonso
Originally posted by Jeff Browndyke:
From individuals reading of the Eklund et
al paper, would it be appropriate to drop SPM voxel-wise down to
p<0.01 (consistent with the default p-value setting for FSL) and
then just have p-FDR non-parametric cluster-wise <0.05?
Unless the clusters are quite large, I'm not really seeing how
anyone is going to find anything with a modest sample size and
default SPM p<0.001 and cluster non-parametric corrections.
Jeff
Jeff
Threaded View
| Title | Author | Date |
|---|---|---|
| Mary Newsome | Jul 5, 2016 | |
| Jeff Browndyke | Jul 11, 2016 | |
| Alfonso Nieto-Castanon | Jul 13, 2016 | |
| David White | Jul 8, 2016 | |
| Roger Mateu | Jul 12, 2016 | |
| Stephen L. | Nov 23, 2017 | |
| Alfonso Nieto-Castanon | Jul 5, 2016 | |
