help > RE: false positives cluster level of inference
Jul 13, 2016  01:07 AM | Alfonso Nieto-Castanon - Boston University
RE: false positives cluster level of inference
Hi Jeff&Mike,

My reading from the Eklund et al. and Flanding&Friston papers is that if you are using a voxel-wise height threshold p<.001 (this is the default threshold both in SPM and in CONN) then using parametric statistics is perfectly fine, while nonparametric statistics are recommended when you want to use higher (i.e. more liberal) voxel-wise height thresholds (e.g. p<.01, in order to focus on perhaps weaker but large/distributed responses). So yes, either one of the following two approaches should be perfectly fine (the former being more sensitive for strong localized effects, while the latter being more sensitive for weak distributed effects):

  height threshold p-unc<.001, cluster-size threshold p-FDR<.05, parametric statistics
  height threshold p-unc<.01, cluster-mass threshold p-FDR<.05, non-parametric statistics
 
Regarding how to use non-parametric statistics in CONN, simply select 'non-parametric statistics' in the explorer window top-right corner menu. Everything works as in the 'parametric statistics' case, only now the choice of statistics being displayed and the associated thresholds that you can use are slightly different. In particular, the information displayed for each cluster when selecting parametric statistics is:

  cluster position: MNI coordinates of largest peak within this cluster
  cluster size: number of voxels in this cluster
  cluster p-FWE: family-wise error corrected p-value (probability under the null hypothesis of observing one or more clusters of at least this size across the entire brain)
  cluster p-FDR: false discovery rate corrected p-value (expected proportion under the null hypothesis of false discoveries among clusters of at least this size
  cluster p-unc: uncorrected p-value (probability under the null hypothesis of a randomly-selected cluster having at least this size)
  peak p-FWE: family-wise error corrected p-value (probability under the null hypothesis of observing one or more peaks of at least this height across the entire brain)
  peak p-unc: uncorrected p-value (probability under the null hypothesis of a randomly-selected peak having at least this height)

all of the above p-values are obtained using random-field-theory (RFT) assumptions.

When selecting instead non-parametric statistics (and after the corresponding permutation/randomization tests are run) you will then get the following information for each cluster:
 
  cluster position: MNI coordinates of largest peak within this cluster
  cluster size: number of voxels in this cluster
  cluster p-FWE: family-wise error corrected p-value (probability under the null hypothesis of observing one or more clusters of at least this size across the entire brain)
  cluster p-FDR: false discovery rate corrected p-value (expected proportion under the null hypothesis of false discoveries among clusters of at least this size)
  cluster p-unc: uncorrected p-value (probability under the null hypothesis of a randomly-selected cluster having at least this size)
  cluster mass: sum of statistics (F-values or T^2 values) across all voxels within this cluster
  cluster p-FWE: family-wise error corrected p-value (probability under the null hypothesis of observing one or more clusters of at least this mass across the entire brain)
  cluster p-FDR: false discovery rate corrected p-value (expected proportion under the null hypothesis of false discoveries among clusters of at least this mass)
  cluster p-unc: uncorrected p-value (probability under the null hypothesis of a randomly-selected cluster having at least this mass)

and all of the above p-values are obtained using non-parametric assumptions (permutation/randomization analyses). Cluster mass statistics combine information about each cluster size as well as each cluster height/strength, so they are generally considered more sensitive than either cluster-size or peak-level statistics. Typically for non-parametric statistics I would recommend using a cluster-mass p-FDR<.05 threshold by default, since that should typically be one of the most sensitive tests, but your preferences might vary. 

Hope this helps
Alfonso

Originally posted by Jeff Browndyke:
From individuals reading of the Eklund et al paper, would it be appropriate to drop SPM voxel-wise down to p<0.01 (consistent with the default p-value setting for FSL) and then just have p-FDR non-parametric cluster-wise <0.05?  Unless the clusters are quite large, I'm not really seeing how anyone is going to find anything with a modest sample size and default SPM p<0.001 and cluster non-parametric corrections.
 
Jeff

Threaded View

TitleAuthorDate
Mary Newsome Jul 5, 2016
Jeff Browndyke Jul 11, 2016
RE: false positives cluster level of inference
Alfonso Nieto-Castanon Jul 13, 2016
David White Jul 8, 2016
Roger Mateu Jul 12, 2016
Stephen L. Nov 23, 2017
Alfonso Nieto-Castanon Jul 5, 2016