help > RE: Statistical analysis: implications of the paper of Eklund et al?
Jul 11, 2019  12:07 AM | Stephen L. - Coma Science Group, GIGA-Consciousness, Hospital & University of Liege
RE: Statistical analysis: implications of the paper of Eklund et al?
Dear Mickael,

I also had similar questions and I tried to follow the developments on this issue, which I describe here:

https://www.nitrc.org/forum/message.php?...

As Alfonso said, the paper mostly shows that using a voxel-wise threshold of p-uncorrected < 0.01 leads to a very inflated false positive rate (up to 67% if I recall correctly), but using p-uncorrected < 0.001 and cluster-size p-FWE < 0.05 has at worst a false positive rate of about 0.01, so it's still inflated but nowhere near as badly.

Of interesting note, all Eklund's papers on this issue only investigate cluster-size p-FWE, not p-FDR. But since the problems arise from a violation in Random Field Theory assumptions, the basis of the statistical hypothesis testing that is used to correct the values and calculate the significance, and in particular in the spatial autocorrelation function, it's safe to assume that p-FDR is also affected in a similar way.

However, as described in the above link, the latest paper "Cluster Failure Revisited" shows that using a non-neuronal component denoising step (ICA is used in their paper, but they mention that PCA such as aCompCor in CONN should work equally well) fixes the autocorrelation issue as I suspected and essentially mitigates the false positive rate inflation.

So in the end, as long as you are using CONN, and with a voxel-wise threshold p-uncorrected < 0.001, you should not be affected by the issue outlined by these papers. If you want to further ensure a very precisely controlled false positive rate, you can use non-parametric calculations as Alfonso advised, which will not use RFT assumptions and thus totally bypass this issue.

Hope this helps,
Best regards,
Stephen

Threaded View

TitleAuthorDate
Mickael Tordjman Jul 10, 2019
RE: Statistical analysis: implications of the paper of Eklund et al?
Stephen L. Jul 11, 2019
Jeff Browndyke Jul 11, 2019
Stephen L. Jul 11, 2019
Mickael Tordjman Jul 11, 2019