Dear Alfonso,

For the needs of my current project, I dwelved more in-depth in the intricacies of multiple comparison correction and the various solutions currently available, as well as the latest debates on the matter.

I have a few questions following my reading of Eklund et al's paper "Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates" and the follow-up blog post "Keep Calm and Scan On":

1. Since CONN is using SPM under the hood, I guess CONN also relies on RFT for inference, and thus is subject to the inflated rate described in the Eklund et al's paper when one is using parametric cluster-wise correction, right?

2. According to Eklund et al's, this inflated rate should be mainly due to assumption about the constancy of smoothing across the brain, particularly wrong for voxels that are far apart. Knowing that CONN has an additional "Denoising" step that Eklund et al did not test, and that this denoising step clearly corrects for signal variation across voxels distances (as shown by the middle preview plot in the Denoising tab, see attached picture below), does this mean that CONN is significantly less affected by this issue, since the denoising ensures more constant signal across the whole brain?

Thank you very much in advance!
Warm regards!
Stephen

Dear Stephen,

Thank you for your thoughtful comments. Regarding #1, yes, when you select the 'parametric stats' option in the second-level results window, CONN will be using RFT to evaluate cluster-level statistics, so, following Eklund et al., this form of cluster-level inferences is only recommended when used in combination with relatively conservative height-threshold values (e.g. uncorrected voxel-level p<.001) but not with relatively liberal height-threshold values (e.g. uncorrected voxel-level p<.01). For the latter case, simply select the option 'non-parametric stats' in the same second-level results window, and that will use permutation/randomization tests when evaluating cluster-level statistics instead. 

Regarding #2, that is a perfectly valid point. The main departure reported between the RFT assumptions and the data observed in Eklund et al. concerns the presence of relatively longer tails in the observed distribution of cluster sizes under the null hypothesis (consistent with long-range connections affecting the spatial autocorrelation function). While it is true that the Denoising step will considerably reduce the level of artifactual long-range whole-brain connections arising from physiological and subject motion effects, and in doing so it will dramatically change the shape of the distribution of cluster-sizes (mostly acting to reduce the presence of long tails caused by these same artifactual effects), there are still reasons to believe that at least some long-range connections will remain after Denoising (e.g. the standard resting state networks) which simply are not modeled by the RFT framework. I have not seen any papers that address this specifically (the extent to which different Denoising strategies may help normalize the distribution of cluster sizes potentially producing a better match with RFT assumptions) but I would be very interested to hear about this line of research. If I had to guess, my prediction would be that Denoising would in fact help ameliorate the inflation of false positive rates observed in Eklund et al. (but probably some level of inflation would still remain, due to the presence of "true" long-range connectivity).

Hope this helps
Alfonso
Originally posted by Stephen L.:

Dear Alfonso,

Thank you very much for your detailed and very clear explanation, this makes perfect sense!

So in any case, non-parametric methods are the way to go to ensure appropriate cluster-wise correction, but for old papers made with CONN, it is good to know that denoising might have reduced a bit the issue :-)

Addendum: in Eklund's paper's supplementary PDF, the following can be found:

«Supplementary Figure 14 shows that the SACFs are far from a squared exponential. The empirical SACFs are close to a squared
exponential for small distances, but the autocorrelation is higher than expected for large distances. This could be the reason
why the parametric methods work rather well for a high cluster defining threshold (p = 0.001), and not at all for a low threshold
(p = 0.01). A low threshold gives large clusters with a large radius, for which the tail of the SACF is quite important. For a high
threshold, resulting in rather small clusters with a small radius, the tail is not as important.»

This seems to go along the same lines as what we discussed before, so I would say it's likely that CONN's denoising reduces the extent of this issue :-)

Hello there, just a quick update given new development, it seems that aCompCor might not only reduce the issue but fix it altogether, this is what is suggested in the followup paper of Eklund et al, Cluster Failure Revisited, where they tested ICA but not PCA regression of noise but they expect a similar enhancement to the false positive rate inflation :-)

* Eklund, A., Knutsson, H., & Nichols, T. E. (2019). Cluster failure revisited: Impact of first level design and physiological noise on cluster false positive rates. Human brain mapping, 40(7), 2017-2032.

Also another paper shown that modelling accurately the autocorrelation function greatly increases fMRI reliability, which suggests the fp inflation is mainly due to the autocorrelation function being either incorrectly modelled in some software packages and/or not enough regressed.

* Olszowy, W., Aston, J., Rua, C., & Williams, G. B. (2019). Accurate autocorrelation modeling substantially improves fMRI reliability. Nature communications, 10(1), 1220.

Best regards,
Stephen