Hi all,
I have a reviewer concerned that I did not correct for multiple comparisons *across* number of components. That is, I have 36 components, and they suggest the significant pFWE cluster resulting from a behavior/ICN correlation should also be corrected for multiple comparisons. I think this is overcorrected, anyone care to weigh in?

Hi Teodora,

It all depends on whether:

a) you evaluated the behavior/ICN correlation across one component only (e.g. you were interested in the default mode network, so among the original 36 components you first selected the component that seemed to best identify this network, and then evaluated whether this component was correlated with behavior); in this case you do not need any additional multiple-comparison correction (it does not matter that this component was selected among 36 components because the selection criterion was unrelated to the behavioral variable) 

or b) you evaluated the behavior/ICN correlation across the 36 components and found one (or several) that showed some significant suprathreshold clusters; in this case you DO need to apply an additional correction (across the 36 test being evaluated)

In general, all group-level analysis will perform an appropriate correction for multiple comparisons at the analysis-level (e.g. the chance of false positives under the null hypothesis is below 5% for this particular analysis), so if your study involved performing multiple group-level analyses and then reporting the ones where you find significant results, you will want to correct for multiple comparisons at the study-level (e.g. the chance of false positives under the null hypothesis is below 5% for this particular study), which is often done by using a Bonferroni-corrected analysis-level threshold (e.g. for each individual analysis, you only consider a cluster significant if it falls below a p-FDR  < 0.05/N cluster-level threshold, where N is the number of different group-level analyses performed). 

Hope this helps
Alfonso
Originally posted by Teodora Stoica:

As always, thank you for your detailed and prompt response. I did A, so I will have to make that clear to the reviewer.

Hi Alfonso,

Is there a need to correct for multiple comparisons when you are investigating multiple behavioral variables? Ie - I am selecting one behavioral subscale at a time (that is part of a larger aggregate scale) and looking at the connectivity of that subscale within one particular ICN of interest, the repeat for each subscale and then for the aggregate. 
Thanks,
Teodora