Hi Alfonso,

Thank you for the clarification and recommendations on the 2nd-level MVPA approach. I will pass this information on to my collaborators.

I should apologise first of all as my use of the word "correct" in terms of the F-statistic is not really accurate as you are indeed right, the whitening will guarantee (when all other assumptions are valid) that the test statistic will follow an F-distribution under the null. This is confirmed by your simulations.

The question is really one about the most appropriate F-statistic to use to test specific hypotheses, which comes down to the issue of partitioned vs pooled errors. The majority of statistical packages use the traditional "split-plot" approach for testing hypotheses in repeated-measures models because the tests are more sensitive. As such, we would do well in neuroimaging to take the same approach. I would hazard that this is the expectation of most researchers, namely that their models can match the models implemented in other packages.  

Unfortunately, as far as I can tell, the two-stage approach advocated in the SPM chapter you indicated, and one the SPM Wiki (https://en.wikibooks.org/wiki/SPM/Group_...) doesn't agree with other stats packages when the within-subject factor has > 2 levels. Correct partitioned errors seem to be only achievable through the use of the flexible factorial approach (using a full over-parameterised design) or through careful combination of first-level contrasts and first-level averages.

I have attached a script and some data demonstrating this. The models looking at within-subject effects with 2-levels can actually be corrected by using two-sample t-test models for all comparisons (rather than the one-sample models advocated on the Wiki), however there is no such simple fix for those with > 3 levels, as far as I can tell. The models given for the portioned error examples are actually simplifications of the more complex over-parameterised models that I discuss in a recent preprint (https://psyarxiv.com/a5469/) but the principles are still the same.

In terms of the approach for CONN, my feeling would be that partitioned-errors are still necessary when looking across components at the second-level. Taking through differential effects of the second within-subject factor is akin to the approach advocated on the wiki, which does not result in the same statistics as produced by standard packages. The fact that you are looking across components rather than between components I don't believe should make a difference to how the data are modelled. My feeling is that the same error term that you would use for looking between components should be used when looking across components, as you are still looking at a within-subject effect (although I'd be interested to hear your take on this).

This is still a thorny issue in the community as there are questions about it almost weekly on the SPM list. I'd be interested to hear your take on this, although I realise we've gone beyond the original remit of my questions!

Best wishes,
- Martyn