Hello CONN Experts,

I've looked over the manual and past message board questions but still have some doubts on if I'm properly coding my 2nd level contrasts.

I am looking at relationship of RS connectivity to symptom severity in a multi-site study of multiple patient groups in a number of ROIs.
I would like to primarily look at main effects of symptom for the all the patients and also group x symptom interactions. 

Example of my current Subject Effect I have coded:
QCSubj: Subset of subjects that are okay quality to use in analysis e.g. [ 1 1 1 NaN 1 1 1 NaN 1 1 1 NaN]
OnlyPts: Only patients, exclude healthy controls. eg. [ 1 1 1 1 1 1 1 1 1 NaN NaN NaN]
Group A: e.g.  [ 1 0 0 1 0 0 1 0 0]
Group B: e.g. [  0 1 0 0 1 0 0 1 0]
Group C: e.g. [  0 0 1 0 0 1 0 0 1]
DxGroup: e.g. [ 1 2 3 1 2 3 1 2 3]
SymA: e.g. [ 1 0 0 4 0 0 7 0 0] - has NaN for healthy controls
SymB: e.g. [ 0 2 0 0 5 0 0 8 0]- has NaN for healthy controls
SymC: e.g. [ 0 0 3 0 0 6 0 0 9] - has NaN for healthy controls
Sym:  e.g. [1 2 3 4 5 6 7 8 9]
Age: e.g. [ 21 22 23 24 25 26 27 28 29 30]
Sex: e.g [1 0 1 0 1 0 1 0 1 0]
Site1: e.g. [ 1 1 1 0 0 0 0 0 0]
Site2: e.g. [ 0 0 0 1 1 1  0 0 0]
Site3: e.g. [ 0 0 0 0 0 0 1 1  1]
OnlyGroupAB e.g. [ 1 1 NaN 1 1 NaN 1 1 NaN]
OnlyGroupAC e.g. [ 1 NaN 1  1 NaN 1 1 NaN 1]
OnlyGroupBC e.g. [ NaN 1 1  NaN 1 1 NaN 1 1] 

Also each of the 3 patient groups has ~90-120 subjects.

1. Is this the appropriate F test to look at the symptom effect of any group?

QC OnlyPts  Age Sex S1  S2  S3 GrA GrA GrC SymA SymB SymC

[ 0 0 0 0 0 0 0 0 0 0 1 0 0 ];
[ 0 0 0 0 0 0 0 0 0 0 0 1 0 ];
[ 0 0 0 0 0 0 0 0 0 0 0 0 1 ];

1b) If so I'm wondering if this doesn't inflate the degrees of freedom? I tried using OnlyGroupA OnlyGroupB OnlyGroupC with NaNs as group vectors but ended up with n=0 subjects for the design.
1c) Another concern I have is currently I have my Symptom Group covariates coded with 0 for the non-group members, I'm worried that in this F test for example that zero values are being included in the regression versus if their were somehow NaNs to prevent them being incorporated in the test. Is this not an issue?
1d) In addition I'd also done separate subgroup analyses for each group with symptom and while results are fairly similar I'm wondering theoretically why not identical.  I test each group separately like this QC Age Sex S1 S2 S3 Sym OnlyGroupA - [0 0 0 0 0 0 1 0] which I would think to be ultimately identical to one of the F test conjunction vectors if they were working as I presumed. I'd expect all my individual subgroup results to show up in the F test but all don't. Is it a df or coding issue?

2. Then to test for GroupXSymtom Interactions would I just do pairwise contrasts like this for each group? Is there a way to test all 3 groups at once like an F-test?
   QC Age Sex S1 S2 S3 GrA GrA SymA SymB OnlyGrAB

[ 0 0 0 0 0 0 0 0 1 -1 0]; Interaction for Group A and B for Symptom

 QC Age Sex S1 S2 S3 GrB GrC SymB SymC OnlyGrBC

[ 0 0 0 0 0 0 0 0 1 -1 0]; Interaction for Group B and C for Symptom

...... And similar for Group A and C Symptom Interaction.

3. What is the difference in interpretation for these tests? Which would be appropriate to test for common connectivity association with symptom?
 
QC OnlyPts Age Sex S1 S2 S3  Sym

[0 0 0 0 0 0 0 0 1]
  
   QC OnlyPts Age Sex S1 S2 S3 Sym

  [0 0 0 0 0 0 0 0 0 1]

4. I also saw in old post Alfonso mentioned that I could do the F test, mask for the significant regions and then do the post-hoc interaction tests within the masks. Sorry if this is a basic concept, why isn't this double dipping or p-hacking?  
https://www.nitrc.org/forum/message.php?msg_id=12044 


Thanks in advance for the help!
Victoria Okunye

Just following up because I'd really appreciate any advice on this.
At a bit of a stand still with analyses until I'm sure I'm using proper contrast coding.

Thanks,
Victoria

Hi Victoria,

Regarding your second-level covariate definitions, everything looks ok in those definition examples, the main strange thing is that QCSubj and OnlyPts have 12 subjects while the rest of variables have only 9, but I imagine that is just some typo. Relatedly, you also do not quite specify what happens with the control group subjects (e.g. the last missing 3 subjects?) for the other covariate definitions, e.g. are they from an entirely different site or from the same sites as your patients? 

Regarding (1) everything looks fine in there as well. It is expected to have different sensitivity in individual group contrasts vs. a combined F- contrast across all groups (that is just in the nature of one multivariate vs. multiple univariate analyses, just like a MANOVA is similar but not identical to multiple individual ANOVAs). The 0-value of your SymA covariate for subjects not in GroupA is fine as well (the covariate x group effects -e.g. 'SymA'- are encoded as the product of your covariate -e.g. 'Sym'- and your group -e.g. 'GroupA' variables). Typically those variables (e.g. 'SymA') should also be 0 (instead of NaN) for control subjects, but since perhaps you are not planning any patient vs. control comparisons which include the Symptom score variable this may be just a moot point.

Regarding (2) you can test the Group x Symptom interactions using a contrast of the form:
QC OnlyPts Age Sex S1 S2 S3 GrA GrA GrC SymA SymB SymC
[ 0 0 0 0 0 0 0 0 0 0 1 -1 0 ];
[ 0 0 0 0 0 0 0 0 0 0 0 1 -1 ]

Regarding (3), if by "common connectivity association with symptom" there you mean associations that do not appear to be different across the three groups, then simply intersecting the results from the main effect of symptom (e.g. in (1)) with those that do not show an symptom-by-group interaction (e.g. those not in (2)) would be a reasonable way to address this. If, instead, you mean associations that appear across all subjects (disregarding group interactions) then a contrast like the following will address that: 
QC OnlyPts Age Sex S1 S2 S3 GrA GrA GrC Sym
[ 0 0 0 0 0 0 0 0 0 0 1]

Regarding (4), yes, it depends a bit on the details of how you implement this type of conjunction whether they should be treated as post-hoc analyses (i.e. non-confirmatory, mainly useful as a way to characterize an effect that has already been determined to be significant by other means) or confirmatory (i.e. used to perform statistically valid inferences). In particular, if you use a minimum-T conjunction (in SPM this is referred to as 'conjunction-null') of the two contrasts (e.g. [ 0 0 0 0 0 0 0 0 0 0 1 0 0 ; 0 0 0 0 0 0 0 0 0 0 0 1 0; 0 0 0 0 0 0 0 0 0 0 0 0 1] for main group effects, and [ 0 0 0 0 0 0 0 0 0 0 1 -1 0 ; 0 0 0 0 0 0 0 0 0 0 0 1 -1] for symptom-by-group interactions) then the analyses can be treated as confirmatory (no double-dipping there). If you use instead a conditional conjunction (in SPM this is referred to as 'small volume correction') then the analyses can be treated as confirmatory only iff the conjunction mask is orthogonal to the main effect (more precisely if the data subspaces spanned by the null hypothesis in both contrasts are orthogonal). In our case that can typically be accomplished, for example, by using as a contrast for main group effects something like [ 0 0 0 0 0 0 0 0 0 0 1/3 1/3 1/3] instead of the alternative F-contrast). 

Hope this helps
Alfonso

Originally posted by Victoria Okuneye:

Thank you so much once again for your detailed feedback Alfonso. It is really appreciated!!