Hi Dr Zalesky, 


Thank you for your excellent program. 


I am trying to compare three patient groups, where group 1 is a control, group 2 partial disease expression, and group 3 complete penetrance.
I would like to compare the three groups, assessing for whether group 1 < 2 < 3 and 1 > 2 > 3  and 1 < 3,  1 > 3

I was hoping to assess this using t-tests with FDR to perform pairwise comparisons between groups. 

I initially performed this using the following design matrix and contrast (for all three groups):
 

Patient 1 Group 1: 1 0 0
Patient 2 Group 1: 1 0 0
Patient 3 Group 1: 1 0 0
Patient 4 Group 2: 0 1 0
Patient 5 Group 2: 0 1 0
Patient 6 Group 2: 0 1 0
Patient 7 Group 3: 0 0 1
Patient 8 Group 3: 0 0 1
Patient 9 Group 3: 0 0 1

To compare Group 1 < 2: [-1, 1, 0]
To compare Group 1 > 3: [1, 0, -1]
etc

However when I instead separate my design matrix and groups to run these separately, i.e. 

Patient 1 Group 1: 1 0
Patient 2 Group 1: 1 0
Patient 3 Group 1: 1 0
Patient 4 Group 2: 0 1
Patient 5 Group 2: 0 1
Patient 6 Group 2: 0 1

To compare Group 1 < 2: [-1, 1]

Patient 1 Group 1:  1 0
Patient 2 Group 1: 1 0
Patient 3 Group 1: 1 0
Patient 7 Group 3: 0 1
Patient 8 Group 3: 0 1
Patient 9 Group 3: 0 1

To compare Group 1 > 3: [1,-1] 


I get different results. 


I presume the second way is the correct way to do this, and I am guessing that the [0] in the contrast is not ignored as I thought? What is it calculating?


Is there a better way to do this, ideally allowing covariates to be used. I don't think the ANOVA/ANCOVA will work for me as I would need a posthoc test to examine where the effect lies, which appears problematic to perform from the NBS output?


Many thanks, 

Josh

Hi Josh,

the first and second designs will not necessarily yield the same results. This is a general property of the General Linear Model and isn't related to the NBS in any way.

With the first design, while the contrast only involves two groups, the third group will contribute to estimating the variance of the fitted model and will thus influence any inference performed on the other two groups. The degrees of freedom will also be reduced in the second design, due to the reduced number of subjects. Finally, the first design will offer greater diversity in permutations. 

Both designs are acceptable. My guess is that the results should be somewhat comparable across the two designs (but not exactly the same). I would personally use the second design, but others would disagree me. Once again, this is not really related to the NBS per se, but a general property of the GLM. 

The [0] in the contrast is indeed ignored, but the variance from the additional subjects still contributes to the model.

If you would like to test the linear ordering  1<2<3, you could use the following design matrix: 

Patient 1 Group 1: 1 -1
Patient 2 Group 1: 1 -1
Patient 3 Group 1: 1 -1
Patient 4 Group 2: 1 0
Patient 5 Group 2: 1 0
Patient 6 Group 2: 1 0
Patient 7 Group 3: 1 1
Patient 8 Group 3: 1 1
Patient 9 Group 3: 1 1

Contrast: [0 1]  or [0 -1]. The latter will test 1 > 2 > 3

Test: t-test

Covariates can simply be added as additional columns to the design matrix.

ANOVA/ANCOVA is also possible as follows:

Patient 1 Group 1: 1 0 0
Patient 2 Group 1: 1 0 0
Patient 3 Group 1: 1 0 0
Patient 4 Group 2: 0 1 0
Patient 5 Group 2: 0 1 0
Patient 6 Group 2: 0 1 0
Patient 7 Group 3: 0 0 1
Patient 8 Group 3: 0 0 1
Patient 9 Group 3: 0 0 1

Contrast: [1 1 1]

Test: F-test

Post-hoc testing can be done on the connections identified. I think that this would be a good option as well, but quite conservative.

Andrew

Originally posted by joshkr:

Dear Andrew,
 


Thank you for your very clear and prompt reply. 

It is a relief that the first method isn't completely wrong as I had spent quite some time on using the results. As you say, the results are quote similar. 


Many thanks again, 


Kind regards, 

Josh