Dear Prof. Zalesky,

We are running the NBS on ROI-based resting-state MRI functional connectivity with an Repeated-ANOVA (group1: n=9, group2: n=13, each subject repeated scans 3 times). In detail, g1t1, g1t2, g1t3, g2t1, g2t2, g2t3 in total 66 data(9*3+13*3=66). We wanted to compare the differences between groups at each timepoint(g1t1-g2t1, g1t2-g2t2, g1t3-g2t3) and the changes in brain FC over time in both groups(g1t1-g1t2-g1t3, g2t1-g2t2-g2t3). I tried the t-test and one way repeated measures ANOVA and really hope you could give us some advice on how to proceed.

1.We conducted the two-sample t-test 3 times at each timepoint(g1t1-g2t1, g1t2-g2t2, g1t3-g2t3) and tried different thresholds and found that g1t1 less than g2t1 (9 edges), g1t2 less than g2t2 (10 edges) , no difference between g1t3 and g2t3 when the threshold is 2.1 (t test, p=0.05, extent, 5000). Is this correct?

2.I'm not sure if this NBS software can process one way repeated-measure ANOVA. For simplicity, I present only group1 here. I have created the following design matrix that includes all the g1t1, g1t2 and g1t3(9*3) here:
Test: F-test, threshold=2.1, p=0.05, extent, 5000
Design matrix:
1 0 0 0 0 0 0 0 0 -1
1 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 1
0 1 0 0 0 0 0 0 0 -1
0 1 0 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0 1
0 0 1 0 0 0 0 0 0 -1
0 0 1 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0 1
0 0 0 1 0 0 0 0 0 -1
0 0 0 1 0 0 0 0 0 0
0 0 0 1 0 0 0 0 0 1
0 0 0 0 1 0 0 0 0 -1
0 0 0 0 1 0 0 0 0 0
0 0 0 0 1 0 0 0 0 1
0 0 0 0 0 1 0 0 0 -1
0 0 0 0 0 1 0 0 0 0
0 0 0 0 0 1 0 0 0 1
0 0 0 0 0 0 1 0 0 -1
0 0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 1 0 0 1
0 0 0 0 0 0 0 1 0 -1
0 0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 1 0 1
0 0 0 0 0 0 0 0 1 -1
0 0 0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0 1 1
Contrasts: [0,0,0,0,0,0,0,0,0,1]
Threshold=2.1, Permutations=5000, P=0.05, Extent, NBS
Exchange Blocks: 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 6 6 6 7 7 7 8 8 8 9 9 9 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 6 6 6 7 7 7 8 8 8 9 9 9
I found that there is no significant brain FC changes over time neither in group1 nor in group2. So, it is unnecessary to compare the FC between every two timepoints using paired-t test. Could you tell me if these design matrix and contrasts correct?

3.I also tried to directly apply a paired-sample to compare the FC between every two timepoints in each group(g1t1-g1t2, g1t1-g1t3, g1t2-g1t3, g2t1-g2t2, g2t1-g2t3, g2t2-g2t3). I found that there is difference between g1t2 and g1t3(10 edges) in group1 and there is difference between g2t1 and g2t2(16 edges) in group2. I am wondering if these significantly different FC is ture or just lie in false positive results. Because these unexpected results are hard to explain our experiments.

4. Do you have any other recommendations on how to work with our FC matrices data using the NBS?

Thank you very much for your help!

Best regards,
Yang Yingying

Hi Yang,


Like other statistical controlling procedures, the NBS will control false positives for a given alpha-significance level. Of course it is possible that a significant finding is a false positive, but it is impossible to determine this without knowing the ground truth. You may want to consider a replication experiment. If you replicate your findings in a different cohort/sample, it is less likely that the finding is a false positive. 

1. I am not sure what you mean by correct. I don't know the ground truth and so I cannot determine whether 9 edges and 10 edges are correct. If you are considering between-group differences at only one time point, it may be possible to remove the second time point measurement from your edesign matrix, or average connectivity measurements between time points 1 and 2 for each subject. This will give a more reliable estimate of connectivity. 

2. Yes - NBS can run a one-way ANOVA. The design matrix that you specify however is not a one-way ANOVA. For a one-way ANOVA, add a column for each time point. The last column of your design matrix is not correct. 

3. As per 1, without a ground truth, it is impossible to determine whether a result is a false positive. 

Andrew 
Originally posted by Yang Yingying:

Hi Yang, 

A one-way repeated measures with three subjects and three measurements would look like this:


1 0 0 1 0 S1T1
1 0 0 0 1 S1T2
1 0 0 0 0 S1T3
0 1 0 1 0 S2T1
0 1 0 0 1 S2T2
0 1 0 0 0 S2T3
0 0 1 1 0 S3T1 
0 0 1 0 1 
0 0 1 0 0 

The first three columns are the within-subject means.

The last two columns indicate time point 1 and 2. Note that there is no need to model the third time point. 

The contrast would be [ 0 0 0 1 1] and select F-test

You can include exchange blocks. 


Originally posted by Andrew Zalesky:

Dear Prof. Zalesky,

Thank you so much for your reply. The NBS is amazing for its powerful statistical analysis!

I have read some of your answers to various questions on this forums, and found that maybe 2x3 repeated measures ANOVA is more suitable for my experiment. First of all, I would like to estimate the interact effect of group*time and the main effect of group and time.
2x3 RE ANOVA supposed data: group1=4, group2=5, time points=3
Design matrix:
g1t1sub1 1 -1 1 0 0 0 0 0 0 0 0
g1t1sub2 1 -1 0 1 0 0 0 0 0 0 0
g1t1sub3 1 -1 0 0 1 0 0 0 0 0 0
g1t1sub4 1 -1 0 0 0 1 0 0 0 0 0
g1t2sub1 -1 1 1 0 0 0 0 0 0 0 0
g1t2sub2 -1 1 0 1 0 0 0 0 0 0 0
g1t2sub3 -1 1 0 0 1 0 0 0 0 0 0
g1t2sub4 -1 1 0 0 0 1 0 0 0 0 0
g1t3sub1 0 0 1 0 0 0 0 0 0 0 0
g1t3sub2 0 0 0 1 0 0 0 0 0 0 0
g1t3sub3 0 0 0 0 1 0 0 0 0 0 0
g1t3sub4 0 0 0 0 0 1 0 0 0 0 0
g2t1sub5 1 1 0 0 0 0 1 0 0 0 0
g2t1sub6 1 1 0 0 0 0 0 1 0 0 0
g2t1sub7 1 1 0 0 0 0 0 0 1 0 0
g2t1sub8 1 1 0 0 0 0 0 0 0 1 0
g2t1sub9 1 1 0 0 0 0 0 0 0 0 1
g2t2sub5 -1 -1 0 0 0 0 1 0 0 0 0
g2t2sub6 -1 -1 0 0 0 0 0 1 0 0 0
g2t2sub7 -1 -1 0 0 0 0 0 0 1 0 0
g2t2sub8 -1 -1 0 0 0 0 0 0 0 1 0
g2t2sub9 -1 -1 0 0 0 0 0 0 0 0 1
g2t3sub5 0 0 0 0 0 0 1 0 0 0 0
g2t3sub6 0 0 0 0 0 0 0 1 0 0 0
g2t3sub7 0 0 0 0 0 0 0 0 1 0 0
g2t3sub8 0 0 0 0 0 0 0 0 0 1 0
g2t3sub9 0 0 0 0 0 0 0 0 0 0 1
describe t t*g sub1 sub2 sub3 sub4 sub5 sub6 sub7 sub8 sub9
Where the first column models the main effect of time, and the 2th column models the interact effect of group*time, and the last 9 columns model the individual means of each subject.
The contrast for the main effect of time is either: [1 0 0 0 0 0 0 0 0 0 0] or [-1 0 0 0 0 0 0 0 0 0 0]
The contrast for the interaction is: [0 1 0 0 0 0 0 0 0 0 0] or [0 -1 0 0 0 0 0 0 0 0 0]
Exchange block: [1 2 3 4 1 2 3 4 1 2 3 4 5 6 7 8 9 5 6 7 8 9 5 6 7 8 9]
T-test, threshold=2.0, permutations=5000, NBS, p=0.05, extent/ intensity

(1) Although the NBS software runs successfully, I am still not sure whether my design matrix and comparison are correct. I was hoping for you to please verify if my design matrix is correct?

(2) To test the main effect of group, you advise to average across the 3 time points per subject and perform a two sample t-test on the forums before. Can I just use the above connectivity matrices without Exchange Blocks, and change the design matrix as bellow? Contrast[1,-1] or [-1, 1]. Is this simple method correct (without averaging)?
1 0
1 0
1 0
1 0
1 0
1 0
1 0
1 0
1 0
1 0
1 0
1 0
0 1
0 1
0 1
0 1
0 1
0 1
0 1
0 1
0 1
0 1
0 1
0 1
0 1
0 1
0 1


Best regards,
Yang Yingying

Hi Yang, 

yes your design matrix looks to be correct and the contrasts are correct. 

For the main effect of group, a repeated measures design is not needed because the groups will not change at the different time points. So you could take the average of the connectivity matrices across the three time points for each subject. Alternatively, you could run a separate t-test for each of the three time points and compare the results. 

Regarding your suggestion in (2), this is not quite correct because you will be treating three measurements in the same subject as three different subjects. This will inflate the degrees of freedom and give you optimistic p-values. 

Andrew


Originally posted by Yang Yingying:

Dear Prof. Zalesky,

Thank you very much for your help.
All the best!

Yours,
Yang Yingying