help > Post-hoc testing of One-way RE ANOVA
Showing 1-7 of 7 posts
Feb 25, 2022 04:02 PM | Yang Yingying
Post-hoc testing of One-way RE ANOVA
Dear Dr. Zalesky,
Many thanks for your help, and I do have a few follow-up questions.
A one-way repeated measures with three subjects and three measurements
1 0 1 0 0
1 0 0 1 0
1 0 0 0 1
0 1 1 0 0
0 1 0 1 0
0 1 0 0 1
0 0 1 0 0
0 0 0 1 0
0 0 0 0 1
The first 2 columns indicate time point, and the last 3 columns are the within-subject means.
Contrast [ 1 1 0 0 0]. Exchange Blocks: 1, 2, 3,1, 2, 3,1, 2, 3. F-test,Threshold=2.1
We found the significant 49 edges, but I am confused by the post-hoc comparisons. I want to know, how the these 49 edges what changes between each time point? (decreasing at tp2 or tp3 or gradually? increasing at tp2 then decreasing at tp3? or others)
1.Dose NBS can test these post-hoc comparisons? I did the simply paired-test(t1-t2,t1-t3,t2-t3) and found that some significant edges(the results of one way RE ANOVA ) have no difference between any two time points at all.
2.I read some discussions through the forums, and found that you advice to test a specific hypothesis such that connectivity strictly increases (or decreases) with time. I can excluded a priori tp3>tp1, because the tp3 is a aging state with FC decreasing. But I am not sure if there's a compensatory increase in FC at the tp2. Although using liner decreasing is somehow adventure, the overall trend is indeed downward.
1 1 1 0 0
1 1 0 1 0
1 1 0 0 1
1 -1 1 0 0
1 -1 0 1 0
1 -1 0 0 1
1 4 1 0 0
1 4 0 1 0
1 4 0 0 1
The first 2 columns indicate time point, and the last 3 columns are the within-subject means.
Exchange Blocks: 1, 2, 3,1, 2, 3,1, 2, 3. . T-test with contrast [ 0 1 0 0 0] tests t1 ? t2 >> t3. Maybe more complicated advanced models design and assumes a decrease over time, as well as other assumptions might be more suitable for evaluating the FC changes with aging. Can you give me some advice?
3.Because I really care about the changing trend of brain FC over time, what do you think of extracting the connection values of the 49 edges and comparing them one by one between 2 time points? Is it reasonable method?
4.How about averaging these 49 FC correlation coefficient as a 'network overall correlation coefficient', and then comparing between time point?
5.If NBS simply paired-test could not do the post-hoc comparisons right now, is is reasonable just to apply paired T-tests 3 times using NBS (t1-t2,t1-t3,t2-t3)? How to explain the inconsistencies results between the paired test and the one-way RE ANOVA?
Sorry for so much questions but I'm quite lost! Thanks a lot in advance for your help and insights!
So much appreciated.
Yours,
Yang Yingying
Many thanks for your help, and I do have a few follow-up questions.
A one-way repeated measures with three subjects and three measurements
1 0 1 0 0
1 0 0 1 0
1 0 0 0 1
0 1 1 0 0
0 1 0 1 0
0 1 0 0 1
0 0 1 0 0
0 0 0 1 0
0 0 0 0 1
The first 2 columns indicate time point, and the last 3 columns are the within-subject means.
Contrast [ 1 1 0 0 0]. Exchange Blocks: 1, 2, 3,1, 2, 3,1, 2, 3. F-test,Threshold=2.1
We found the significant 49 edges, but I am confused by the post-hoc comparisons. I want to know, how the these 49 edges what changes between each time point? (decreasing at tp2 or tp3 or gradually? increasing at tp2 then decreasing at tp3? or others)
1.Dose NBS can test these post-hoc comparisons? I did the simply paired-test(t1-t2,t1-t3,t2-t3) and found that some significant edges(the results of one way RE ANOVA ) have no difference between any two time points at all.
2.I read some discussions through the forums, and found that you advice to test a specific hypothesis such that connectivity strictly increases (or decreases) with time. I can excluded a priori tp3>tp1, because the tp3 is a aging state with FC decreasing. But I am not sure if there's a compensatory increase in FC at the tp2. Although using liner decreasing is somehow adventure, the overall trend is indeed downward.
1 1 1 0 0
1 1 0 1 0
1 1 0 0 1
1 -1 1 0 0
1 -1 0 1 0
1 -1 0 0 1
1 4 1 0 0
1 4 0 1 0
1 4 0 0 1
The first 2 columns indicate time point, and the last 3 columns are the within-subject means.
Exchange Blocks: 1, 2, 3,1, 2, 3,1, 2, 3. . T-test with contrast [ 0 1 0 0 0] tests t1 ? t2 >> t3. Maybe more complicated advanced models design and assumes a decrease over time, as well as other assumptions might be more suitable for evaluating the FC changes with aging. Can you give me some advice?
3.Because I really care about the changing trend of brain FC over time, what do you think of extracting the connection values of the 49 edges and comparing them one by one between 2 time points? Is it reasonable method?
4.How about averaging these 49 FC correlation coefficient as a 'network overall correlation coefficient', and then comparing between time point?
5.If NBS simply paired-test could not do the post-hoc comparisons right now, is is reasonable just to apply paired T-tests 3 times using NBS (t1-t2,t1-t3,t2-t3)? How to explain the inconsistencies results between the paired test and the one-way RE ANOVA?
Sorry for so much questions but I'm quite lost! Thanks a lot in advance for your help and insights!
So much appreciated.
Yours,
Yang Yingying
Feb 25, 2022 10:02 PM | Andrew Zalesky
RE: Post-hoc testing of One-way RE ANOVA
Hi Yang,
In brief, I suggest option 4 - extract average FC values across the 49 edges and compare them between time points. There is example code in the NBS manual showing how to extract FC values for specific connections.
Running three paired t-tests would also be reasonable, but this would create a multiple comparisons issue and so you may need to consider a threshold that is more stringent than p=0.05.
The design matrix in option 2 specifically assumes an effect that is 4 times greater in one of the groups compared to the other. Unless you have a very specific hypothesis, it may be hard to justify the choice of using 4.
best,
Andrew
Originally posted by Yang Yingying:
In brief, I suggest option 4 - extract average FC values across the 49 edges and compare them between time points. There is example code in the NBS manual showing how to extract FC values for specific connections.
Running three paired t-tests would also be reasonable, but this would create a multiple comparisons issue and so you may need to consider a threshold that is more stringent than p=0.05.
The design matrix in option 2 specifically assumes an effect that is 4 times greater in one of the groups compared to the other. Unless you have a very specific hypothesis, it may be hard to justify the choice of using 4.
best,
Andrew
Originally posted by Yang Yingying:
Dear Dr. Zalesky,
Many thanks for your help, and I do have a few follow-up questions.
A one-way repeated measures with three subjects and three measurements
1 0 1 0 0
1 0 0 1 0
1 0 0 0 1
0 1 1 0 0
0 1 0 1 0
0 1 0 0 1
0 0 1 0 0
0 0 0 1 0
0 0 0 0 1
The first 2 columns indicate time point, and the last 3 columns are the within-subject means.
Contrast [ 1 1 0 0 0]. Exchange Blocks: 1, 2, 3,1, 2, 3,1, 2, 3. F-test,Threshold=2.1
We found the significant 49 edges, but I am confused by the post-hoc comparisons. I want to know, how the these 49 edges what changes between each time point? (decreasing at tp2 or tp3 or gradually? increasing at tp2 then decreasing at tp3? or others)
1.Dose NBS can test these post-hoc comparisons? I did the simply paired-test(t1-t2,t1-t3,t2-t3) and found that some significant edges(the results of one way RE ANOVA ) have no difference between any two time points at all.
2.I read some discussions through the forums, and found that you advice to test a specific hypothesis such that connectivity strictly increases (or decreases) with time. I can excluded a priori tp3>tp1, because the tp3 is a aging state with FC decreasing. But I am not sure if there's a compensatory increase in FC at the tp2. Although using liner decreasing is somehow adventure, the overall trend is indeed downward.
1 1 1 0 0
1 1 0 1 0
1 1 0 0 1
1 -1 1 0 0
1 -1 0 1 0
1 -1 0 0 1
1 4 1 0 0
1 4 0 1 0
1 4 0 0 1
The first 2 columns indicate time point, and the last 3 columns are the within-subject means.
Exchange Blocks: 1, 2, 3,1, 2, 3,1, 2, 3. . T-test with contrast [ 0 1 0 0 0] tests t1 ? t2 >> t3. Maybe more complicated advanced models design and assumes a decrease over time, as well as other assumptions might be more suitable for evaluating the FC changes with aging. Can you give me some advice?
3.Because I really care about the changing trend of brain FC over time, what do you think of extracting the connection values of the 49 edges and comparing them one by one between 2 time points? Is it reasonable method?
4.How about averaging these 49 FC correlation coefficient as a 'network overall correlation coefficient', and then comparing between time point?
5.If NBS simply paired-test could not do the post-hoc comparisons right now, is is reasonable just to apply paired T-tests 3 times using NBS (t1-t2,t1-t3,t2-t3)? How to explain the inconsistencies results between the paired test and the one-way RE ANOVA?
Sorry for so much questions but I'm quite lost! Thanks a lot in advance for your help and insights!
So much appreciated.
Yours,
Yang Yingying
Many thanks for your help, and I do have a few follow-up questions.
A one-way repeated measures with three subjects and three measurements
1 0 1 0 0
1 0 0 1 0
1 0 0 0 1
0 1 1 0 0
0 1 0 1 0
0 1 0 0 1
0 0 1 0 0
0 0 0 1 0
0 0 0 0 1
The first 2 columns indicate time point, and the last 3 columns are the within-subject means.
Contrast [ 1 1 0 0 0]. Exchange Blocks: 1, 2, 3,1, 2, 3,1, 2, 3. F-test,Threshold=2.1
We found the significant 49 edges, but I am confused by the post-hoc comparisons. I want to know, how the these 49 edges what changes between each time point? (decreasing at tp2 or tp3 or gradually? increasing at tp2 then decreasing at tp3? or others)
1.Dose NBS can test these post-hoc comparisons? I did the simply paired-test(t1-t2,t1-t3,t2-t3) and found that some significant edges(the results of one way RE ANOVA ) have no difference between any two time points at all.
2.I read some discussions through the forums, and found that you advice to test a specific hypothesis such that connectivity strictly increases (or decreases) with time. I can excluded a priori tp3>tp1, because the tp3 is a aging state with FC decreasing. But I am not sure if there's a compensatory increase in FC at the tp2. Although using liner decreasing is somehow adventure, the overall trend is indeed downward.
1 1 1 0 0
1 1 0 1 0
1 1 0 0 1
1 -1 1 0 0
1 -1 0 1 0
1 -1 0 0 1
1 4 1 0 0
1 4 0 1 0
1 4 0 0 1
The first 2 columns indicate time point, and the last 3 columns are the within-subject means.
Exchange Blocks: 1, 2, 3,1, 2, 3,1, 2, 3. . T-test with contrast [ 0 1 0 0 0] tests t1 ? t2 >> t3. Maybe more complicated advanced models design and assumes a decrease over time, as well as other assumptions might be more suitable for evaluating the FC changes with aging. Can you give me some advice?
3.Because I really care about the changing trend of brain FC over time, what do you think of extracting the connection values of the 49 edges and comparing them one by one between 2 time points? Is it reasonable method?
4.How about averaging these 49 FC correlation coefficient as a 'network overall correlation coefficient', and then comparing between time point?
5.If NBS simply paired-test could not do the post-hoc comparisons right now, is is reasonable just to apply paired T-tests 3 times using NBS (t1-t2,t1-t3,t2-t3)? How to explain the inconsistencies results between the paired test and the one-way RE ANOVA?
Sorry for so much questions but I'm quite lost! Thanks a lot in advance for your help and insights!
So much appreciated.
Yours,
Yang Yingying
Feb 26, 2022 03:02 PM | Yang Yingying
RE: Post-hoc testing of One-way RE ANOVA
Dear Dr. Zalesky,
Thank you so much for your prompt reply and the helpful suggestions!I still have some questions for help.
My study is evaluate the rats brain FC changes with disease and aging. T1 is early stage of disease in early adulthood; T2 is middle stage of disease in late adulthood; T3 is late stage of disease in aging rats.
I perform the post-hoc testing of One-way RE ANOVA by extracting average FC values across the 49 edges and comparing them between time points. The overall trend of FC is decreasing, but only t1>t3 was statistically significant. I also test a specific hypothesis whether FC time1 > time2 or time2 > time1 using paired-test (p=0.05/3). I found that there are significant edges in t1 > t2, but no significant edges in t1 < t2. Hence, we can confirm that FC strictly liner decreases with time, and I could perform the much power statics.
Design matrix:
1 1 1 0 0
1 1 0 1 0
1 1 0 0 1
1 2 1 0 0
1 2 0 1 0
1 2 0 0 1
1 4 1 0 0
1 4 0 1 0
1 4 0 0 1
Exchange Blocks: 1, 2, 3,1, 2, 3,1, 2, 3. T-test
Q1: Because the time interval between t2 and t3 is just 2 times longer than the gap between t1 and t2, I select the value 4 for t3 in the design matrix. There '4' means longer time interval or greater decline degree of constructions? I really confused by you said 'the increase in connectivity between t2 and t3 is exactly twice that of the increase between t1 and t2'.
Q2: Contrast is [0 -1 0 0 0 0] to test for t1> t2> t3 and [0 1 0 0 0 0] to test for t1 < t2 < t3, I hope I did not make it wrong. Significantly 7 edges were found with contrast -1, while 9 edges with contrast 1. This strange results means there are connections that some go up and some others go down over time, is it reasonable?
Q3: When I perform the NBS, it run successful except pop up orange warnings: 'Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND=1.138690e-18. ...Rank deficient, rank = 10, tol =1.295112e-14'. Can I ignore these warnings?
Q4: Since the linearly decreasing design matrix has greater statistical power, why only 7 edges were found, whereas 49 were were found before? I am really lost.
So much appreciated. I really learned a lot under your guidance.
Yours,
Yang Yingying
Thank you so much for your prompt reply and the helpful suggestions!I still have some questions for help.
My study is evaluate the rats brain FC changes with disease and aging. T1 is early stage of disease in early adulthood; T2 is middle stage of disease in late adulthood; T3 is late stage of disease in aging rats.
I perform the post-hoc testing of One-way RE ANOVA by extracting average FC values across the 49 edges and comparing them between time points. The overall trend of FC is decreasing, but only t1>t3 was statistically significant. I also test a specific hypothesis whether FC time1 > time2 or time2 > time1 using paired-test (p=0.05/3). I found that there are significant edges in t1 > t2, but no significant edges in t1 < t2. Hence, we can confirm that FC strictly liner decreases with time, and I could perform the much power statics.
Design matrix:
1 1 1 0 0
1 1 0 1 0
1 1 0 0 1
1 2 1 0 0
1 2 0 1 0
1 2 0 0 1
1 4 1 0 0
1 4 0 1 0
1 4 0 0 1
Exchange Blocks: 1, 2, 3,1, 2, 3,1, 2, 3. T-test
Q1: Because the time interval between t2 and t3 is just 2 times longer than the gap between t1 and t2, I select the value 4 for t3 in the design matrix. There '4' means longer time interval or greater decline degree of constructions? I really confused by you said 'the increase in connectivity between t2 and t3 is exactly twice that of the increase between t1 and t2'.
Q2: Contrast is [0 -1 0 0 0 0] to test for t1> t2> t3 and [0 1 0 0 0 0] to test for t1 < t2 < t3, I hope I did not make it wrong. Significantly 7 edges were found with contrast -1, while 9 edges with contrast 1. This strange results means there are connections that some go up and some others go down over time, is it reasonable?
Q3: When I perform the NBS, it run successful except pop up orange warnings: 'Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND=1.138690e-18. ...Rank deficient, rank = 10, tol =1.295112e-14'. Can I ignore these warnings?
Q4: Since the linearly decreasing design matrix has greater statistical power, why only 7 edges were found, whereas 49 were were found before? I am really lost.
So much appreciated. I really learned a lot under your guidance.
Yours,
Yang Yingying
Feb 26, 2022 11:02 PM | Andrew Zalesky
RE: Post-hoc testing of One-way RE ANOVA
Hi Yang,
1. Yes - I understand. You are using 4 to model the longer time interval between t2 and t3. This sounds reasonable.
2. Yes - the contrasts seem reasonable. It may be that the connections that go up represent compensatory effects. Without further information about the experiment, it is hard to know.
3. The warning is due to the column of 1s (first column). The column of 1's should be removed - it is not necessary because within-subject means are being modelled. Best not to ignore the warnings. I suggest removing the columns of 1's.
4. In the case of the F-test where 49 edges were found, you are testing for all possible group differences. For example, the F-test will detect t2>t1=t3, but this will not be detected by linear decreasing/increasing design matrix. So you are testing more possibilities with the F-test (Anova).
Andrew
Originally posted by Yang Yingying:
1. Yes - I understand. You are using 4 to model the longer time interval between t2 and t3. This sounds reasonable.
2. Yes - the contrasts seem reasonable. It may be that the connections that go up represent compensatory effects. Without further information about the experiment, it is hard to know.
3. The warning is due to the column of 1s (first column). The column of 1's should be removed - it is not necessary because within-subject means are being modelled. Best not to ignore the warnings. I suggest removing the columns of 1's.
4. In the case of the F-test where 49 edges were found, you are testing for all possible group differences. For example, the F-test will detect t2>t1=t3, but this will not be detected by linear decreasing/increasing design matrix. So you are testing more possibilities with the F-test (Anova).
Andrew
Originally posted by Yang Yingying:
Dear Dr. Zalesky,
Thank you so much for your prompt reply and the helpful suggestions!I still have some questions for help.
My study is evaluate the rats brain FC changes with disease and aging. T1 is early stage of disease in early adulthood; T2 is middle stage of disease in late adulthood; T3 is late stage of disease in aging rats.
I perform the post-hoc testing of One-way RE ANOVA by extracting average FC values across the 49 edges and comparing them between time points. The overall trend of FC is decreasing, but only t1>t3 was statistically significant. I also test a specific hypothesis whether FC time1 > time2 or time2 > time1 using paired-test (p=0.05/3). I found that there are significant edges in t1 > t2, but no significant edges in t1 < t2. Hence, we can confirm that FC strictly liner decreases with time, and I could perform the much power statics.
Design matrix:
1 1 1 0 0
1 1 0 1 0
1 1 0 0 1
1 2 1 0 0
1 2 0 1 0
1 2 0 0 1
1 4 1 0 0
1 4 0 1 0
1 4 0 0 1
Exchange Blocks: 1, 2, 3,1, 2, 3,1, 2, 3. T-test
Q1: Because the time interval between t2 and t3 is just 2 times longer than the gap between t1 and t2, I select the value 4 for t3 in the design matrix. There '4' means longer time interval or greater decline degree of constructions? I really confused by you said 'the increase in connectivity between t2 and t3 is exactly twice that of the increase between t1 and t2'.
Q2: Contrast is [0 -1 0 0 0 0] to test for t1> t2> t3 and [0 1 0 0 0 0] to test for t1 < t2 < t3, I hope I did not make it wrong. Significantly 7 edges were found with contrast -1, while 9 edges with contrast 1. This strange results means there are connections that some go up and some others go down over time, is it reasonable?
Q3: When I perform the NBS, it run successful except pop up orange warnings: 'Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND=1.138690e-18. ...Rank deficient, rank = 10, tol =1.295112e-14'. Can I ignore these warnings?
Q4: Since the linearly decreasing design matrix has greater statistical power, why only 7 edges were found, whereas 49 were were found before? I am really lost.
So much appreciated. I really learned a lot under your guidance.
Yours,
Yang Yingying
Thank you so much for your prompt reply and the helpful suggestions!I still have some questions for help.
My study is evaluate the rats brain FC changes with disease and aging. T1 is early stage of disease in early adulthood; T2 is middle stage of disease in late adulthood; T3 is late stage of disease in aging rats.
I perform the post-hoc testing of One-way RE ANOVA by extracting average FC values across the 49 edges and comparing them between time points. The overall trend of FC is decreasing, but only t1>t3 was statistically significant. I also test a specific hypothesis whether FC time1 > time2 or time2 > time1 using paired-test (p=0.05/3). I found that there are significant edges in t1 > t2, but no significant edges in t1 < t2. Hence, we can confirm that FC strictly liner decreases with time, and I could perform the much power statics.
Design matrix:
1 1 1 0 0
1 1 0 1 0
1 1 0 0 1
1 2 1 0 0
1 2 0 1 0
1 2 0 0 1
1 4 1 0 0
1 4 0 1 0
1 4 0 0 1
Exchange Blocks: 1, 2, 3,1, 2, 3,1, 2, 3. T-test
Q1: Because the time interval between t2 and t3 is just 2 times longer than the gap between t1 and t2, I select the value 4 for t3 in the design matrix. There '4' means longer time interval or greater decline degree of constructions? I really confused by you said 'the increase in connectivity between t2 and t3 is exactly twice that of the increase between t1 and t2'.
Q2: Contrast is [0 -1 0 0 0 0] to test for t1> t2> t3 and [0 1 0 0 0 0] to test for t1 < t2 < t3, I hope I did not make it wrong. Significantly 7 edges were found with contrast -1, while 9 edges with contrast 1. This strange results means there are connections that some go up and some others go down over time, is it reasonable?
Q3: When I perform the NBS, it run successful except pop up orange warnings: 'Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND=1.138690e-18. ...Rank deficient, rank = 10, tol =1.295112e-14'. Can I ignore these warnings?
Q4: Since the linearly decreasing design matrix has greater statistical power, why only 7 edges were found, whereas 49 were were found before? I am really lost.
So much appreciated. I really learned a lot under your guidance.
Yours,
Yang Yingying
Feb 28, 2022 03:02 PM | Yang Yingying
RE: Post-hoc testing of One-way RE ANOVA
Dear Dr. Zalesky,
Thank you so much for your prompt reply and the helpful suggestion!
Best wishes,
Yours,
Yang Yingying
Thank you so much for your prompt reply and the helpful suggestion!
Best wishes,
Yours,
Yang Yingying
Mar 1, 2022 02:03 PM | Yang Yingying
RE: Post-hoc testing of One-way RE ANOVA
Dear Prof. Zalesky,
Would you please tell me whether NBS software could statistic local indicators of graph theory, such as node efficiency (one value per node). I tried, but failed.
Thank you very much.
Best wishes!
Yours,
Yang Yingying
Would you please tell me whether NBS software could statistic local indicators of graph theory, such as node efficiency (one value per node). I tried, but failed.
Thank you very much.
Best wishes!
Yours,
Yang Yingying
Mar 1, 2022 10:03 PM | Andrew Zalesky
RE: Post-hoc testing of One-way RE ANOVA
Hi Yang,
NBS performs inference on connections, not nodes. So it is not suited to nodal measures such as nodal efficiency.
Andrew
Originally posted by Yang Yingying:
NBS performs inference on connections, not nodes. So it is not suited to nodal measures such as nodal efficiency.
Andrew
Originally posted by Yang Yingying:
Dear Prof. Zalesky,
Would you please tell me whether NBS software could statistic local indicators of graph theory, such as node efficiency (one value per node). I tried, but failed.
Thank you very much.
Best wishes!
Yours,
Yang Yingying
Would you please tell me whether NBS software could statistic local indicators of graph theory, such as node efficiency (one value per node). I tried, but failed.
Thank you very much.
Best wishes!
Yours,
Yang Yingying