help
486 Subscribers 
help > 2nd Level Multi-Session Result Interpretation
Showing 1-6 of 6 posts
Jul 14, 2017 03:07 PM | Nicole Nissim - University of Florida
2nd Level Multi-Session Result Interpretation
Hello CONN Users,
I have a question about my second level results and want to be sure I am interpreting the significant ROI-to-ROI results correctly. I have included a screenshot of my results.
Background on the study design: 16 subjects, 6 runs per subject (341 volumes each), 12 conditions, in a 2 (2-back, 0-back) by 3 (pre-, during, post-stimulation) by 2 (active stim vs sham) design (2x2x3). gPPI analysis for ROI-to-ROI results. I added my own ROIs which are labeled 1-15 and correspond to the working memory network.
Brief description of the study: within group design, participants came in for two separate visits (one active stimulation, one sham stimulation inside the scanner during 3 runs of an N-back task, baseline, during, and post-stimulation, thus they have 6 conditions, 3 for active and 3 for sham)
To understand my labeling:
Ba = pre active
Da = during active
Pa = post active
Bs = pre sham
Ds = during sham
Ps = post sham
In the image attached, I have defined my contrast as 1 -1 1 -1 0 0 0 0 0 0 0 0 (selecting 2-back_Ba, 0-back_Ba, 2-Back_Da, and 0-Back_Da in this contrast).
My question: seed 5 has significantly increased connectivity to targets 13, 7, and 6. With this contrast, I am interpreting this to mean: connectivity is increased during active stimulation for 2>0 relative to baseline active 2>0. Is this correct? Or does this result mean: Increased connectivity is occurring in the baseline active condition relative to during active stimulation?
If I select another contrast instead, for example: 0 0 1 -1 0 0 0 0 1 -1 0 0 , and had significant connectivity results, would this mean for 2>0 during active is increased relative to during sham?
I am unclear about the ordering of the different conditions in terms of interpreting the directionality of significant results.
Any help with this would be greatly appreciated!
Best,
Nicole
I have a question about my second level results and want to be sure I am interpreting the significant ROI-to-ROI results correctly. I have included a screenshot of my results.
Background on the study design: 16 subjects, 6 runs per subject (341 volumes each), 12 conditions, in a 2 (2-back, 0-back) by 3 (pre-, during, post-stimulation) by 2 (active stim vs sham) design (2x2x3). gPPI analysis for ROI-to-ROI results. I added my own ROIs which are labeled 1-15 and correspond to the working memory network.
Brief description of the study: within group design, participants came in for two separate visits (one active stimulation, one sham stimulation inside the scanner during 3 runs of an N-back task, baseline, during, and post-stimulation, thus they have 6 conditions, 3 for active and 3 for sham)
To understand my labeling:
Ba = pre active
Da = during active
Pa = post active
Bs = pre sham
Ds = during sham
Ps = post sham
In the image attached, I have defined my contrast as 1 -1 1 -1 0 0 0 0 0 0 0 0 (selecting 2-back_Ba, 0-back_Ba, 2-Back_Da, and 0-Back_Da in this contrast).
My question: seed 5 has significantly increased connectivity to targets 13, 7, and 6. With this contrast, I am interpreting this to mean: connectivity is increased during active stimulation for 2>0 relative to baseline active 2>0. Is this correct? Or does this result mean: Increased connectivity is occurring in the baseline active condition relative to during active stimulation?
If I select another contrast instead, for example: 0 0 1 -1 0 0 0 0 1 -1 0 0 , and had significant connectivity results, would this mean for 2>0 during active is increased relative to during sham?
I am unclear about the ordering of the different conditions in terms of interpreting the directionality of significant results.
Any help with this would be greatly appreciated!
Best,
Nicole
Aug 30, 2017 06:08 AM | Alfonso Nieto-Castanon - Boston University
2nd Level Multi-Session Result Interpretation
Hi Nicole,
The contrast [1 -1 1 -1 0 0 0 0 0 0 0 0] (selecting 2-back_Ba, 0-back_Ba, 2-Back_Da, and 0-Back_Da in this contrast) is looking at the differences in connectivity between 2-back and 0-back conditions, averaged/summed across both the 'Pre-active' and 'During-active' conditions. Positive values there indicate that, on average across pre- and during- stimulation) functional connectivity was higher during 2-back compared to 0-back tasks.
If you want, instead, to look at 2-back vs. 0-back differences in connectivity, and evaluate whether those differences are constant (or not) before- and during- stimulation, the contrast for that would be: [1 -1 -1 1 0 0 0 0 0 0 0 0]
The same applies to the contrast [0 0 1 -1 0 0 0 0 1 -1 0 0], which is now looking at the differences in connectivity between 2-back compared to 0-back tasks, averaged/summed across both the 'during-active stimulation' condition and the 'during-sham stimulation' conditions. If you want, instead to look at whether those 2-back vs. 0-back differences in connectivity are themselves different during active simulation compared to during sham stimulation, the contrast for that would be: [0 0 1 -1 0 0 0 0 -1 1 0 0]
In general, since you have a full factorial design, you can break-down the contrast that you care about into three parts:
A) memory-load contrast (two values: across 2-back and 0-back levels): e.g. [1 0] to look at 2-back connectivity only, [0 1] to look at 0-back connectivity only, [1 -1] to look at differences between 2-back vs. 0-back connectivity, or [.5 .5] to look at average 2-back and 0-back connectivity
B) time contrast (three values: across pre- during- and post- stimulation levels): e.g. [1 0 0] to look at pre-stimulation connectivity only, [-1 1 0] to look at differences between during- and pre- stimulation, [-1 0 1] to look at differences in connectivity between post- vs. pre- stimulation, etc.
and C) stimulation contrast (two values: across the active and sham stimulation levels): e.g. [1 0] to look at connectivity in the active stimulation condition only, [1 -1] to look at connectivity differences between the active compared to sham stimulation conditions
Once you have defined these three contrast A,B, and C, use the syntax in Matlab command-line (or in the 'between-conditions contrast' GUI field):
kron(C, kron(B, A))
to get the actual contrast vector. For example, in your second example:
A) [1 -1] (to look at differences in connectivity between 2-back vs. 0-back conditions)
B) [0 1 0] (to look only at the effects during-stimulation (disregarding pre- and post- stimulation)
C) [1 -1] (to look at differences in connectivity between the active vs. sham stimulation conditions)
could be typed as the contrast:
kron( [1 -1] , kron( [0 1 0] , [1 -1] ))
which would result in the contrast vector:
[0 0 1 -1 0 0 0 0 -1 1 0 0]
Hope this helps
Alfonso
Originally posted by Nicole Nissim:
The contrast [1 -1 1 -1 0 0 0 0 0 0 0 0] (selecting 2-back_Ba, 0-back_Ba, 2-Back_Da, and 0-Back_Da in this contrast) is looking at the differences in connectivity between 2-back and 0-back conditions, averaged/summed across both the 'Pre-active' and 'During-active' conditions. Positive values there indicate that, on average across pre- and during- stimulation) functional connectivity was higher during 2-back compared to 0-back tasks.
If you want, instead, to look at 2-back vs. 0-back differences in connectivity, and evaluate whether those differences are constant (or not) before- and during- stimulation, the contrast for that would be: [1 -1 -1 1 0 0 0 0 0 0 0 0]
The same applies to the contrast [0 0 1 -1 0 0 0 0 1 -1 0 0], which is now looking at the differences in connectivity between 2-back compared to 0-back tasks, averaged/summed across both the 'during-active stimulation' condition and the 'during-sham stimulation' conditions. If you want, instead to look at whether those 2-back vs. 0-back differences in connectivity are themselves different during active simulation compared to during sham stimulation, the contrast for that would be: [0 0 1 -1 0 0 0 0 -1 1 0 0]
In general, since you have a full factorial design, you can break-down the contrast that you care about into three parts:
A) memory-load contrast (two values: across 2-back and 0-back levels): e.g. [1 0] to look at 2-back connectivity only, [0 1] to look at 0-back connectivity only, [1 -1] to look at differences between 2-back vs. 0-back connectivity, or [.5 .5] to look at average 2-back and 0-back connectivity
B) time contrast (three values: across pre- during- and post- stimulation levels): e.g. [1 0 0] to look at pre-stimulation connectivity only, [-1 1 0] to look at differences between during- and pre- stimulation, [-1 0 1] to look at differences in connectivity between post- vs. pre- stimulation, etc.
and C) stimulation contrast (two values: across the active and sham stimulation levels): e.g. [1 0] to look at connectivity in the active stimulation condition only, [1 -1] to look at connectivity differences between the active compared to sham stimulation conditions
Once you have defined these three contrast A,B, and C, use the syntax in Matlab command-line (or in the 'between-conditions contrast' GUI field):
kron(C, kron(B, A))
to get the actual contrast vector. For example, in your second example:
A) [1 -1] (to look at differences in connectivity between 2-back vs. 0-back conditions)
B) [0 1 0] (to look only at the effects during-stimulation (disregarding pre- and post- stimulation)
C) [1 -1] (to look at differences in connectivity between the active vs. sham stimulation conditions)
could be typed as the contrast:
kron( [1 -1] , kron( [0 1 0] , [1 -1] ))
which would result in the contrast vector:
[0 0 1 -1 0 0 0 0 -1 1 0 0]
Hope this helps
Alfonso
Originally posted by Nicole Nissim:
Hello CONN Users,
I have a question about my second level results and want to be sure I am interpreting the significant ROI-to-ROI results correctly. I have included a screenshot of my results.
Background on the study design: 16 subjects, 6 runs per subject (341 volumes each), 12 conditions, in a 2 (2-back, 0-back) by 3 (pre-, during, post-stimulation) by 2 (active stim vs sham) design (2x2x3). gPPI analysis for ROI-to-ROI results. I added my own ROIs which are labeled 1-15 and correspond to the working memory network.
Brief description of the study: within group design, participants came in for two separate visits (one active stimulation, one sham stimulation inside the scanner during 3 runs of an N-back task, baseline, during, and post-stimulation, thus they have 6 conditions, 3 for active and 3 for sham)
To understand my labeling:
Ba = pre active
Da = during active
Pa = post active
Bs = pre sham
Ds = during sham
Ps = post sham
In the image attached, I have defined my contrast as 1 -1 1 -1 0 0 0 0 0 0 0 0 (selecting 2-back_Ba, 0-back_Ba, 2-Back_Da, and 0-Back_Da in this contrast).
My question: seed 5 has significantly increased connectivity to targets 13, 7, and 6. With this contrast, I am interpreting this to mean: connectivity is increased during active stimulation for 2>0 relative to baseline active 2>0. Is this correct? Or does this result mean: Increased connectivity is occurring in the baseline active condition relative to during active stimulation?
If I select another contrast instead, for example: 0 0 1 -1 0 0 0 0 1 -1 0 0 , and had significant connectivity results, would this mean for 2>0 during active is increased relative to during sham?
I am unclear about the ordering of the different conditions in terms of interpreting the directionality of significant results.
Any help with this would be greatly appreciated!
Best,
Nicole
I have a question about my second level results and want to be sure I am interpreting the significant ROI-to-ROI results correctly. I have included a screenshot of my results.
Background on the study design: 16 subjects, 6 runs per subject (341 volumes each), 12 conditions, in a 2 (2-back, 0-back) by 3 (pre-, during, post-stimulation) by 2 (active stim vs sham) design (2x2x3). gPPI analysis for ROI-to-ROI results. I added my own ROIs which are labeled 1-15 and correspond to the working memory network.
Brief description of the study: within group design, participants came in for two separate visits (one active stimulation, one sham stimulation inside the scanner during 3 runs of an N-back task, baseline, during, and post-stimulation, thus they have 6 conditions, 3 for active and 3 for sham)
To understand my labeling:
Ba = pre active
Da = during active
Pa = post active
Bs = pre sham
Ds = during sham
Ps = post sham
In the image attached, I have defined my contrast as 1 -1 1 -1 0 0 0 0 0 0 0 0 (selecting 2-back_Ba, 0-back_Ba, 2-Back_Da, and 0-Back_Da in this contrast).
My question: seed 5 has significantly increased connectivity to targets 13, 7, and 6. With this contrast, I am interpreting this to mean: connectivity is increased during active stimulation for 2>0 relative to baseline active 2>0. Is this correct? Or does this result mean: Increased connectivity is occurring in the baseline active condition relative to during active stimulation?
If I select another contrast instead, for example: 0 0 1 -1 0 0 0 0 1 -1 0 0 , and had significant connectivity results, would this mean for 2>0 during active is increased relative to during sham?
I am unclear about the ordering of the different conditions in terms of interpreting the directionality of significant results.
Any help with this would be greatly appreciated!
Best,
Nicole
Sep 7, 2017 08:09 PM | Nicole Nissim - University of Florida
RE: 2nd Level Multi-Session Results
Hi Alfonso,
Thank you so much for the reply, this has cleared up some things.
We are really trying to get at the between conditions contrast, so to look at 2-back active over 0-back active (2Da>0Da) Versus 2-back sham over 0-back sham (2Ds>0Ds) for each condition of time (ex. 2Da>0Da versus 2Ds>0Ds; 2Da>0Da versus 2Ba>0Ba and the same for post-active, baseline sham, and post-sham etc.)
The contrast vector you mentioned that will let us look at 2 vs 0 back differences during active compared to sham as: 0 0 1 -1 0 0 0 0 -1 1 0 0
I am unclear about how it is grouping the values for that. If it is summing/averaging the 1's in that contrast (and the -1 together), is that contrast putting together 2back during active stimulation + zero back during sham, and comparing that to 0-back during active + 2-Back during sham?
Is it possible to look at these contrasts the way we want to as we have set up this experiment in conn?
Best regards,
Nicole
Thank you so much for the reply, this has cleared up some things.
We are really trying to get at the between conditions contrast, so to look at 2-back active over 0-back active (2Da>0Da) Versus 2-back sham over 0-back sham (2Ds>0Ds) for each condition of time (ex. 2Da>0Da versus 2Ds>0Ds; 2Da>0Da versus 2Ba>0Ba and the same for post-active, baseline sham, and post-sham etc.)
The contrast vector you mentioned that will let us look at 2 vs 0 back differences during active compared to sham as: 0 0 1 -1 0 0 0 0 -1 1 0 0
I am unclear about how it is grouping the values for that. If it is summing/averaging the 1's in that contrast (and the -1 together), is that contrast putting together 2back during active stimulation + zero back during sham, and comparing that to 0-back during active + 2-Back during sham?
Is it possible to look at these contrasts the way we want to as we have set up this experiment in conn?
Best regards,
Nicole
Sep 8, 2017 03:09 AM | Jeff Browndyke
2nd Level Multi-Session Result Interpretation
========================================================================
Added from other thread as it has direct applicability to Nicole's question and possible needs for post-hoc explanation
========================================================================
Hi, Alfonso.
Just to tag onto your response regarding the three-way Group X Condition x Condition interaction model and set-up in CONN.
As you know, we performed something similar for our patients and controls at two time points (baseline and follow-up) looking at a working memory load effect (2-back > 1-back). We found a region for this three-way group x time x working memory load contrast interaction, and we have compared that region with performance on a separate measure of cognition. The results were written up and sent off for review, and I received the following question from reviewer with respect to the CONN model and three-way interaction:
"Please provide justification for the (unusual) approach to test an interaction analysis on the first level. Which analyses were conducted on the second level? Two-sample t-test? How do you know what the results on the second level mean? There are many possibilities that can produce a significant interaction effect. Thus, you need to show post-hoc t-tests to disentangle interaction findings."
Any ideas on how to best address these questions? My understanding is that CONN is comparing the difference of time and condition factors between groups at the 2nd level. Are post-hoc t-tests even necessary to "disentangle interaction findings" when handled as t-tests of difference images?
Warm regards,
Jeff
Added from other thread as it has direct applicability to Nicole's question and possible needs for post-hoc explanation
========================================================================
Hi, Alfonso.
Just to tag onto your response regarding the three-way Group X Condition x Condition interaction model and set-up in CONN.
As you know, we performed something similar for our patients and controls at two time points (baseline and follow-up) looking at a working memory load effect (2-back > 1-back). We found a region for this three-way group x time x working memory load contrast interaction, and we have compared that region with performance on a separate measure of cognition. The results were written up and sent off for review, and I received the following question from reviewer with respect to the CONN model and three-way interaction:
"Please provide justification for the (unusual) approach to test an interaction analysis on the first level. Which analyses were conducted on the second level? Two-sample t-test? How do you know what the results on the second level mean? There are many possibilities that can produce a significant interaction effect. Thus, you need to show post-hoc t-tests to disentangle interaction findings."
Any ideas on how to best address these questions? My understanding is that CONN is comparing the difference of time and condition factors between groups at the 2nd level. Are post-hoc t-tests even necessary to "disentangle interaction findings" when handled as t-tests of difference images?
Warm regards,
Jeff
Sep 16, 2017 11:09 PM | Alfonso Nieto-Castanon - Boston University
2nd Level Multi-Session Result Interpretation
Hi Jeff,
Regarding the first reviewer question, the approach is not unusual at all, it is in fact rather standard (you may cite Henson and Penny's "ANOVAs and SPM" as one of the reference first introducing this approach in the neuroimaging literature, but the actual math equivalences behind this and described in https://www.nitrc.org/forum/message.php?... date back literally centuries)
Regarding the interpretation and need for post-hoc analyses, if I am interpreting it correctly the reviewer is simply right and this applies to all ANOVA and/or factorial designs/analyses. This is already true for a simple one-way ANOVAs (e.g. if you find a significant difference between the groups in connectivity, how do you interpret it? is the connection present in one group and not in the other? or is it present in both but with different strengths? are these positive or anticorrelations that we are talking about? etc. all of these things affect your interpretation and all of them are consistent with the found significant main effect). In two-way, three-way, or higher-order ANOVAs the need for interpretation, and the amount of different interpretations consistent with a significant interaction, simply grows exponentially. For example, your three-way group x time x working-memory-load interaction, would be consistent with many different scenarios. E.g. it might be that in control subjects, the differences in connectivity between the 2-back and 1-back tasks are constant in time (baseline and follow-up) while in patients those differences in connectivity decrease/increase? or it might be that in control subjects there are no differences at all in connectivity between the 2-back and 1-back tasks at neither baseline nor follow-up, while in patients there is are abnormal connectivity values in 2-back at baseline only? or many/many other potential scenarios all consistent with a significant three-way interaction but with different interpreations or clinical consequences. So the recommendation in ANOVA is always to display the connectivity in each of the individual cells (in your case 8 cells = 2x2x2) and base your interpretation on those or additional post-hoc analyses.
Hope this helps
Alfonso
Originally posted by Jeff Browndyke:
Regarding the first reviewer question, the approach is not unusual at all, it is in fact rather standard (you may cite Henson and Penny's "ANOVAs and SPM" as one of the reference first introducing this approach in the neuroimaging literature, but the actual math equivalences behind this and described in https://www.nitrc.org/forum/message.php?... date back literally centuries)
Regarding the interpretation and need for post-hoc analyses, if I am interpreting it correctly the reviewer is simply right and this applies to all ANOVA and/or factorial designs/analyses. This is already true for a simple one-way ANOVAs (e.g. if you find a significant difference between the groups in connectivity, how do you interpret it? is the connection present in one group and not in the other? or is it present in both but with different strengths? are these positive or anticorrelations that we are talking about? etc. all of these things affect your interpretation and all of them are consistent with the found significant main effect). In two-way, three-way, or higher-order ANOVAs the need for interpretation, and the amount of different interpretations consistent with a significant interaction, simply grows exponentially. For example, your three-way group x time x working-memory-load interaction, would be consistent with many different scenarios. E.g. it might be that in control subjects, the differences in connectivity between the 2-back and 1-back tasks are constant in time (baseline and follow-up) while in patients those differences in connectivity decrease/increase? or it might be that in control subjects there are no differences at all in connectivity between the 2-back and 1-back tasks at neither baseline nor follow-up, while in patients there is are abnormal connectivity values in 2-back at baseline only? or many/many other potential scenarios all consistent with a significant three-way interaction but with different interpreations or clinical consequences. So the recommendation in ANOVA is always to display the connectivity in each of the individual cells (in your case 8 cells = 2x2x2) and base your interpretation on those or additional post-hoc analyses.
Hope this helps
Alfonso
Originally posted by Jeff Browndyke:
========================================================================
Added from other thread as it has direct applicability to Nicole's question and possible needs for post-hoc explanation
========================================================================
Hi, Alfonso.
Just to tag onto your response regarding the three-way Group X Condition x Condition interaction model and set-up in CONN.
As you know, we performed something similar for our patients and controls at two time points (baseline and follow-up) looking at a working memory load effect (2-back > 1-back). We found a region for this three-way group x time x working memory load contrast interaction, and we have compared that region with performance on a separate measure of cognition. The results were written up and sent off for review, and I received the following question from reviewer with respect to the CONN model and three-way interaction:
"Please provide justification for the (unusual) approach to test an interaction analysis on the first level. Which analyses were conducted on the second level? Two-sample t-test? How do you know what the results on the second level mean? There are many possibilities that can produce a significant interaction effect. Thus, you need to show post-hoc t-tests to disentangle interaction findings."
Any ideas on how to best address these questions? My understanding is that CONN is comparing the difference of time and condition factors between groups at the 2nd level. Are post-hoc t-tests even necessary to "disentangle interaction findings" when handled as t-tests of difference images?
Warm regards,
Jeff
Added from other thread as it has direct applicability to Nicole's question and possible needs for post-hoc explanation
========================================================================
Hi, Alfonso.
Just to tag onto your response regarding the three-way Group X Condition x Condition interaction model and set-up in CONN.
As you know, we performed something similar for our patients and controls at two time points (baseline and follow-up) looking at a working memory load effect (2-back > 1-back). We found a region for this three-way group x time x working memory load contrast interaction, and we have compared that region with performance on a separate measure of cognition. The results were written up and sent off for review, and I received the following question from reviewer with respect to the CONN model and three-way interaction:
"Please provide justification for the (unusual) approach to test an interaction analysis on the first level. Which analyses were conducted on the second level? Two-sample t-test? How do you know what the results on the second level mean? There are many possibilities that can produce a significant interaction effect. Thus, you need to show post-hoc t-tests to disentangle interaction findings."
Any ideas on how to best address these questions? My understanding is that CONN is comparing the difference of time and condition factors between groups at the 2nd level. Are post-hoc t-tests even necessary to "disentangle interaction findings" when handled as t-tests of difference images?
Warm regards,
Jeff
Sep 17, 2017 05:09 PM | Jeff Browndyke
2nd Level Multi-Session Result Interpretation
Thanks, Alfonso. We did reference the Henson and Penny
materials with the following write up:
"A two-stage partitioned variance approach mixed within-/between-subjects ANCOVA model (Henson, 2015; Henson and Penny, 2003) was used to examine for a three-way interaction in ICC and ILC values between groups (i.e., between-subject factor; surgical patients > non-surgical controls) and two within-subject factors (e.g., time = 6-week > baseline; and working memory load = 2-back > 1-back). In this two-stage analysis approach, interaction images were first computed for each subject, after which they were entered into a second-level, random effects two-sample analysis with covariates in SPM12."
I hope I explained the Henson & Penny method correctly, and I guess the reviewer isn't familiar with this approach?
As for the need for follow-up post-hoc analyses, we found a significant region associated with the group (-1 1) x time-condition (-1 1 1 -1) interaction. I plotted the interaction in a figure, but as you and the reviewer suggest I would need to show post-hoc analyses. What contrasts would you recommend to tease the interaction apart? I will plot the bars for each group x condition x time, but would I suspect it would be preferable to also have hard numbers and results to support the interaction.
Thanks again,
Jeff
"A two-stage partitioned variance approach mixed within-/between-subjects ANCOVA model (Henson, 2015; Henson and Penny, 2003) was used to examine for a three-way interaction in ICC and ILC values between groups (i.e., between-subject factor; surgical patients > non-surgical controls) and two within-subject factors (e.g., time = 6-week > baseline; and working memory load = 2-back > 1-back). In this two-stage analysis approach, interaction images were first computed for each subject, after which they were entered into a second-level, random effects two-sample analysis with covariates in SPM12."
I hope I explained the Henson & Penny method correctly, and I guess the reviewer isn't familiar with this approach?
As for the need for follow-up post-hoc analyses, we found a significant region associated with the group (-1 1) x time-condition (-1 1 1 -1) interaction. I plotted the interaction in a figure, but as you and the reviewer suggest I would need to show post-hoc analyses. What contrasts would you recommend to tease the interaction apart? I will plot the bars for each group x condition x time, but would I suspect it would be preferable to also have hard numbers and results to support the interaction.
Thanks again,
Jeff