help
help > RE: second-level analysis in Conn
Oct 16, 2015 04:10 PM | Diana Parvinchi - McMaster University
RE: second-level analysis in Conn
Hi Alfonso,
Yes, in the Calculator display, I select my group 2 & score 2 from the predictor variables, enter the contrast [0 1] and select my FC variable as my outcome variable. Thanks very much for looking into this! Looking forward to hearing back from you.
Best,
Diana.
Originally posted by Alfonso Nieto-Castanon:
Yes, in the Calculator display, I select my group 2 & score 2 from the predictor variables, enter the contrast [0 1] and select my FC variable as my outcome variable. Thanks very much for looking into this! Looking forward to hearing back from you.
Best,
Diana.
Originally posted by Alfonso Nieto-Castanon:
Hi
Diana,
That is curious, the Calculator display will attempt to somehow "smartly" only display R2 values when they are relevant (i.e. when you have a regression-like equation) so perhaps the logic behind that code is not as "smart" as intended. I will double-check the code to see if I can figure out what might be causing this behavior; in the meantime could you please confirm that for your Group2 analyses you are selecting "Group2" and "Score2" as your predictor variables (contrast [0 1]) and your FC variable as your outcome variable?
Thanks
Alfonso
Originally posted by Diana Parvinchi:
That is curious, the Calculator display will attempt to somehow "smartly" only display R2 values when they are relevant (i.e. when you have a regression-like equation) so perhaps the logic behind that code is not as "smart" as intended. I will double-check the code to see if I can figure out what might be causing this behavior; in the meantime could you please confirm that for your Group2 analyses you are selecting "Group2" and "Score2" as your predictor variables (contrast [0 1]) and your FC variable as your outcome variable?
Thanks
Alfonso
Originally posted by Diana Parvinchi:
Hi
Alfonso,
Thank you very much for writing back. In the second level results window, I select my groups (group 1, group 2, group 3) and their scores (score1, score2, score3), enter the contrast [0 0 0 1 0 0;0 0 0 0 1 0; 0 0 0 0 0 1], select all of our ROIs (36) and leave the default eye (n) in the between sources contrast (this is to investigate association between FC and symptom severity in three age groups of cohorts with ASD). I don't enter "all subjects" in the Subject effects list, is that ok? I also looked in Setup.CovariatesSecondLevel and did not see NaN values for other subjects. Why do you think the R2 is missing in the calculator window for group 2 only (we get stats and the regression graph, but not R2 for this group)? Many many thanks for your help.
Best,
Diana.
Originally posted by Alfonso Nieto-Castanon:
Thank you very much for writing back. In the second level results window, I select my groups (group 1, group 2, group 3) and their scores (score1, score2, score3), enter the contrast [0 0 0 1 0 0;0 0 0 0 1 0; 0 0 0 0 0 1], select all of our ROIs (36) and leave the default eye (n) in the between sources contrast (this is to investigate association between FC and symptom severity in three age groups of cohorts with ASD). I don't enter "all subjects" in the Subject effects list, is that ok? I also looked in Setup.CovariatesSecondLevel and did not see NaN values for other subjects. Why do you think the R2 is missing in the calculator window for group 2 only (we get stats and the regression graph, but not R2 for this group)? Many many thanks for your help.
Best,
Diana.
Originally posted by Alfonso Nieto-Castanon:
Hi
Diana,
My guess would be that perhaps you used the 'import values' on an analysis that was only looking at Group1 subjects. If that was the case then the imported covariate will contain the connectivity values only for those subjects (and the rest of subjects are filled with NaN's; you may double-check this by looking in Setup.CovariatesSecondLevel and seeing whether the extracted covariates contain NaN values for other subjects). If that was the case then delete those partial second-level covariates, and simply go back to the Second-level results ROI-to-ROI tab with your original analysis, select the target regions of interest (e.g. the ones showing a significant effect in these analyses), then change the subject-effects list to select only the 'AllSubjects' effect before using 'import values' to now extract the connectivity values for all subjects instead.
Let me know if that seems to fix the missing-R2 issue.
Best
Alfonso
Originally posted by Diana Parvinchi:
My guess would be that perhaps you used the 'import values' on an analysis that was only looking at Group1 subjects. If that was the case then the imported covariate will contain the connectivity values only for those subjects (and the rest of subjects are filled with NaN's; you may double-check this by looking in Setup.CovariatesSecondLevel and seeing whether the extracted covariates contain NaN values for other subjects). If that was the case then delete those partial second-level covariates, and simply go back to the Second-level results ROI-to-ROI tab with your original analysis, select the target regions of interest (e.g. the ones showing a significant effect in these analyses), then change the subject-effects list to select only the 'AllSubjects' effect before using 'import values' to now extract the connectivity values for all subjects instead.
Let me know if that seems to fix the missing-R2 issue.
Best
Alfonso
Originally posted by Diana Parvinchi:
Hi Alfonso,
This is a follow up question to our discussion below. I've selected all of the seed ROIs in the "sources" list and performed an F-test across all of our seeds. In order to explore each significant effect further (to determine where the effect is coming from), I select each significant effect from the analysis results and imported the values. Then, I use the option "calculator" to explore the effects further. In the calculator window, I select my group and their score (e.g. group 1, score 1), from the predictor variables, and enter the contrast [0 1] in the Between-subjects contrast and select a single pair of ROI from the list of outcome variables. A set of stats and a regression graph is provided here. Above the graph the R2 value is also displayed. However, for some reason, the R2 value is not displayed for my group 2, consistently. I'm not sure why and hoping that you could help me get this value. How can I get Conn to display the R2 for my group 2 as well?
Thank you very much for your help.
Best,
Diana.
Originally posted by Diana Parvinchi:
This is a follow up question to our discussion below. I've selected all of the seed ROIs in the "sources" list and performed an F-test across all of our seeds. In order to explore each significant effect further (to determine where the effect is coming from), I select each significant effect from the analysis results and imported the values. Then, I use the option "calculator" to explore the effects further. In the calculator window, I select my group and their score (e.g. group 1, score 1), from the predictor variables, and enter the contrast [0 1] in the Between-subjects contrast and select a single pair of ROI from the list of outcome variables. A set of stats and a regression graph is provided here. Above the graph the R2 value is also displayed. However, for some reason, the R2 value is not displayed for my group 2, consistently. I'm not sure why and hoping that you could help me get this value. How can I get Conn to display the R2 for my group 2 as well?
Thank you very much for your help.
Best,
Diana.
Originally posted by Diana Parvinchi:
Hi Alfonso,
That's very helpful. Thank you very much! We deeply appreciate your help. Many thanks.
Best,
Diana.
Originally posted by Alfonso Nieto-Castanon:
That's very helpful. Thank you very much! We deeply appreciate your help. Many thanks.
Best,
Diana.
Originally posted by Alfonso Nieto-Castanon:
Hi
Diana,
Some thoughts on your questions below
Best
Alfonso
Originally posted by Diana Parvinchi:
No, the "analysis results" table multiple-comparison correction only corrects at the analysis-level (i.e. for that particular analysis -with a particular seed ROI and particular symptom score- it corrects across multiple target ROIs). Since you are performing multiple analyses (e.g. you are repeating this procedure for multiple seed ROIs) you would typically need to apply an additional correction to take that into account (unless each repeated analysis can be reasonably framed as a separate individual experimental hypothesis). Typically, in addition to simply using a more conservative Bonferroni-corrected set of individual thresholds, there are a couple of more-senstive ways to correct across multiple seed ROIs:
a) if the set of seed ROIs is not too large, you may simply perform an F-test across all of your seeds: i.e. repeat your analyses, but instead of selecting a single seed ROI in the "sources" list select now all of the seed ROIs that you want to test simultaneously and leave the default eye(N) between-sources contrast. That will test for connectivity associations with clinical scores across any of the selected seed ROIs
or b) alternatively click on "results explorer", then select in the "sources" list all of your seed ROIs. You will be now looking at the matrix of effects between all selected seed ROIs and all selected target ROIs, and you have multiple ways to threshold the results here while properly correcting for multiple comparisons (e.g. using seed-level thresholds as well as connection-level thresholds, using network-based-statistics, etc.)
2) Given that the symptoms severity scores are not standardized, meaning a score of "0" indicates the absence of that symptom, not the average, is our interpretation (e.g. functional connectivity between right Intracalcarine and right Frontal Orbital Cortex is negatively correlated with severity of Repetitive Behaviour in our third group) correct in the attached graph?
Yes, the centering of the symptom score variables does not affect the interpretation of the sign of the association (between connectivity values and symptom scores). What changes with centering is the interpretation of the y-intercepts (the first three bars in your plot). In your case it means that the connectivity between those regions is typically (in the absence of symptoms) positive, and that this positive connectivity decreases with severity of symptoms.
3) When we see a significant effect in one of our three age-groups, how can I run pairwise comparisons (following an interaction effect) to report the p-value associated with that particular effect? For example, in the attached graph, there's significant correlation between connectivity and repetitive behaviour only in our third group, how can I determine the p-value for this particular effect?
To look at the association between connectivity and symptom scores only in your third group, simply select the 'Group3' and 'Group3_scores' effects in the "subject-effects" list and enter a [0 1] between-subjects contrast.
Hope this helps
Alfonso
Some thoughts on your questions below
Best
Alfonso
Originally posted by Diana Parvinchi:
Hi Alfonso,
Thank you very much for all your help. I have three questions and deeply appreciate your help in confirming these issues before we submit the results for publication purposes. Just to refresh your mind, we examined the correlation between functional connectivity and core symptoms of autism spectrum disorder in three age-groups of participants. This is how I analyzed the data - followed by my three questions:
I selected 6 regressors (three age-groups and their corresponding scores on a given clinical measure) from the "subject effects" list in the 2nd-level results (ROI to ROI), entered the contrast [0 0 0 1 0 0;0 0 0 0 1 0; 0 0 0 0 0 1] in the "between-groups contrasts", then selected a single ROI from our list of ROIs and ran the contrast. I repeated this process for each ROI in our list (for each symptom - total of 5 symptoms).
1) Are the results shown in the "analysis results" table corrected for multiple comparisons, given the route I took explained above?
Thank you very much for all your help. I have three questions and deeply appreciate your help in confirming these issues before we submit the results for publication purposes. Just to refresh your mind, we examined the correlation between functional connectivity and core symptoms of autism spectrum disorder in three age-groups of participants. This is how I analyzed the data - followed by my three questions:
I selected 6 regressors (three age-groups and their corresponding scores on a given clinical measure) from the "subject effects" list in the 2nd-level results (ROI to ROI), entered the contrast [0 0 0 1 0 0;0 0 0 0 1 0; 0 0 0 0 0 1] in the "between-groups contrasts", then selected a single ROI from our list of ROIs and ran the contrast. I repeated this process for each ROI in our list (for each symptom - total of 5 symptoms).
1) Are the results shown in the "analysis results" table corrected for multiple comparisons, given the route I took explained above?
No, the "analysis results" table multiple-comparison correction only corrects at the analysis-level (i.e. for that particular analysis -with a particular seed ROI and particular symptom score- it corrects across multiple target ROIs). Since you are performing multiple analyses (e.g. you are repeating this procedure for multiple seed ROIs) you would typically need to apply an additional correction to take that into account (unless each repeated analysis can be reasonably framed as a separate individual experimental hypothesis). Typically, in addition to simply using a more conservative Bonferroni-corrected set of individual thresholds, there are a couple of more-senstive ways to correct across multiple seed ROIs:
a) if the set of seed ROIs is not too large, you may simply perform an F-test across all of your seeds: i.e. repeat your analyses, but instead of selecting a single seed ROI in the "sources" list select now all of the seed ROIs that you want to test simultaneously and leave the default eye(N) between-sources contrast. That will test for connectivity associations with clinical scores across any of the selected seed ROIs
or b) alternatively click on "results explorer", then select in the "sources" list all of your seed ROIs. You will be now looking at the matrix of effects between all selected seed ROIs and all selected target ROIs, and you have multiple ways to threshold the results here while properly correcting for multiple comparisons (e.g. using seed-level thresholds as well as connection-level thresholds, using network-based-statistics, etc.)
2) Given that the symptoms severity scores are not standardized, meaning a score of "0" indicates the absence of that symptom, not the average, is our interpretation (e.g. functional connectivity between right Intracalcarine and right Frontal Orbital Cortex is negatively correlated with severity of Repetitive Behaviour in our third group) correct in the attached graph?
Yes, the centering of the symptom score variables does not affect the interpretation of the sign of the association (between connectivity values and symptom scores). What changes with centering is the interpretation of the y-intercepts (the first three bars in your plot). In your case it means that the connectivity between those regions is typically (in the absence of symptoms) positive, and that this positive connectivity decreases with severity of symptoms.
3) When we see a significant effect in one of our three age-groups, how can I run pairwise comparisons (following an interaction effect) to report the p-value associated with that particular effect? For example, in the attached graph, there's significant correlation between connectivity and repetitive behaviour only in our third group, how can I determine the p-value for this particular effect?
To look at the association between connectivity and symptom scores only in your third group, simply select the 'Group3' and 'Group3_scores' effects in the "subject-effects" list and enter a [0 1] between-subjects contrast.
Hope this helps
Alfonso