help
489 Subscribers 
help > ICA: Parameter Choice and 2nd Level Effects ?
Showing 1-19 of 19 posts
Sep 18, 2016 06:09 PM | Shady El Damaty - Georgetown University
ICA: Parameter Choice and 2nd Level Effects ?
Dear fellow CONN-artists,
I've recently begun diving in ICA based methods for extracting functional networks in a modestly large data set (110 subjects, 3 time points). One major question when running ICA is the choice of parameters for data reduction and # of components to be estimated.
The Calhoun paper presents an AIC and MDL approach for performing these estimates. Is this algorithm implemented in CONN? If not, how would one go about extracting the parameters to compute the AIC and MDL? The major parameters of interest would be the log of the maximum likelihood estimate of the model parameters as estimated from the fMRI data, ML (the number of time points following individual subject data reduction) and the number of sources N.
Also, is there an easy way to reconstruct individual subject ICA maps? How would one go about reconstructing individual subject time courses for time course ICA decomposition?
Additionally, how should one pick a Z value threshold? Calhoun suggests calculating the mean and variance of each component across the subjects and then perform random effects inference for a one-sample t-test. Using a standard 0.001 threshold often results in over-saturated maps for some components, whereas other components might tolerate only low thresholds. How does one go about picking a consistent threshold across components?
And lastly, what exactly happens when you contrast an ICA component at the group level? I've performed a contrast between two subject groups (drug users vs non users, [1 -1]) with a single ICA component selected. However the difference between those two subject groups is a cluster that falls outside of the spatial extent of that ICA component (after thresholding). Shouldn't the difference between these two subjects be constrained to only the spatial extent of the chosen ICA component?
I've recently begun diving in ICA based methods for extracting functional networks in a modestly large data set (110 subjects, 3 time points). One major question when running ICA is the choice of parameters for data reduction and # of components to be estimated.
The Calhoun paper presents an AIC and MDL approach for performing these estimates. Is this algorithm implemented in CONN? If not, how would one go about extracting the parameters to compute the AIC and MDL? The major parameters of interest would be the log of the maximum likelihood estimate of the model parameters as estimated from the fMRI data, ML (the number of time points following individual subject data reduction) and the number of sources N.
Also, is there an easy way to reconstruct individual subject ICA maps? How would one go about reconstructing individual subject time courses for time course ICA decomposition?
Additionally, how should one pick a Z value threshold? Calhoun suggests calculating the mean and variance of each component across the subjects and then perform random effects inference for a one-sample t-test. Using a standard 0.001 threshold often results in over-saturated maps for some components, whereas other components might tolerate only low thresholds. How does one go about picking a consistent threshold across components?
And lastly, what exactly happens when you contrast an ICA component at the group level? I've performed a contrast between two subject groups (drug users vs non users, [1 -1]) with a single ICA component selected. However the difference between those two subject groups is a cluster that falls outside of the spatial extent of that ICA component (after thresholding). Shouldn't the difference between these two subjects be constrained to only the spatial extent of the chosen ICA component?
Sep 28, 2016 06:09 PM | Shady El Damaty - Georgetown University
RE: ICA: Parameter Choice and 2nd Level Effects ?
bump
Sep 29, 2016 02:09 AM | Alfonso Nieto-Castanon - Boston University
RE: ICA: Parameter Choice and 2nd Level Effects ?
Dear Shady
The current ICA implementation in CONN does not include any model-selection approach to help determine the "optimal" number of components in ICA (we will likely end up offering some form of cross-validation approach to help determine the relative significance of different components, but that is still in the works). If you need to compute any of the AIC or MDL model-selection measures, you can probably do that from the group-level spatial maps stored in ICA.ROIs.nii and the associated timeseries stored in ICA.Timeseries.mat (but this is not really straightforward to do).
Regarding your question about reconstructing individual-subject ICA maps (back-projection), those are already computed by CONN, they are stored in the files named BETA_Subject*_Condition*_Measure*.nii, and these are the volumes that are entered into your second-level analyses when looking at the ICA.SpatialComponents tab in the second-level results window. If, on the other hand, you mean that you would like to reconstruct individual-subject ICA maps for other subjects (other than those included in your original ICA analysis), then the way to do this would be to go to Setup.ROIs and click on the 'ROI tools. Add network-ICA ROIs' button. This will add a new weighted-ROI that represents the group-level spatial maps computed in the group-ICA step. Using these network-ROIs as seeds in a new first-level analysis that uses multivariate-regression measures will produce the same individual-subject ICA maps as those computed in the back-projection step, and this can be applied to the same or other subjects (not necessarily those included in the original ICA analyses).
Regarding your question about Z-value thresholds, in CONN this approach can be performed simply by going to the ICA.SpatialComponents tab in the second-level results window, and defining a new one-sample t-test second-level analysis (i.e. simply select the 'AllSubjects' effect, contrast 1). The choice of threshold in those one-sample t-test results is somewhat arbitrary. Because of the nature of ICA analyses, these tests are actually post-hoc tests which will tend to produce strongly significant results. This is fine because these results are mainly used to help identify/represent the network associated with each component, so one is relatively free to choose the threshold values that results in the "cleaner" interpretation separately for each component (there is no confirmatory hypothesis testing here regarding these spatial maps, you just to want to characterize which aspect of the connectivity this component is capturing/representing, so one often uses relatively high/conservative thresholds in order to emphasize each specific network).
Last, regarding your question about between-group comparisons of spatial components, this is related to the above question. The average spatial map within a group does not just represents the network associated with this component but rather the connectivity pattern with this network (i.e. the connectivity between this network and the rest of the brain). Of course, when thresholding these single-group maps using very conservative thresholds you just get the network itself (in the same way that if you look at seed-to-voxel connectivity and use very conservative thresholds you will just get the seed itself). When comparing the spatial map across two groups you are effectively comparing the connectivity with this network across the two groups, so it is perfectly fine to find "out-of-network" areas that show differences in connectivity between the groups (that simply indicates that the connectivity between this network and those areas differs between your two groups). If you are only interested in within-network connectivity, then yes, you may constrain the between-group comparison to only include "within-network" areas, but otherwise (when interested in both within- and between- network connectivity) it is fine to perform these between-group comparisons across the entire brain.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
The current ICA implementation in CONN does not include any model-selection approach to help determine the "optimal" number of components in ICA (we will likely end up offering some form of cross-validation approach to help determine the relative significance of different components, but that is still in the works). If you need to compute any of the AIC or MDL model-selection measures, you can probably do that from the group-level spatial maps stored in ICA.ROIs.nii and the associated timeseries stored in ICA.Timeseries.mat (but this is not really straightforward to do).
Regarding your question about reconstructing individual-subject ICA maps (back-projection), those are already computed by CONN, they are stored in the files named BETA_Subject*_Condition*_Measure*.nii, and these are the volumes that are entered into your second-level analyses when looking at the ICA.SpatialComponents tab in the second-level results window. If, on the other hand, you mean that you would like to reconstruct individual-subject ICA maps for other subjects (other than those included in your original ICA analysis), then the way to do this would be to go to Setup.ROIs and click on the 'ROI tools. Add network-ICA ROIs' button. This will add a new weighted-ROI that represents the group-level spatial maps computed in the group-ICA step. Using these network-ROIs as seeds in a new first-level analysis that uses multivariate-regression measures will produce the same individual-subject ICA maps as those computed in the back-projection step, and this can be applied to the same or other subjects (not necessarily those included in the original ICA analyses).
Regarding your question about Z-value thresholds, in CONN this approach can be performed simply by going to the ICA.SpatialComponents tab in the second-level results window, and defining a new one-sample t-test second-level analysis (i.e. simply select the 'AllSubjects' effect, contrast 1). The choice of threshold in those one-sample t-test results is somewhat arbitrary. Because of the nature of ICA analyses, these tests are actually post-hoc tests which will tend to produce strongly significant results. This is fine because these results are mainly used to help identify/represent the network associated with each component, so one is relatively free to choose the threshold values that results in the "cleaner" interpretation separately for each component (there is no confirmatory hypothesis testing here regarding these spatial maps, you just to want to characterize which aspect of the connectivity this component is capturing/representing, so one often uses relatively high/conservative thresholds in order to emphasize each specific network).
Last, regarding your question about between-group comparisons of spatial components, this is related to the above question. The average spatial map within a group does not just represents the network associated with this component but rather the connectivity pattern with this network (i.e. the connectivity between this network and the rest of the brain). Of course, when thresholding these single-group maps using very conservative thresholds you just get the network itself (in the same way that if you look at seed-to-voxel connectivity and use very conservative thresholds you will just get the seed itself). When comparing the spatial map across two groups you are effectively comparing the connectivity with this network across the two groups, so it is perfectly fine to find "out-of-network" areas that show differences in connectivity between the groups (that simply indicates that the connectivity between this network and those areas differs between your two groups). If you are only interested in within-network connectivity, then yes, you may constrain the between-group comparison to only include "within-network" areas, but otherwise (when interested in both within- and between- network connectivity) it is fine to perform these between-group comparisons across the entire brain.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Dear fellow CONN-artists,
I've recently begun diving in ICA based methods for extracting functional networks in a modestly large data set (110 subjects, 3 time points). One major question when running ICA is the choice of parameters for data reduction and # of components to be estimated.
The Calhoun paper presents an AIC and MDL approach for performing these estimates. Is this algorithm implemented in CONN? If not, how would one go about extracting the parameters to compute the AIC and MDL? The major parameters of interest would be the log of the maximum likelihood estimate of the model parameters as estimated from the fMRI data, ML (the number of time points following individual subject data reduction) and the number of sources N.
Also, is there an easy way to reconstruct individual subject ICA maps? How would one go about reconstructing individual subject time courses for time course ICA decomposition?
Additionally, how should one pick a Z value threshold? Calhoun suggests calculating the mean and variance of each component across the subjects and then perform random effects inference for a one-sample t-test. Using a standard 0.001 threshold often results in over-saturated maps for some components, whereas other components might tolerate only low thresholds. How does one go about picking a consistent threshold across components?
And lastly, what exactly happens when you contrast an ICA component at the group level? I've performed a contrast between two subject groups (drug users vs non users, [1 -1]) with a single ICA component selected. However the difference between those two subject groups is a cluster that falls outside of the spatial extent of that ICA component (after thresholding). Shouldn't the difference between these two subjects be constrained to only the spatial extent of the chosen ICA component?
I've recently begun diving in ICA based methods for extracting functional networks in a modestly large data set (110 subjects, 3 time points). One major question when running ICA is the choice of parameters for data reduction and # of components to be estimated.
The Calhoun paper presents an AIC and MDL approach for performing these estimates. Is this algorithm implemented in CONN? If not, how would one go about extracting the parameters to compute the AIC and MDL? The major parameters of interest would be the log of the maximum likelihood estimate of the model parameters as estimated from the fMRI data, ML (the number of time points following individual subject data reduction) and the number of sources N.
Also, is there an easy way to reconstruct individual subject ICA maps? How would one go about reconstructing individual subject time courses for time course ICA decomposition?
Additionally, how should one pick a Z value threshold? Calhoun suggests calculating the mean and variance of each component across the subjects and then perform random effects inference for a one-sample t-test. Using a standard 0.001 threshold often results in over-saturated maps for some components, whereas other components might tolerate only low thresholds. How does one go about picking a consistent threshold across components?
And lastly, what exactly happens when you contrast an ICA component at the group level? I've performed a contrast between two subject groups (drug users vs non users, [1 -1]) with a single ICA component selected. However the difference between those two subject groups is a cluster that falls outside of the spatial extent of that ICA component (after thresholding). Shouldn't the difference between these two subjects be constrained to only the spatial extent of the chosen ICA component?
Oct 12, 2016 09:10 AM | Julia Binnewies
RE: ICA: Parameter Choice and 2nd Level Effects ?
Dear Alfonso,
In this post you mentioned that it is possible to constrain the between group analysis to only look into within-network effects.
I was wondering how I can do that? I guess I can use the masks generated with ICA as ROIs (using 'Add ICA-network-ROIs' in Setup.ROIs?). But what is then the best way to analyse the group differences? And is this the correct way to create the ROIs?
Thank you!
Best wishes,
Julia
Originally posted by Alfonso Nieto-Castanon:
In this post you mentioned that it is possible to constrain the between group analysis to only look into within-network effects.
I was wondering how I can do that? I guess I can use the masks generated with ICA as ROIs (using 'Add ICA-network-ROIs' in Setup.ROIs?). But what is then the best way to analyse the group differences? And is this the correct way to create the ROIs?
Thank you!
Best wishes,
Julia
Originally posted by Alfonso Nieto-Castanon:
Dear
Shady
The current ICA implementation in CONN does not include any model-selection approach to help determine the "optimal" number of components in ICA (we will likely end up offering some form of cross-validation approach to help determine the relative significance of different components, but that is still in the works). If you need to compute any of the AIC or MDL model-selection measures, you can probably do that from the group-level spatial maps stored in ICA.ROIs.nii and the associated timeseries stored in ICA.Timeseries.mat (but this is not really straightforward to do).
Regarding your question about reconstructing individual-subject ICA maps (back-projection), those are already computed by CONN, they are stored in the files named BETA_Subject*_Condition*_Measure*.nii, and these are the volumes that are entered into your second-level analyses when looking at the ICA.SpatialComponents tab in the second-level results window. If, on the other hand, you mean that you would like to reconstruct individual-subject ICA maps for other subjects (other than those included in your original ICA analysis), then the way to do this would be to go to Setup.ROIs and click on the 'ROI tools. Add network-ICA ROIs' button. This will add a new weighted-ROI that represents the group-level spatial maps computed in the group-ICA step. Using these network-ROIs as seeds in a new first-level analysis that uses multivariate-regression measures will produce the same individual-subject ICA maps as those computed in the back-projection step, and this can be applied to the same or other subjects (not necessarily those included in the original ICA analyses).
Regarding your question about Z-value thresholds, in CONN this approach can be performed simply by going to the ICA.SpatialComponents tab in the second-level results window, and defining a new one-sample t-test second-level analysis (i.e. simply select the 'AllSubjects' effect, contrast 1). The choice of threshold in those one-sample t-test results is somewhat arbitrary. Because of the nature of ICA analyses, these tests are actually post-hoc tests which will tend to produce strongly significant results. This is fine because these results are mainly used to help identify/represent the network associated with each component, so one is relatively free to choose the threshold values that results in the "cleaner" interpretation separately for each component (there is no confirmatory hypothesis testing here regarding these spatial maps, you just to want to characterize which aspect of the connectivity this component is capturing/representing, so one often uses relatively high/conservative thresholds in order to emphasize each specific network).
Last, regarding your question about between-group comparisons of spatial components, this is related to the above question. The average spatial map within a group does not just represents the network associated with this component but rather the connectivity pattern with this network (i.e. the connectivity between this network and the rest of the brain). Of course, when thresholding these single-group maps using very conservative thresholds you just get the network itself (in the same way that if you look at seed-to-voxel connectivity and use very conservative thresholds you will just get the seed itself). When comparing the spatial map across two groups you are effectively comparing the connectivity with this network across the two groups, so it is perfectly fine to find "out-of-network" areas that show differences in connectivity between the groups (that simply indicates that the connectivity between this network and those areas differs between your two groups). If you are only interested in within-network connectivity, then yes, you may constrain the between-group comparison to only include "within-network" areas, but otherwise (when interested in both within- and between- network connectivity) it is fine to perform these between-group comparisons across the entire brain.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
The current ICA implementation in CONN does not include any model-selection approach to help determine the "optimal" number of components in ICA (we will likely end up offering some form of cross-validation approach to help determine the relative significance of different components, but that is still in the works). If you need to compute any of the AIC or MDL model-selection measures, you can probably do that from the group-level spatial maps stored in ICA.ROIs.nii and the associated timeseries stored in ICA.Timeseries.mat (but this is not really straightforward to do).
Regarding your question about reconstructing individual-subject ICA maps (back-projection), those are already computed by CONN, they are stored in the files named BETA_Subject*_Condition*_Measure*.nii, and these are the volumes that are entered into your second-level analyses when looking at the ICA.SpatialComponents tab in the second-level results window. If, on the other hand, you mean that you would like to reconstruct individual-subject ICA maps for other subjects (other than those included in your original ICA analysis), then the way to do this would be to go to Setup.ROIs and click on the 'ROI tools. Add network-ICA ROIs' button. This will add a new weighted-ROI that represents the group-level spatial maps computed in the group-ICA step. Using these network-ROIs as seeds in a new first-level analysis that uses multivariate-regression measures will produce the same individual-subject ICA maps as those computed in the back-projection step, and this can be applied to the same or other subjects (not necessarily those included in the original ICA analyses).
Regarding your question about Z-value thresholds, in CONN this approach can be performed simply by going to the ICA.SpatialComponents tab in the second-level results window, and defining a new one-sample t-test second-level analysis (i.e. simply select the 'AllSubjects' effect, contrast 1). The choice of threshold in those one-sample t-test results is somewhat arbitrary. Because of the nature of ICA analyses, these tests are actually post-hoc tests which will tend to produce strongly significant results. This is fine because these results are mainly used to help identify/represent the network associated with each component, so one is relatively free to choose the threshold values that results in the "cleaner" interpretation separately for each component (there is no confirmatory hypothesis testing here regarding these spatial maps, you just to want to characterize which aspect of the connectivity this component is capturing/representing, so one often uses relatively high/conservative thresholds in order to emphasize each specific network).
Last, regarding your question about between-group comparisons of spatial components, this is related to the above question. The average spatial map within a group does not just represents the network associated with this component but rather the connectivity pattern with this network (i.e. the connectivity between this network and the rest of the brain). Of course, when thresholding these single-group maps using very conservative thresholds you just get the network itself (in the same way that if you look at seed-to-voxel connectivity and use very conservative thresholds you will just get the seed itself). When comparing the spatial map across two groups you are effectively comparing the connectivity with this network across the two groups, so it is perfectly fine to find "out-of-network" areas that show differences in connectivity between the groups (that simply indicates that the connectivity between this network and those areas differs between your two groups). If you are only interested in within-network connectivity, then yes, you may constrain the between-group comparison to only include "within-network" areas, but otherwise (when interested in both within- and between- network connectivity) it is fine to perform these between-group comparisons across the entire brain.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Dear fellow CONN-artists,
I've recently begun diving in ICA based methods for extracting functional networks in a modestly large data set (110 subjects, 3 time points). One major question when running ICA is the choice of parameters for data reduction and # of components to be estimated.
The Calhoun paper presents an AIC and MDL approach for performing these estimates. Is this algorithm implemented in CONN? If not, how would one go about extracting the parameters to compute the AIC and MDL? The major parameters of interest would be the log of the maximum likelihood estimate of the model parameters as estimated from the fMRI data, ML (the number of time points following individual subject data reduction) and the number of sources N.
Also, is there an easy way to reconstruct individual subject ICA maps? How would one go about reconstructing individual subject time courses for time course ICA decomposition?
Additionally, how should one pick a Z value threshold? Calhoun suggests calculating the mean and variance of each component across the subjects and then perform random effects inference for a one-sample t-test. Using a standard 0.001 threshold often results in over-saturated maps for some components, whereas other components might tolerate only low thresholds. How does one go about picking a consistent threshold across components?
And lastly, what exactly happens when you contrast an ICA component at the group level? I've performed a contrast between two subject groups (drug users vs non users, [1 -1]) with a single ICA component selected. However the difference between those two subject groups is a cluster that falls outside of the spatial extent of that ICA component (after thresholding). Shouldn't the difference between these two subjects be constrained to only the spatial extent of the chosen ICA component?
I've recently begun diving in ICA based methods for extracting functional networks in a modestly large data set (110 subjects, 3 time points). One major question when running ICA is the choice of parameters for data reduction and # of components to be estimated.
The Calhoun paper presents an AIC and MDL approach for performing these estimates. Is this algorithm implemented in CONN? If not, how would one go about extracting the parameters to compute the AIC and MDL? The major parameters of interest would be the log of the maximum likelihood estimate of the model parameters as estimated from the fMRI data, ML (the number of time points following individual subject data reduction) and the number of sources N.
Also, is there an easy way to reconstruct individual subject ICA maps? How would one go about reconstructing individual subject time courses for time course ICA decomposition?
Additionally, how should one pick a Z value threshold? Calhoun suggests calculating the mean and variance of each component across the subjects and then perform random effects inference for a one-sample t-test. Using a standard 0.001 threshold often results in over-saturated maps for some components, whereas other components might tolerate only low thresholds. How does one go about picking a consistent threshold across components?
And lastly, what exactly happens when you contrast an ICA component at the group level? I've performed a contrast between two subject groups (drug users vs non users, [1 -1]) with a single ICA component selected. However the difference between those two subject groups is a cluster that falls outside of the spatial extent of that ICA component (after thresholding). Shouldn't the difference between these two subjects be constrained to only the spatial extent of the chosen ICA component?
Oct 12, 2016 07:10 PM | Shady El Damaty - Georgetown University
RE: ICA: Parameter Choice and 2nd Level Effects ?
Wow! great response Alfonso --
There are all sorts of issues I ran into trying to implement this on my own (memory issues nonwithstanding) so I have been using existing implementations. I've managed to estimate MDL for each individual subject's denoised data using the icatb_estimate_dimension.m function in the GIFT toolbox. However I'm not too confident in the results since the MDL estimate hasn't been performed using the group level data but rather individual subjects (which all turned out to be 56...). I've attached a plot of the MDL/AIC estimate with shaded error bars across subjects. Does this look right to you?
Thank you so much for your continued support!!
There are all sorts of issues I ran into trying to implement this on my own (memory issues nonwithstanding) so I have been using existing implementations. I've managed to estimate MDL for each individual subject's denoised data using the icatb_estimate_dimension.m function in the GIFT toolbox. However I'm not too confident in the results since the MDL estimate hasn't been performed using the group level data but rather individual subjects (which all turned out to be 56...). I've attached a plot of the MDL/AIC estimate with shaded error bars across subjects. Does this look right to you?
Thank you so much for your continued support!!
Oct 14, 2016 04:10 PM | Alfonso Nieto-Castanon - Boston University
RE: ICA: Parameter Choice and 2nd Level Effects ?
Hi Shady,
Those MDL estimates based on single-subject data are likely not terribly meaningful, as the plots seem to suggest. They are probably all turning out to be 56 because that may be the degrees of freedom of your single-subject data after filtering (i.e. approximately the number of frequency samples within your band-pass window = NumberOfScans * FilterWidth / NyquistFrequency). This only partially relates to the "optimal" number of components for the group-level dimensionality reduction or ICA steps, as the number of subjects in your study plays a very important role in limiting the number of components that you can reliably estimate.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Those MDL estimates based on single-subject data are likely not terribly meaningful, as the plots seem to suggest. They are probably all turning out to be 56 because that may be the degrees of freedom of your single-subject data after filtering (i.e. approximately the number of frequency samples within your band-pass window = NumberOfScans * FilterWidth / NyquistFrequency). This only partially relates to the "optimal" number of components for the group-level dimensionality reduction or ICA steps, as the number of subjects in your study plays a very important role in limiting the number of components that you can reliably estimate.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Wow! great response Alfonso --
There are all sorts of issues I ran into trying to implement this on my own (memory issues nonwithstanding) so I have been using existing implementations. I've managed to estimate MDL for each individual subject's denoised data using the icatb_estimate_dimension.m function in the GIFT toolbox. However I'm not too confident in the results since the MDL estimate hasn't been performed using the group level data but rather individual subjects (which all turned out to be 56...). I've attached a plot of the MDL/AIC estimate with shaded error bars across subjects. Does this look right to you?
Thank you so much for your continued support!!
There are all sorts of issues I ran into trying to implement this on my own (memory issues nonwithstanding) so I have been using existing implementations. I've managed to estimate MDL for each individual subject's denoised data using the icatb_estimate_dimension.m function in the GIFT toolbox. However I'm not too confident in the results since the MDL estimate hasn't been performed using the group level data but rather individual subjects (which all turned out to be 56...). I've attached a plot of the MDL/AIC estimate with shaded error bars across subjects. Does this look right to you?
Thank you so much for your continued support!!
Oct 16, 2016 02:10 AM | Shady El Damaty - Georgetown University
RE: ICA: Parameter Choice and 2nd Level Effects ?
Hi Alfonso,
Ok that makes a bit of sense -- I tried calculating the degrees of freedom using the equation you listed as follows:
TR = 2.28
NumScans = 150
Filter Width = 0.09-0.008 = 0.082
NumScans * FilterWidth/NyqFreq = 150*0.0820/((1/2.28)*.5 = 56.1
So how do I incorporate the number of subjects (N=133) to calculate the final number of components?
PS -- are there any resources I can consult to understand where you got that equation from? this is all very interesting!!
Originally posted by Alfonso Nieto-Castanon:
Ok that makes a bit of sense -- I tried calculating the degrees of freedom using the equation you listed as follows:
TR = 2.28
NumScans = 150
Filter Width = 0.09-0.008 = 0.082
NumScans * FilterWidth/NyqFreq = 150*0.0820/((1/2.28)*.5 = 56.1
So how do I incorporate the number of subjects (N=133) to calculate the final number of components?
PS -- are there any resources I can consult to understand where you got that equation from? this is all very interesting!!
Originally posted by Alfonso Nieto-Castanon:
Hi
Shady,
Those MDL estimates based on single-subject data are likely not terribly meaningful, as the plots seem to suggest. They are probably all turning out to be 56 because that may be the degrees of freedom of your single-subject data after filtering (i.e. approximately the number of frequency samples within your band-pass window = NumberOfScans * FilterWidth / NyquistFrequency). This only partially relates to the "optimal" number of components for the group-level dimensionality reduction or ICA steps, as the number of subjects in your study plays a very important role in limiting the number of components that you can reliably estimate.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Those MDL estimates based on single-subject data are likely not terribly meaningful, as the plots seem to suggest. They are probably all turning out to be 56 because that may be the degrees of freedom of your single-subject data after filtering (i.e. approximately the number of frequency samples within your band-pass window = NumberOfScans * FilterWidth / NyquistFrequency). This only partially relates to the "optimal" number of components for the group-level dimensionality reduction or ICA steps, as the number of subjects in your study plays a very important role in limiting the number of components that you can reliably estimate.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Wow! great response Alfonso --
There are all sorts of issues I ran into trying to implement this on my own (memory issues nonwithstanding) so I have been using existing implementations. I've managed to estimate MDL for each individual subject's denoised data using the icatb_estimate_dimension.m function in the GIFT toolbox. However I'm not too confident in the results since the MDL estimate hasn't been performed using the group level data but rather individual subjects (which all turned out to be 56...). I've attached a plot of the MDL/AIC estimate with shaded error bars across subjects. Does this look right to you?
Thank you so much for your continued support!!
There are all sorts of issues I ran into trying to implement this on my own (memory issues nonwithstanding) so I have been using existing implementations. I've managed to estimate MDL for each individual subject's denoised data using the icatb_estimate_dimension.m function in the GIFT toolbox. However I'm not too confident in the results since the MDL estimate hasn't been performed using the group level data but rather individual subjects (which all turned out to be 56...). I've attached a plot of the MDL/AIC estimate with shaded error bars across subjects. Does this look right to you?
Thank you so much for your continued support!!
Oct 17, 2016 05:10 PM | Alfonso Nieto-Castanon - Boston University
RE: ICA: Parameter Choice and 2nd Level Effects ?
Hi Shady,
Unfortunately there is no such formula for the multiple-subject case. To be a bit more precise, the actual degrees of freedom of the multiple-subject data will just be in this case min(56.1*Nsubjects,Nvoxels) (because the components across subjects cannot possibly be perfectly aligned) but what this really tells you is only that the multiple-subject data is not going to show the very sharp drop that you found in the single-subject data (or at least it is not going to show it until ~7.500 components), but unfortunately this tells you nothing about how many of these components are really replicable or significant. For that you need something like Bartlett's test, model comparison measures, etc. which gets us back to our starting point. In practical terms, one easy option to identify significant/reliable components when you have sufficient subjects (as you do), is to repeat the ICA analyses across multiple smaller subsets of subjects, and then simply select those components that appear consistently across many or most of those smaller analyses.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Unfortunately there is no such formula for the multiple-subject case. To be a bit more precise, the actual degrees of freedom of the multiple-subject data will just be in this case min(56.1*Nsubjects,Nvoxels) (because the components across subjects cannot possibly be perfectly aligned) but what this really tells you is only that the multiple-subject data is not going to show the very sharp drop that you found in the single-subject data (or at least it is not going to show it until ~7.500 components), but unfortunately this tells you nothing about how many of these components are really replicable or significant. For that you need something like Bartlett's test, model comparison measures, etc. which gets us back to our starting point. In practical terms, one easy option to identify significant/reliable components when you have sufficient subjects (as you do), is to repeat the ICA analyses across multiple smaller subsets of subjects, and then simply select those components that appear consistently across many or most of those smaller analyses.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Hi Alfonso,
Ok that makes a bit of sense -- I tried calculating the degrees of freedom using the equation you listed as follows:
TR = 2.28
NumScans = 150
Filter Width = 0.09-0.008 = 0.082
NumScans * FilterWidth/NyqFreq = 150*0.0820/((1/2.28)*.5 = 56.1
So how do I incorporate the number of subjects (N=133) to calculate the final number of components?
PS -- are there any resources I can consult to understand where you got that equation from? this is all very interesting!!
Originally posted by Alfonso Nieto-Castanon:
Ok that makes a bit of sense -- I tried calculating the degrees of freedom using the equation you listed as follows:
TR = 2.28
NumScans = 150
Filter Width = 0.09-0.008 = 0.082
NumScans * FilterWidth/NyqFreq = 150*0.0820/((1/2.28)*.5 = 56.1
So how do I incorporate the number of subjects (N=133) to calculate the final number of components?
PS -- are there any resources I can consult to understand where you got that equation from? this is all very interesting!!
Originally posted by Alfonso Nieto-Castanon:
Hi
Shady,
Those MDL estimates based on single-subject data are likely not terribly meaningful, as the plots seem to suggest. They are probably all turning out to be 56 because that may be the degrees of freedom of your single-subject data after filtering (i.e. approximately the number of frequency samples within your band-pass window = NumberOfScans * FilterWidth / NyquistFrequency). This only partially relates to the "optimal" number of components for the group-level dimensionality reduction or ICA steps, as the number of subjects in your study plays a very important role in limiting the number of components that you can reliably estimate.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Those MDL estimates based on single-subject data are likely not terribly meaningful, as the plots seem to suggest. They are probably all turning out to be 56 because that may be the degrees of freedom of your single-subject data after filtering (i.e. approximately the number of frequency samples within your band-pass window = NumberOfScans * FilterWidth / NyquistFrequency). This only partially relates to the "optimal" number of components for the group-level dimensionality reduction or ICA steps, as the number of subjects in your study plays a very important role in limiting the number of components that you can reliably estimate.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Wow! great response Alfonso --
There are all sorts of issues I ran into trying to implement this on my own (memory issues nonwithstanding) so I have been using existing implementations. I've managed to estimate MDL for each individual subject's denoised data using the icatb_estimate_dimension.m function in the GIFT toolbox. However I'm not too confident in the results since the MDL estimate hasn't been performed using the group level data but rather individual subjects (which all turned out to be 56...). I've attached a plot of the MDL/AIC estimate with shaded error bars across subjects. Does this look right to you?
Thank you so much for your continued support!!
There are all sorts of issues I ran into trying to implement this on my own (memory issues nonwithstanding) so I have been using existing implementations. I've managed to estimate MDL for each individual subject's denoised data using the icatb_estimate_dimension.m function in the GIFT toolbox. However I'm not too confident in the results since the MDL estimate hasn't been performed using the group level data but rather individual subjects (which all turned out to be 56...). I've attached a plot of the MDL/AIC estimate with shaded error bars across subjects. Does this look right to you?
Thank you so much for your continued support!!
Oct 17, 2016 07:10 PM | Shady El Damaty - Georgetown University
RE: ICA: Parameter Choice and 2nd Level Effects ?
Hi Alfonso,
I'm not quite sure what you the notation min(56.1*Nsubjects,Nvoxels) means or how you got 7.5 components. Is this the minimum of 56.1*133 as the number of voxels increase? The square root of 56 is ~7.5.. also (56.1*133)/1000 = ~7.5.
Thanks for the hint about validation by partitioning the dataset into smaller subject pools. I'll probably end up trying that to avoid developing a whole new toolbox for fmri ica model validation :P
Originally posted by Alfonso Nieto-Castanon:
I'm not quite sure what you the notation min(56.1*Nsubjects,Nvoxels) means or how you got 7.5 components. Is this the minimum of 56.1*133 as the number of voxels increase? The square root of 56 is ~7.5.. also (56.1*133)/1000 = ~7.5.
Thanks for the hint about validation by partitioning the dataset into smaller subject pools. I'll probably end up trying that to avoid developing a whole new toolbox for fmri ica model validation :P
Originally posted by Alfonso Nieto-Castanon:
Hi
Shady,
Unfortunately there is no such formula for the multiple-subject case. To be a bit more precise, the actual degrees of freedom of the multiple-subject data will just be in this case min(56.1*Nsubjects,Nvoxels) (because the components across subjects cannot possibly be perfectly aligned) but what this really tells you is only that the multiple-subject data is not going to show the very sharp drop that you found in the single-subject data (or at least it is not going to show it until ~7.500 components), but unfortunately this tells you nothing about how many of these components are really replicable or significant. For that you need something like Bartlett's test, model comparison measures, etc. which gets us back to our starting point. In practical terms, one easy option to identify significant/reliable components when you have sufficient subjects (as you do), is to repeat the ICA analyses across multiple smaller subsets of subjects, and then simply select those components that appear consistently across many or most of those smaller analyses.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Unfortunately there is no such formula for the multiple-subject case. To be a bit more precise, the actual degrees of freedom of the multiple-subject data will just be in this case min(56.1*Nsubjects,Nvoxels) (because the components across subjects cannot possibly be perfectly aligned) but what this really tells you is only that the multiple-subject data is not going to show the very sharp drop that you found in the single-subject data (or at least it is not going to show it until ~7.500 components), but unfortunately this tells you nothing about how many of these components are really replicable or significant. For that you need something like Bartlett's test, model comparison measures, etc. which gets us back to our starting point. In practical terms, one easy option to identify significant/reliable components when you have sufficient subjects (as you do), is to repeat the ICA analyses across multiple smaller subsets of subjects, and then simply select those components that appear consistently across many or most of those smaller analyses.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Hi Alfonso,
Ok that makes a bit of sense -- I tried calculating the degrees of freedom using the equation you listed as follows:
TR = 2.28
NumScans = 150
Filter Width = 0.09-0.008 = 0.082
NumScans * FilterWidth/NyqFreq = 150*0.0820/((1/2.28)*.5 = 56.1
So how do I incorporate the number of subjects (N=133) to calculate the final number of components?
PS -- are there any resources I can consult to understand where you got that equation from? this is all very interesting!!
Originally posted by Alfonso Nieto-Castanon:
Ok that makes a bit of sense -- I tried calculating the degrees of freedom using the equation you listed as follows:
TR = 2.28
NumScans = 150
Filter Width = 0.09-0.008 = 0.082
NumScans * FilterWidth/NyqFreq = 150*0.0820/((1/2.28)*.5 = 56.1
So how do I incorporate the number of subjects (N=133) to calculate the final number of components?
PS -- are there any resources I can consult to understand where you got that equation from? this is all very interesting!!
Originally posted by Alfonso Nieto-Castanon:
Hi
Shady,
Those MDL estimates based on single-subject data are likely not terribly meaningful, as the plots seem to suggest. They are probably all turning out to be 56 because that may be the degrees of freedom of your single-subject data after filtering (i.e. approximately the number of frequency samples within your band-pass window = NumberOfScans * FilterWidth / NyquistFrequency). This only partially relates to the "optimal" number of components for the group-level dimensionality reduction or ICA steps, as the number of subjects in your study plays a very important role in limiting the number of components that you can reliably estimate.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Those MDL estimates based on single-subject data are likely not terribly meaningful, as the plots seem to suggest. They are probably all turning out to be 56 because that may be the degrees of freedom of your single-subject data after filtering (i.e. approximately the number of frequency samples within your band-pass window = NumberOfScans * FilterWidth / NyquistFrequency). This only partially relates to the "optimal" number of components for the group-level dimensionality reduction or ICA steps, as the number of subjects in your study plays a very important role in limiting the number of components that you can reliably estimate.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Wow! great response Alfonso --
There are all sorts of issues I ran into trying to implement this on my own (memory issues nonwithstanding) so I have been using existing implementations. I've managed to estimate MDL for each individual subject's denoised data using the icatb_estimate_dimension.m function in the GIFT toolbox. However I'm not too confident in the results since the MDL estimate hasn't been performed using the group level data but rather individual subjects (which all turned out to be 56...). I've attached a plot of the MDL/AIC estimate with shaded error bars across subjects. Does this look right to you?
Thank you so much for your continued support!!
There are all sorts of issues I ran into trying to implement this on my own (memory issues nonwithstanding) so I have been using existing implementations. I've managed to estimate MDL for each individual subject's denoised data using the icatb_estimate_dimension.m function in the GIFT toolbox. However I'm not too confident in the results since the MDL estimate hasn't been performed using the group level data but rather individual subjects (which all turned out to be 56...). I've attached a plot of the MDL/AIC estimate with shaded error bars across subjects. Does this look right to you?
Thank you so much for your continued support!!
Oct 18, 2016 07:10 AM | Julia Binnewies
RE: ICA: Parameter Choice and 2nd Level Effects ?
I am still not sure how to constrain
the between group analysis to only within-networks effects.
Does anybody have any ideas?
Best wishes,
Julia
Originally posted by Julia Binnewies:
Does anybody have any ideas?
Best wishes,
Julia
Originally posted by Julia Binnewies:
Dear
Alfonso,
In this post you mentioned that it is possible to constrain the between group analysis to only look into within-network effects.
I was wondering how I can do that? I guess I can use the masks generated with ICA as ROIs (using 'Add ICA-network-ROIs' in Setup.ROIs?). But what is then the best way to analyze the group differences? And is this the correct way to create the ROIs?
Thank you!
Best wishes,
Julia
Originally posted by Alfonso Nieto-Castanon:
In this post you mentioned that it is possible to constrain the between group analysis to only look into within-network effects.
I was wondering how I can do that? I guess I can use the masks generated with ICA as ROIs (using 'Add ICA-network-ROIs' in Setup.ROIs?). But what is then the best way to analyze the group differences? And is this the correct way to create the ROIs?
Thank you!
Best wishes,
Julia
Originally posted by Alfonso Nieto-Castanon:
Dear
Shady
The current ICA implementation in CONN does not include any model-selection approach to help determine the "optimal" number of components in ICA (we will likely end up offering some form of cross-validation approach to help determine the relative significance of different components, but that is still in the works). If you need to compute any of the AIC or MDL model-selection measures, you can probably do that from the group-level spatial maps stored in ICA.ROIs.nii and the associated timeseries stored in ICA.Timeseries.mat (but this is not really straightforward to do).
Regarding your question about reconstructing individual-subject ICA maps (back-projection), those are already computed by CONN, they are stored in the files named BETA_Subject*_Condition*_Measure*.nii, and these are the volumes that are entered into your second-level analyses when looking at the ICA.SpatialComponents tab in the second-level results window. If, on the other hand, you mean that you would like to reconstruct individual-subject ICA maps for other subjects (other than those included in your original ICA analysis), then the way to do this would be to go to Setup.ROIs and click on the 'ROI tools. Add network-ICA ROIs' button. This will add a new weighted-ROI that represents the group-level spatial maps computed in the group-ICA step. Using these network-ROIs as seeds in a new first-level analysis that uses multivariate-regression measures will produce the same individual-subject ICA maps as those computed in the back-projection step, and this can be applied to the same or other subjects (not necessarily those included in the original ICA analyses).
Regarding your question about Z-value thresholds, in CONN this approach can be performed simply by going to the ICA.SpatialComponents tab in the second-level results window, and defining a new one-sample t-test second-level analysis (i.e. simply select the 'AllSubjects' effect, contrast 1). The choice of threshold in those one-sample t-test results is somewhat arbitrary. Because of the nature of ICA analyses, these tests are actually post-hoc tests which will tend to produce strongly significant results. This is fine because these results are mainly used to help identify/represent the network associated with each component, so one is relatively free to choose the threshold values that results in the "cleaner" interpretation separately for each component (there is no confirmatory hypothesis testing here regarding these spatial maps, you just to want to characterize which aspect of the connectivity this component is capturing/representing, so one often uses relatively high/conservative thresholds in order to emphasize each specific network).
Last, regarding your question about between-group comparisons of spatial components, this is related to the above question. The average spatial map within a group does not just represents the network associated with this component but rather the connectivity pattern with this network (i.e. the connectivity between this network and the rest of the brain). Of course, when thresholding these single-group maps using very conservative thresholds you just get the network itself (in the same way that if you look at seed-to-voxel connectivity and use very conservative thresholds you will just get the seed itself). When comparing the spatial map across two groups you are effectively comparing the connectivity with this network across the two groups, so it is perfectly fine to find "out-of-network" areas that show differences in connectivity between the groups (that simply indicates that the connectivity between this network and those areas differs between your two groups). If you are only interested in within-network connectivity, then yes, you may constrain the between-group comparison to only include "within-network" areas, but otherwise (when interested in both within- and between- network connectivity) it is fine to perform these between-group comparisons across the entire brain.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
The current ICA implementation in CONN does not include any model-selection approach to help determine the "optimal" number of components in ICA (we will likely end up offering some form of cross-validation approach to help determine the relative significance of different components, but that is still in the works). If you need to compute any of the AIC or MDL model-selection measures, you can probably do that from the group-level spatial maps stored in ICA.ROIs.nii and the associated timeseries stored in ICA.Timeseries.mat (but this is not really straightforward to do).
Regarding your question about reconstructing individual-subject ICA maps (back-projection), those are already computed by CONN, they are stored in the files named BETA_Subject*_Condition*_Measure*.nii, and these are the volumes that are entered into your second-level analyses when looking at the ICA.SpatialComponents tab in the second-level results window. If, on the other hand, you mean that you would like to reconstruct individual-subject ICA maps for other subjects (other than those included in your original ICA analysis), then the way to do this would be to go to Setup.ROIs and click on the 'ROI tools. Add network-ICA ROIs' button. This will add a new weighted-ROI that represents the group-level spatial maps computed in the group-ICA step. Using these network-ROIs as seeds in a new first-level analysis that uses multivariate-regression measures will produce the same individual-subject ICA maps as those computed in the back-projection step, and this can be applied to the same or other subjects (not necessarily those included in the original ICA analyses).
Regarding your question about Z-value thresholds, in CONN this approach can be performed simply by going to the ICA.SpatialComponents tab in the second-level results window, and defining a new one-sample t-test second-level analysis (i.e. simply select the 'AllSubjects' effect, contrast 1). The choice of threshold in those one-sample t-test results is somewhat arbitrary. Because of the nature of ICA analyses, these tests are actually post-hoc tests which will tend to produce strongly significant results. This is fine because these results are mainly used to help identify/represent the network associated with each component, so one is relatively free to choose the threshold values that results in the "cleaner" interpretation separately for each component (there is no confirmatory hypothesis testing here regarding these spatial maps, you just to want to characterize which aspect of the connectivity this component is capturing/representing, so one often uses relatively high/conservative thresholds in order to emphasize each specific network).
Last, regarding your question about between-group comparisons of spatial components, this is related to the above question. The average spatial map within a group does not just represents the network associated with this component but rather the connectivity pattern with this network (i.e. the connectivity between this network and the rest of the brain). Of course, when thresholding these single-group maps using very conservative thresholds you just get the network itself (in the same way that if you look at seed-to-voxel connectivity and use very conservative thresholds you will just get the seed itself). When comparing the spatial map across two groups you are effectively comparing the connectivity with this network across the two groups, so it is perfectly fine to find "out-of-network" areas that show differences in connectivity between the groups (that simply indicates that the connectivity between this network and those areas differs between your two groups). If you are only interested in within-network connectivity, then yes, you may constrain the between-group comparison to only include "within-network" areas, but otherwise (when interested in both within- and between- network connectivity) it is fine to perform these between-group comparisons across the entire brain.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Dear fellow CONN-artists,
I've recently begun diving in ICA based methods for extracting functional networks in a modestly large data set (110 subjects, 3 time points). One major question when running ICA is the choice of parameters for data reduction and # of components to be estimated.
The Calhoun paper presents an AIC and MDL approach for performing these estimates. Is this algorithm implemented in CONN? If not, how would one go about extracting the parameters to compute the AIC and MDL? The major parameters of interest would be the log of the maximum likelihood estimate of the model parameters as estimated from the fMRI data, ML (the number of time points following individual subject data reduction) and the number of sources N.
Also, is there an easy way to reconstruct individual subject ICA maps? How would one go about reconstructing individual subject time courses for time course ICA decomposition?
Additionally, how should one pick a Z value threshold? Calhoun suggests calculating the mean and variance of each component across the subjects and then perform random effects inference for a one-sample t-test. Using a standard 0.001 threshold often results in over-saturated maps for some components, whereas other components might tolerate only low thresholds. How does one go about picking a consistent threshold across components?
And lastly, what exactly happens when you contrast an ICA component at the group level? I've performed a contrast between two subject groups (drug users vs non users, [1 -1]) with a single ICA component selected. However the difference between those two subject groups is a cluster that falls outside of the spatial extent of that ICA component (after thresholding). Shouldn't the difference between these two subjects be constrained to only the spatial extent of the chosen ICA component?
I've recently begun diving in ICA based methods for extracting functional networks in a modestly large data set (110 subjects, 3 time points). One major question when running ICA is the choice of parameters for data reduction and # of components to be estimated.
The Calhoun paper presents an AIC and MDL approach for performing these estimates. Is this algorithm implemented in CONN? If not, how would one go about extracting the parameters to compute the AIC and MDL? The major parameters of interest would be the log of the maximum likelihood estimate of the model parameters as estimated from the fMRI data, ML (the number of time points following individual subject data reduction) and the number of sources N.
Also, is there an easy way to reconstruct individual subject ICA maps? How would one go about reconstructing individual subject time courses for time course ICA decomposition?
Additionally, how should one pick a Z value threshold? Calhoun suggests calculating the mean and variance of each component across the subjects and then perform random effects inference for a one-sample t-test. Using a standard 0.001 threshold often results in over-saturated maps for some components, whereas other components might tolerate only low thresholds. How does one go about picking a consistent threshold across components?
And lastly, what exactly happens when you contrast an ICA component at the group level? I've performed a contrast between two subject groups (drug users vs non users, [1 -1]) with a single ICA component selected. However the difference between those two subject groups is a cluster that falls outside of the spatial extent of that ICA component (after thresholding). Shouldn't the difference between these two subjects be constrained to only the spatial extent of the chosen ICA component?
Oct 18, 2016 02:10 PM | Shady El Damaty - Georgetown University
RE: ICA: Parameter Choice and 2nd Level Effects ?
Hi Julia,
I think the thing to do here is to first generate ROI maps at the second level by clicking the 'Spatial Summary' tab under 'ICA networks'. This is done by using the 'Results explorer' to constrain the ICA connectivity map to just the network you are interested in. You can then export the mask as an ROI and then load it into ROIs under the 'Setup' tab. Sometimes I edit these ROIs in freeview, making each independent cluster a different color (or value as conn sees it). You can then do an ROI-ROI analysis using these ROIs.
Originally posted by Julia Binnewies:
I think the thing to do here is to first generate ROI maps at the second level by clicking the 'Spatial Summary' tab under 'ICA networks'. This is done by using the 'Results explorer' to constrain the ICA connectivity map to just the network you are interested in. You can then export the mask as an ROI and then load it into ROIs under the 'Setup' tab. Sometimes I edit these ROIs in freeview, making each independent cluster a different color (or value as conn sees it). You can then do an ROI-ROI analysis using these ROIs.
Originally posted by Julia Binnewies:
I am still not sure
how to constrain the between group analysis to only within-networks
effects.
Does anybody have any ideas?
Best wishes,
Julia
Originally posted by Julia Binnewies:
Does anybody have any ideas?
Best wishes,
Julia
Originally posted by Julia Binnewies:
Dear
Alfonso,
In this post you mentioned that it is possible to constrain the between group analysis to only look into within-network effects.
I was wondering how I can do that? I guess I can use the masks generated with ICA as ROIs (using 'Add ICA-network-ROIs' in Setup.ROIs?). But what is then the best way to analyze the group differences? And is this the correct way to create the ROIs?
Thank you!
Best wishes,
Julia
Originally posted by Alfonso Nieto-Castanon:
In this post you mentioned that it is possible to constrain the between group analysis to only look into within-network effects.
I was wondering how I can do that? I guess I can use the masks generated with ICA as ROIs (using 'Add ICA-network-ROIs' in Setup.ROIs?). But what is then the best way to analyze the group differences? And is this the correct way to create the ROIs?
Thank you!
Best wishes,
Julia
Originally posted by Alfonso Nieto-Castanon:
Dear
Shady
The current ICA implementation in CONN does not include any model-selection approach to help determine the "optimal" number of components in ICA (we will likely end up offering some form of cross-validation approach to help determine the relative significance of different components, but that is still in the works). If you need to compute any of the AIC or MDL model-selection measures, you can probably do that from the group-level spatial maps stored in ICA.ROIs.nii and the associated timeseries stored in ICA.Timeseries.mat (but this is not really straightforward to do).
Regarding your question about reconstructing individual-subject ICA maps (back-projection), those are already computed by CONN, they are stored in the files named BETA_Subject*_Condition*_Measure*.nii, and these are the volumes that are entered into your second-level analyses when looking at the ICA.SpatialComponents tab in the second-level results window. If, on the other hand, you mean that you would like to reconstruct individual-subject ICA maps for other subjects (other than those included in your original ICA analysis), then the way to do this would be to go to Setup.ROIs and click on the 'ROI tools. Add network-ICA ROIs' button. This will add a new weighted-ROI that represents the group-level spatial maps computed in the group-ICA step. Using these network-ROIs as seeds in a new first-level analysis that uses multivariate-regression measures will produce the same individual-subject ICA maps as those computed in the back-projection step, and this can be applied to the same or other subjects (not necessarily those included in the original ICA analyses).
Regarding your question about Z-value thresholds, in CONN this approach can be performed simply by going to the ICA.SpatialComponents tab in the second-level results window, and defining a new one-sample t-test second-level analysis (i.e. simply select the 'AllSubjects' effect, contrast 1). The choice of threshold in those one-sample t-test results is somewhat arbitrary. Because of the nature of ICA analyses, these tests are actually post-hoc tests which will tend to produce strongly significant results. This is fine because these results are mainly used to help identify/represent the network associated with each component, so one is relatively free to choose the threshold values that results in the "cleaner" interpretation separately for each component (there is no confirmatory hypothesis testing here regarding these spatial maps, you just to want to characterize which aspect of the connectivity this component is capturing/representing, so one often uses relatively high/conservative thresholds in order to emphasize each specific network).
Last, regarding your question about between-group comparisons of spatial components, this is related to the above question. The average spatial map within a group does not just represents the network associated with this component but rather the connectivity pattern with this network (i.e. the connectivity between this network and the rest of the brain). Of course, when thresholding these single-group maps using very conservative thresholds you just get the network itself (in the same way that if you look at seed-to-voxel connectivity and use very conservative thresholds you will just get the seed itself). When comparing the spatial map across two groups you are effectively comparing the connectivity with this network across the two groups, so it is perfectly fine to find "out-of-network" areas that show differences in connectivity between the groups (that simply indicates that the connectivity between this network and those areas differs between your two groups). If you are only interested in within-network connectivity, then yes, you may constrain the between-group comparison to only include "within-network" areas, but otherwise (when interested in both within- and between- network connectivity) it is fine to perform these between-group comparisons across the entire brain.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
The current ICA implementation in CONN does not include any model-selection approach to help determine the "optimal" number of components in ICA (we will likely end up offering some form of cross-validation approach to help determine the relative significance of different components, but that is still in the works). If you need to compute any of the AIC or MDL model-selection measures, you can probably do that from the group-level spatial maps stored in ICA.ROIs.nii and the associated timeseries stored in ICA.Timeseries.mat (but this is not really straightforward to do).
Regarding your question about reconstructing individual-subject ICA maps (back-projection), those are already computed by CONN, they are stored in the files named BETA_Subject*_Condition*_Measure*.nii, and these are the volumes that are entered into your second-level analyses when looking at the ICA.SpatialComponents tab in the second-level results window. If, on the other hand, you mean that you would like to reconstruct individual-subject ICA maps for other subjects (other than those included in your original ICA analysis), then the way to do this would be to go to Setup.ROIs and click on the 'ROI tools. Add network-ICA ROIs' button. This will add a new weighted-ROI that represents the group-level spatial maps computed in the group-ICA step. Using these network-ROIs as seeds in a new first-level analysis that uses multivariate-regression measures will produce the same individual-subject ICA maps as those computed in the back-projection step, and this can be applied to the same or other subjects (not necessarily those included in the original ICA analyses).
Regarding your question about Z-value thresholds, in CONN this approach can be performed simply by going to the ICA.SpatialComponents tab in the second-level results window, and defining a new one-sample t-test second-level analysis (i.e. simply select the 'AllSubjects' effect, contrast 1). The choice of threshold in those one-sample t-test results is somewhat arbitrary. Because of the nature of ICA analyses, these tests are actually post-hoc tests which will tend to produce strongly significant results. This is fine because these results are mainly used to help identify/represent the network associated with each component, so one is relatively free to choose the threshold values that results in the "cleaner" interpretation separately for each component (there is no confirmatory hypothesis testing here regarding these spatial maps, you just to want to characterize which aspect of the connectivity this component is capturing/representing, so one often uses relatively high/conservative thresholds in order to emphasize each specific network).
Last, regarding your question about between-group comparisons of spatial components, this is related to the above question. The average spatial map within a group does not just represents the network associated with this component but rather the connectivity pattern with this network (i.e. the connectivity between this network and the rest of the brain). Of course, when thresholding these single-group maps using very conservative thresholds you just get the network itself (in the same way that if you look at seed-to-voxel connectivity and use very conservative thresholds you will just get the seed itself). When comparing the spatial map across two groups you are effectively comparing the connectivity with this network across the two groups, so it is perfectly fine to find "out-of-network" areas that show differences in connectivity between the groups (that simply indicates that the connectivity between this network and those areas differs between your two groups). If you are only interested in within-network connectivity, then yes, you may constrain the between-group comparison to only include "within-network" areas, but otherwise (when interested in both within- and between- network connectivity) it is fine to perform these between-group comparisons across the entire brain.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Dear fellow CONN-artists,
I've recently begun diving in ICA based methods for extracting functional networks in a modestly large data set (110 subjects, 3 time points). One major question when running ICA is the choice of parameters for data reduction and # of components to be estimated.
The Calhoun paper presents an AIC and MDL approach for performing these estimates. Is this algorithm implemented in CONN? If not, how would one go about extracting the parameters to compute the AIC and MDL? The major parameters of interest would be the log of the maximum likelihood estimate of the model parameters as estimated from the fMRI data, ML (the number of time points following individual subject data reduction) and the number of sources N.
Also, is there an easy way to reconstruct individual subject ICA maps? How would one go about reconstructing individual subject time courses for time course ICA decomposition?
Additionally, how should one pick a Z value threshold? Calhoun suggests calculating the mean and variance of each component across the subjects and then perform random effects inference for a one-sample t-test. Using a standard 0.001 threshold often results in over-saturated maps for some components, whereas other components might tolerate only low thresholds. How does one go about picking a consistent threshold across components?
And lastly, what exactly happens when you contrast an ICA component at the group level? I've performed a contrast between two subject groups (drug users vs non users, [1 -1]) with a single ICA component selected. However the difference between those two subject groups is a cluster that falls outside of the spatial extent of that ICA component (after thresholding). Shouldn't the difference between these two subjects be constrained to only the spatial extent of the chosen ICA component?
I've recently begun diving in ICA based methods for extracting functional networks in a modestly large data set (110 subjects, 3 time points). One major question when running ICA is the choice of parameters for data reduction and # of components to be estimated.
The Calhoun paper presents an AIC and MDL approach for performing these estimates. Is this algorithm implemented in CONN? If not, how would one go about extracting the parameters to compute the AIC and MDL? The major parameters of interest would be the log of the maximum likelihood estimate of the model parameters as estimated from the fMRI data, ML (the number of time points following individual subject data reduction) and the number of sources N.
Also, is there an easy way to reconstruct individual subject ICA maps? How would one go about reconstructing individual subject time courses for time course ICA decomposition?
Additionally, how should one pick a Z value threshold? Calhoun suggests calculating the mean and variance of each component across the subjects and then perform random effects inference for a one-sample t-test. Using a standard 0.001 threshold often results in over-saturated maps for some components, whereas other components might tolerate only low thresholds. How does one go about picking a consistent threshold across components?
And lastly, what exactly happens when you contrast an ICA component at the group level? I've performed a contrast between two subject groups (drug users vs non users, [1 -1]) with a single ICA component selected. However the difference between those two subject groups is a cluster that falls outside of the spatial extent of that ICA component (after thresholding). Shouldn't the difference between these two subjects be constrained to only the spatial extent of the chosen ICA component?
Oct 18, 2016 03:10 PM | Jeff Browndyke
RE: ICA: Parameter Choice and 2nd Level Effects ?
I don't have CONN open in front of me, but I thought there was a
way to run a 2nd level analysis between groups (and/or conditions)
for a particular selected ICA network under the Spatial
Tab? I'm not certain what the metric is here, but it might be
an ICC constrained to only those areas within the ICA
network/ROIs. I would be helpful to get clarification on the
best possible way to approach a within-network constrained analysis
via ICA. Is one relegated to only using ROI-to-ROI analyses
on a specific a priori ICA network, or could you find the network
you are interested in via ICA, and then run voxel-wise analyses
looking for within-network degree centrality issues (ICC) or local
connectivity issues (ILC)?
Jeff
Jeff
Oct 18, 2016 05:10 PM | Shady El Damaty - Georgetown University
RE: ICA: Parameter Choice and 2nd Level Effects ?
Perhaps you are referring to what Alfonso meant in his earlier
response?
"If, on the other hand, you mean that you would like to reconstruct individual-subject ICA maps for other subjects (other than those included in your original ICA analysis), then the way to do this would be to go to Setup.ROIs and click on the 'ROI tools. Add network-ICA ROIs' button. This will add a new weighted-ROI that represents the group-level spatial maps computed in the group-ICA step. Using these network-ROIs as seeds in a new first-level analysis that uses multivariate-regression measures will produce the same individual-subject ICA maps as those computed in the back-projection step, and this can be applied to the same or other subjects (not necessarily those included in the original ICA analyses)."
"If, on the other hand, you mean that you would like to reconstruct individual-subject ICA maps for other subjects (other than those included in your original ICA analysis), then the way to do this would be to go to Setup.ROIs and click on the 'ROI tools. Add network-ICA ROIs' button. This will add a new weighted-ROI that represents the group-level spatial maps computed in the group-ICA step. Using these network-ROIs as seeds in a new first-level analysis that uses multivariate-regression measures will produce the same individual-subject ICA maps as those computed in the back-projection step, and this can be applied to the same or other subjects (not necessarily those included in the original ICA analyses)."
Oct 19, 2016 01:10 AM | Jeff Browndyke
RE: ICA: Parameter Choice and 2nd Level Effects ?
Actually, I was wondering more about what exactly the metric of
comparison is when one conducts a 2nd level analysis in the
"spatial components" tab. For instance, let's say I was
interested in simple regression of a variable [between-subjects
contrast; 01] to change between conditions (follow-up >
baseline) and selected "group-ICA_3" and "group_ICA_10" (i.e., DMN
network components) in the ICA network section and set those as
[10;01] in the between-measures contrast section. What is
this exactly comparing within the two ICA networks?
ICC?
Jeff
Jeff
Oct 20, 2016 05:10 PM | Alfonso Nieto-Castanon - Boston University
RE: ICA: Parameter Choice and 2nd Level Effects ?
Hi Jeff,
That is an interesting/complex question. ICA spatial components (backprojected to each subject) have somewhat of a dual conceptual interpreation, both as: 1) as a measure of the spatial distribution of a given network for each subject/condition; and 2) as a measure of the funcional connectivity between a given network and every voxel in the brain for each subject/condition. In more practical/mechanistic terms, what these "spatial component" backprojected map values actually represent is the functional connectivity between a given network and every voxel, something like a "network-to-voxel" analyses, just like standard "seed-to-voxel" analyses but now using a distributed network as a seed (in other words, the exact values/metric in the GICA1 backprojected spatial maps represent regression coefficients from a seed-to-voxel multivariate-regression analysis that used ICA group-level maps as seeds). Of course, just in the same way that you may use "seed-to-voxel" connectivity measures to look both at local connectivity effects (which may be interpreted in terms of functional homogeneity of the seed ROI) as well as distant/distributed connectivity patterns (which may be interpreted in terms of functional connectivity between the seed ROI and other regions), the equivalent "network-to-voxel" measures from the ICA spatial component analyses can be used to look both at local within-network effects, which may be more representative of the spatial distribution of a network, as well as distant/between-network effects, which may be more representative of the connectivity between this network and other areas.
So, coming back to your question, if you select the ICA_3 and ICA_10 components in the "spatial component" tab in CONN (with between-measures contrast [1 0;0 1]) and define a between-subjects regression with a covariate of interest (e.g. AllSubjects, age; contrast [0 1]) and a followup-baseline comparison (e.g. baseline, followup; contrast [-1 1]), what you are asking, conceptually, is whether the change between baseline and followup in any of these two networks spatial distribution / connecivity is associated with your covariate of interest. And, just to be precise, the "spatial distribution / connectivity" in the sentence above is represented by multivariate regression measures between the network ROIs and every voxel in the brain. If it helps, you may also think of the same analyses if you were to use two seeds instead of two networks, and simply consider that instead of looking at the seed-to-voxel connectivity profiles with two seeds now you are looking at the same seed-to-voxel connectivity profiles with two distributed networks.
Hope this helps
Alfonso
Originally posted by Jeff Browndyke:
That is an interesting/complex question. ICA spatial components (backprojected to each subject) have somewhat of a dual conceptual interpreation, both as: 1) as a measure of the spatial distribution of a given network for each subject/condition; and 2) as a measure of the funcional connectivity between a given network and every voxel in the brain for each subject/condition. In more practical/mechanistic terms, what these "spatial component" backprojected map values actually represent is the functional connectivity between a given network and every voxel, something like a "network-to-voxel" analyses, just like standard "seed-to-voxel" analyses but now using a distributed network as a seed (in other words, the exact values/metric in the GICA1 backprojected spatial maps represent regression coefficients from a seed-to-voxel multivariate-regression analysis that used ICA group-level maps as seeds). Of course, just in the same way that you may use "seed-to-voxel" connectivity measures to look both at local connectivity effects (which may be interpreted in terms of functional homogeneity of the seed ROI) as well as distant/distributed connectivity patterns (which may be interpreted in terms of functional connectivity between the seed ROI and other regions), the equivalent "network-to-voxel" measures from the ICA spatial component analyses can be used to look both at local within-network effects, which may be more representative of the spatial distribution of a network, as well as distant/between-network effects, which may be more representative of the connectivity between this network and other areas.
So, coming back to your question, if you select the ICA_3 and ICA_10 components in the "spatial component" tab in CONN (with between-measures contrast [1 0;0 1]) and define a between-subjects regression with a covariate of interest (e.g. AllSubjects, age; contrast [0 1]) and a followup-baseline comparison (e.g. baseline, followup; contrast [-1 1]), what you are asking, conceptually, is whether the change between baseline and followup in any of these two networks spatial distribution / connecivity is associated with your covariate of interest. And, just to be precise, the "spatial distribution / connectivity" in the sentence above is represented by multivariate regression measures between the network ROIs and every voxel in the brain. If it helps, you may also think of the same analyses if you were to use two seeds instead of two networks, and simply consider that instead of looking at the seed-to-voxel connectivity profiles with two seeds now you are looking at the same seed-to-voxel connectivity profiles with two distributed networks.
Hope this helps
Alfonso
Originally posted by Jeff Browndyke:
Actually, I was wondering more about what
exactly the metric of comparison is when one conducts a 2nd level
analysis in the "spatial components" tab. For instance, let's
say I was interested in simple regression of a variable
[between-subjects contrast; 01] to change between conditions
(follow-up > baseline) and selected "group-ICA_3" and
"group_ICA_10" (i.e., DMN network components) in the ICA network
section and set those as [10;01] in the between-measures contrast
section. What is this exactly comparing within the two ICA
networks? ICC?
Jeff
Jeff
Oct 20, 2016 08:10 PM | Jeff Browndyke
RE: ICA: Parameter Choice and 2nd Level Effects ?
Thanks so much for the tutelage and explanation, Alfonso.
Just to clarify when conducting these sorts of network-to-voxel analyses could the resulting blobs be both within and outside of the ICA network mask/regions? If both, I suspect I will need to use an exclusion mask from the ICA network to only get those areas that are extra-network but associated with the network x behavior regression?
How would one go about conducting a network-to-network connectivity timing analysis in CONN, similar to that implemented in FNC?
Warm regards,
Jeff
Just to clarify when conducting these sorts of network-to-voxel analyses could the resulting blobs be both within and outside of the ICA network mask/regions? If both, I suspect I will need to use an exclusion mask from the ICA network to only get those areas that are extra-network but associated with the network x behavior regression?
How would one go about conducting a network-to-network connectivity timing analysis in CONN, similar to that implemented in FNC?
Warm regards,
Jeff
Oct 21, 2016 09:10 PM | Shady El Damaty - Georgetown University
RE: ICA: Parameter Choice and 2nd Level Effects ?
Hmm, couldn't you just use the ICA networks as seeds in seed to
voxel analyses to accomplish this?
Oct 25, 2016 09:10 PM | Alfonso Nieto-Castanon - Boston University
RE: ICA: Parameter Choice and 2nd Level Effects ?
Yes, although FNC is really looking at
network-to-network measures only so that would almost identical to
simply performing ROI-to-ROI analyses instead of seed-to-voxel
using the same "ICA network" ROIs.
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Hope this helps
Alfonso
Originally posted by Shady El Damaty:
Hmm, couldn't you just use the ICA networks as
seeds in seed to voxel analyses to accomplish this?
Jan 10, 2018 10:01 PM | David Pagliaccio - New York State Psychiatric Institute at Columbia University
RE: ICA: Parameter Choice and 2nd Level Effects ?
Hello
I was just wondering if there was anything new regarding model-selection approach to help determine the optimal number of components in ICA
I was just wondering if there was anything new regarding model-selection approach to help determine the optimal number of components in ICA