help > 2nd level null results
Showing 1-16 of 16 posts
Sep 29, 2015 05:09 PM | Erik de Water
2nd level null results
Dear Dr. McLaren,
I have managed to run the gPPI toolbox on my data, and the 1st level gPPI contrasts seem to be specified correctly, and I get no error messages. However, when I submitted these 1st level contrasts to a 2nd-level one-sample t-test in SPM8, there were no significant voxels, not even at very lenient uncorrected thresholds. In fact, when I set the threshold to p<.05 uncorrected, all I found was "clusters" of 1 voxel. This is very unusual to me, especially since I have a relatively large sample (n = 58), and in the one-sample t-test that I used to select the seed region for the gPPI, a large fronto-parietal network was active, that survived FWE-correction. I would expect that the parietal seed I selected would show functional connectivity with at least the other parietal regions (e.g., the region contralateral to the seed region). What makes it even more odd, is that I tried 3 different seed regions, and 2 different contrasts, and the 2nd-level results are similar, in that there are no significant voxels, not even at very liberal thresholds. When I checked the 1st-level results for each subject, there were definitely regions that were functionally connected to the seed region, but there was large variability in the exact regions across subjects (which might explain the null findings at the 2nd level). Again, this seems unusual to me, given that I would expect some degree of consistency in functional connectivity across subjects.
Therefore, I get the feeling that I did something wrong, but I can't figure out exactly what it might be. I am still in doubt whether I specified the tasks and contrasts correctly. I have an event-related design, with parametric regressors to model the delay and amount of rewards that were used in a temporal discounting task (choices between a small immediate reward, and larger delayed reward). For the gPPI contrast, I'm only interested in comparing choices, not the parametric modulators. I have tried both specifying the names of the parametric modulators under "P.tasks", and only specifying the name of the regressor to which they are added, but that doesn't seem to make any difference.
I have attached my script. Note that I used a separate script to define the list fo subjects (not included).
Best,
Erik
I have managed to run the gPPI toolbox on my data, and the 1st level gPPI contrasts seem to be specified correctly, and I get no error messages. However, when I submitted these 1st level contrasts to a 2nd-level one-sample t-test in SPM8, there were no significant voxels, not even at very lenient uncorrected thresholds. In fact, when I set the threshold to p<.05 uncorrected, all I found was "clusters" of 1 voxel. This is very unusual to me, especially since I have a relatively large sample (n = 58), and in the one-sample t-test that I used to select the seed region for the gPPI, a large fronto-parietal network was active, that survived FWE-correction. I would expect that the parietal seed I selected would show functional connectivity with at least the other parietal regions (e.g., the region contralateral to the seed region). What makes it even more odd, is that I tried 3 different seed regions, and 2 different contrasts, and the 2nd-level results are similar, in that there are no significant voxels, not even at very liberal thresholds. When I checked the 1st-level results for each subject, there were definitely regions that were functionally connected to the seed region, but there was large variability in the exact regions across subjects (which might explain the null findings at the 2nd level). Again, this seems unusual to me, given that I would expect some degree of consistency in functional connectivity across subjects.
Therefore, I get the feeling that I did something wrong, but I can't figure out exactly what it might be. I am still in doubt whether I specified the tasks and contrasts correctly. I have an event-related design, with parametric regressors to model the delay and amount of rewards that were used in a temporal discounting task (choices between a small immediate reward, and larger delayed reward). For the gPPI contrast, I'm only interested in comparing choices, not the parametric modulators. I have tried both specifying the names of the parametric modulators under "P.tasks", and only specifying the name of the regressor to which they are added, but that doesn't seem to make any difference.
I have attached my script. Note that I used a separate script to define the list fo subjects (not included).
Best,
Erik
Sep 30, 2015 01:09 AM | Donald McLaren
RE: 2nd level null results
Erik,
There doesn't seem to be anything wrong with your script, so it's likely the design matrix that is causing the issue. Upon review of your design matrix, I noticed high correlations between several of the PPI and task regressors. These can be traced back to a limited number of trials in each run for some conditions. If you have less than 10 events of a specific event type, then you should use the concatenated approach in gPPI to reduce the correlation between regressors. The reduction in correlation is due to an increase in the number of trials. To use the concatenated approach, set P.ConcatR=1.
Let me know if you still end up with odd values.
-Donald
There doesn't seem to be anything wrong with your script, so it's likely the design matrix that is causing the issue. Upon review of your design matrix, I noticed high correlations between several of the PPI and task regressors. These can be traced back to a limited number of trials in each run for some conditions. If you have less than 10 events of a specific event type, then you should use the concatenated approach in gPPI to reduce the correlation between regressors. The reduction in correlation is due to an increase in the number of trials. To use the concatenated approach, set P.ConcatR=1.
Let me know if you still end up with odd values.
-Donald
Jan 22, 2016 12:01 PM | Chris Vriend
RE: 2nd level null results
Dear Donald,
I'm having a similar issue as Erik did; My BOLD effects are very robust (stop signal task) but the gPPI first levels and group main effects show almost no results. Inspection of my design matrix did indeed also show that there were high correlations between several regressors so I tried your solution of enabling P.ConcatR=1. That analyses, however, shows almost exactly the same results as before even though non-orthogonality seems to be less of an issue in the design matrix (the third event -Failedstop was moved to the first column). Do you happen to have any additional comments or possible solutions for this issue?
@Erik, I would love to hear of P.ConcatR=1 worked for you.
all the best and many thanks,
Chris
P looks like this:
subject: []
VOI: []
Region: []
SPMver: 8
directory: []
Estimate: 1
contrast: {'Omnibus F-test for PPI Analyses'}
extract: 'eig'
Tasks: {'1' 'SucGo' 'SucStop' 'FailStop'}
Weights: []
maskdir: []
equalroi: 0
FLmask: 0
VOI2: {}
analysis: 'psy'
method: 'cond'
CompContrasts: 1
Weighted: []
Contrasts: [1x2 struct]
ConcatR: 1
I'm having a similar issue as Erik did; My BOLD effects are very robust (stop signal task) but the gPPI first levels and group main effects show almost no results. Inspection of my design matrix did indeed also show that there were high correlations between several regressors so I tried your solution of enabling P.ConcatR=1. That analyses, however, shows almost exactly the same results as before even though non-orthogonality seems to be less of an issue in the design matrix (the third event -Failedstop was moved to the first column). Do you happen to have any additional comments or possible solutions for this issue?
@Erik, I would love to hear of P.ConcatR=1 worked for you.
all the best and many thanks,
Chris
P looks like this:
subject: []
VOI: []
Region: []
SPMver: 8
directory: []
Estimate: 1
contrast: {'Omnibus F-test for PPI Analyses'}
extract: 'eig'
Tasks: {'1' 'SucGo' 'SucStop' 'FailStop'}
Weights: []
maskdir: []
equalroi: 0
FLmask: 0
VOI2: {}
analysis: 'psy'
method: 'cond'
CompContrasts: 1
Weighted: []
Contrasts: [1x2 struct]
ConcatR: 1
Feb 17, 2016 02:02 AM | Donald McLaren
RE: 2nd level null results
Chris,
Sorry I missed your email and didn't respond sooner.
Some additional thoughts and suggestions:
(1) How many trials of each condition do you have in your data? It's possible that there aren't enough trials.
(2) If you are using a 0sec duration, I would change the duration to the RT of each event or the mean RT. It turns out that there can be quite a big difference in the PPI regressors between 0 second duration event and non-zero duration event.
(3) It's also possible that the connectivity with your region doesn't change a lot with a specific location in the brain. To address this issue, I would look at the following: (a) individual subject maps; and (b) selecting the top 5% of voxels in the brain and see if they fall into specific brain networks (for details see my 2014 paper in NeuroImage or Humbert el al. 2014 in Physiological Reports). The idea is that there might not be specific connections that have a large change, but that there is a small change that is distributed to specific cortical networks from your seed region.
(4) Between subject variability is higher and you need more subjects.
Best,
Donald
Sorry I missed your email and didn't respond sooner.
Some additional thoughts and suggestions:
(1) How many trials of each condition do you have in your data? It's possible that there aren't enough trials.
(2) If you are using a 0sec duration, I would change the duration to the RT of each event or the mean RT. It turns out that there can be quite a big difference in the PPI regressors between 0 second duration event and non-zero duration event.
(3) It's also possible that the connectivity with your region doesn't change a lot with a specific location in the brain. To address this issue, I would look at the following: (a) individual subject maps; and (b) selecting the top 5% of voxels in the brain and see if they fall into specific brain networks (for details see my 2014 paper in NeuroImage or Humbert el al. 2014 in Physiological Reports). The idea is that there might not be specific connections that have a large change, but that there is a small change that is distributed to specific cortical networks from your seed region.
(4) Between subject variability is higher and you need more subjects.
Best,
Donald
Mar 26, 2016 06:03 PM | Gina Joue
RE: 2nd level null results
Hi,
I am also trying to run gPPI with P.ConcatR = 1 in order to address a problem with collinearity (regressors depend on subject responses so there are low number of events within runs for some conditions for some subjects).
When I had P.ConcatR = 0, the PPI contrasts estimate, but I get an error when running second level stats that there were no significant voxels. I see that the design matrix for some subjects indeed have collinearity issues (some PPI parameters "not uniquely specified").
When I set P.ConcatR = 1 to concatenate my 4 sessions/subject, I get the error
"Estimation Failed
PPI Contrasts were not estimated for some reason."
and I see that the collinearity problem is not all that better. Additionally, in the resulting gPPI design matrix I see individual session constants for the 2nd to 4th sessions and the first one is weighted across all 4 sessions (is this correct?).
Here is my gPPI configuration for a subject:
P =
subject: 's1'
directory: '/home/data/'
outdir: [1x86 char]
maskdir: [1x86 char]
analysis: 'psy'
extract: 'eig'
method: 'cond'
SPMver: 8
Estimate: 1
Contrast: 1
ConcatR: 1
Region: 'ROI1'
VOI: [1x82 char]
Tasks: {1x9 cell}
CompContrasts: 1
Contrasts: [1x8 struct]
and I have MinEvents set to 10 and also tried 20.
Thanks in advance for any advice!
Gina
I am also trying to run gPPI with P.ConcatR = 1 in order to address a problem with collinearity (regressors depend on subject responses so there are low number of events within runs for some conditions for some subjects).
When I had P.ConcatR = 0, the PPI contrasts estimate, but I get an error when running second level stats that there were no significant voxels. I see that the design matrix for some subjects indeed have collinearity issues (some PPI parameters "not uniquely specified").
When I set P.ConcatR = 1 to concatenate my 4 sessions/subject, I get the error
"Estimation Failed
PPI Contrasts were not estimated for some reason."
and I see that the collinearity problem is not all that better. Additionally, in the resulting gPPI design matrix I see individual session constants for the 2nd to 4th sessions and the first one is weighted across all 4 sessions (is this correct?).
Here is my gPPI configuration for a subject:
P =
subject: 's1'
directory: '/home/data/'
outdir: [1x86 char]
maskdir: [1x86 char]
analysis: 'psy'
extract: 'eig'
method: 'cond'
SPMver: 8
Estimate: 1
Contrast: 1
ConcatR: 1
Region: 'ROI1'
VOI: [1x82 char]
Tasks: {1x9 cell}
CompContrasts: 1
Contrasts: [1x8 struct]
and I have MinEvents set to 10 and also tried 20.
Thanks in advance for any advice!
Gina
Mar 28, 2016 12:03 AM | Donald McLaren
RE: 2nd level null results
Gina,
Can you attach the SPM.mat for the concatenated version?
Best,
Donald
Can you attach the SPM.mat for the concatenated version?
Best,
Donald
Mar 28, 2016 12:03 AM | Gina Joue
RE: 2nd level null results
Here is the SPM for the concatenated version with min 10 events.
THANK YOU, Donald!
THANK YOU, Donald!
Apr 7, 2016 05:04 PM | Donald McLaren
RE: 2nd level null results
The design matrix looks correct.
Can you set P.wb=0 in your gPPI script. This should fix the issues. If it is already set to 0, then please let me know and I'll try to identify another solution.
This will revert the code back to slice-by-slice computations, rather than reading in the whole volume.
Best,
Donald
Can you set P.wb=0 in your gPPI script. This should fix the issues. If it is already set to 0, then please let me know and I'll try to identify another solution.
This will revert the code back to slice-by-slice computations, rather than reading in the whole volume.
Best,
Donald
Apr 8, 2016 12:04 PM | Gina Joue
RE: 2nd level null results
Still the same issues (noncollinearity
and contrasts not estimating) with P.wb=0...
Thank you,
g
Originally posted by Donald McLaren:
Thank you,
g
Originally posted by Donald McLaren:
The design matrix looks correct.
Can you set P.wb=0 in your gPPI script. This should fix the issues. If it is already set to 0, then please let me know and I'll try to identify another solution.
This will revert the code back to slice-by-slice computations, rather than reading in the whole volume.
Best,
Donald
Can you set P.wb=0 in your gPPI script. This should fix the issues. If it is already set to 0, then please let me know and I'll try to identify another solution.
This will revert the code back to slice-by-slice computations, rather than reading in the whole volume.
Best,
Donald
Apr 8, 2016 03:04 PM | Donald McLaren
RE: 2nd level null results
Gina,
It could be either the SPM.mat is bad or the estimate is problematic. I need a bit more information.
What happens if you open SPM, then click estimate, then select the concatenated SPM.mat file.
Does it estimate via the SPM GUI? Does it give you an error message?
Best,
Donald
It could be either the SPM.mat is bad or the estimate is problematic. I need a bit more information.
What happens if you open SPM, then click estimate, then select the concatenated SPM.mat file.
Does it estimate via the SPM GUI? Does it give you an error message?
Best,
Donald
May 3, 2016 03:05 PM | Gina Joue
RE: 2nd level null results
Sorry!!!!! I totally missed your message and was coming back to the
forum to write you because I thought I hadn't heard from you...
You mean load an SPM.mat (output from a concatenated gPPI analysis for which the contrasts did not estimate) for a random subject through the SPM GUI, define a contrast, and see if the contrast estimates? Yes, I tried that and I don't get any errors through the SPM GUI.
Thank you for your help!
Gina
You mean load an SPM.mat (output from a concatenated gPPI analysis for which the contrasts did not estimate) for a random subject through the SPM GUI, define a contrast, and see if the contrast estimates? Yes, I tried that and I don't get any errors through the SPM GUI.
Thank you for your help!
Gina
May 6, 2016 02:05 AM | Donald McLaren
RE: 2nd level null results
Estimation and contrasts are two separate steps. From your email, I
can't tell what steps you did in SPM versus gPPI.
Does the gPPI code estimate the model or does the estimation fail?
If it doesn't estimate the model, can you estimate the failed model through the SPM GUI?
If it does estimate the model, but not the contrasts, what are the contrasts that you were trying to use in the PPI config file?
Does the gPPI code estimate the model or does the estimation fail?
If it doesn't estimate the model, can you estimate the failed model through the SPM GUI?
If it does estimate the model, but not the contrasts, what are the contrasts that you were trying to use in the PPI config file?
May 9, 2016 04:05 PM | Gina Joue
RE: 2nd level null results
Sorry, I have run so many analysis versions of the data that I
cannot reproduce the original gPPI error I had, which was
"Estimation Failed
PPI Contrasts were not estimated for some reason."
when I run gPPI with P.ConcatR = 1 to concatenate my 4 sessions/subject.
Instead, now gPPI estimates the model but not the contrasts, i.e. the error is only now "PPI Contrasts were not estimated for some reason" -- regardless of whether P.wb = 0 or 1. However, if P.ConcatR = 0, I don't get any errors (model and contrasts estimate).
Through the SPM GUI, if I load the now estimated SPM.mat from the concatenated gPPI analysis and define a contrast involving a random PPI regressor, the contrast estimates with no problem....
So now I'm trying to understand what I might perhaps have set up incorrectly so that the contrast estimates are not computed. I have attached the SPM.mat for the latest gPPI run I did.
Thank you!
Gina
"Estimation Failed
PPI Contrasts were not estimated for some reason."
when I run gPPI with P.ConcatR = 1 to concatenate my 4 sessions/subject.
Instead, now gPPI estimates the model but not the contrasts, i.e. the error is only now "PPI Contrasts were not estimated for some reason" -- regardless of whether P.wb = 0 or 1. However, if P.ConcatR = 0, I don't get any errors (model and contrasts estimate).
Through the SPM GUI, if I load the now estimated SPM.mat from the concatenated gPPI analysis and define a contrast involving a random PPI regressor, the contrast estimates with no problem....
So now I'm trying to understand what I might perhaps have set up incorrectly so that the contrast estimates are not computed. I have attached the SPM.mat for the latest gPPI run I did.
Thank you!
Gina
May 10, 2016 12:05 AM | Donald McLaren
RE: 2nd level null results
Can you also attached the config file? I need to see P.Contrasts to
see where the error might be occurring.
Best,
Donald
Best,
Donald
May 10, 2016 09:05 AM | Gina Joue
RE: 2nd level null results
Here's the gPPI config file.
Thank you!
Gina
Thank you!
Gina
May 11, 2016 12:05 AM | Donald McLaren
RE: 2nd level null results
Gina,
The current issue is that the left field has 103 values, but the model only has 52 columns. This is causing SPM to generate an error when it tries to build a contrast. As it errors out, no contrasts are generated.
When you use the task names, don't include the 'PPI_', these are automatically appended by the PPI program. Additionally, adding them in the task names will cause an error with the underlying code in createVec.
I would change the P.Contrasts.left and P.Contrasts.right to contain the task names rather than the left field containing a vector. This protects against the possibility of different task orders in different subjects.
Hope this helps.
Best,
Donald
Originally posted by Gina Joue:
The current issue is that the left field has 103 values, but the model only has 52 columns. This is causing SPM to generate an error when it tries to build a contrast. As it errors out, no contrasts are generated.
When you use the task names, don't include the 'PPI_', these are automatically appended by the PPI program. Additionally, adding them in the task names will cause an error with the underlying code in createVec.
I would change the P.Contrasts.left and P.Contrasts.right to contain the task names rather than the left field containing a vector. This protects against the possibility of different task orders in different subjects.
Hope this helps.
Best,
Donald
Originally posted by Gina Joue:
Sorry, I have run so many analysis versions of
the data that I cannot reproduce the original gPPI error I had,
which was
"Estimation Failed
PPI Contrasts were not estimated for some reason."
when I run gPPI with P.ConcatR = 1 to concatenate my 4 sessions/subject.
Instead, now gPPI estimates the model but not the contrasts, i.e. the error is only now "PPI Contrasts were not estimated for some reason" -- regardless of whether P.wb = 0 or 1. However, if P.ConcatR = 0, I don't get any errors (model and contrasts estimate).
Through the SPM GUI, if I load the now estimated SPM.mat from the concatenated gPPI analysis and define a contrast involving a random PPI regressor, the contrast estimates with no problem....
So now I'm trying to understand what I might perhaps have set up incorrectly so that the contrast estimates are not computed. I have attached the SPM.mat for the latest gPPI run I did.
Thank you!
Gina
"Estimation Failed
PPI Contrasts were not estimated for some reason."
when I run gPPI with P.ConcatR = 1 to concatenate my 4 sessions/subject.
Instead, now gPPI estimates the model but not the contrasts, i.e. the error is only now "PPI Contrasts were not estimated for some reason" -- regardless of whether P.wb = 0 or 1. However, if P.ConcatR = 0, I don't get any errors (model and contrasts estimate).
Through the SPM GUI, if I load the now estimated SPM.mat from the concatenated gPPI analysis and define a contrast involving a random PPI regressor, the contrast estimates with no problem....
So now I'm trying to understand what I might perhaps have set up incorrectly so that the contrast estimates are not computed. I have attached the SPM.mat for the latest gPPI run I did.
Thank you!
Gina
