Hi Andrew,

I am trying to use NBS to analyze data. I have one group of subjects and I would like to test if the means of some connections are greater than or smaller than zero.
1) I chose one sample as the statistic analysis. And the design matrix is listed as follow:
1
1
1
1
...
I set the contrast of 1. Is this the correct way of performing the analysis?
2) According to the manual, "The one sample test randomly flips the sign of each data point for each permutation, which corresponds to a one-sided, one sample t-test. Note that the one sample test always assesses whether the mean is greater than zero, irrespective of the sign of the contrast vector." I guess the one sample test is only to test the positive effect, right? What if I would like to test both the positive and negative effect?
3) What's the difference between "one-sample" and "t-test"? Shouldn't I obtain the same result by using "one-sample" and "t-test"by applying the same design matrix and contrast like I mentioned above?
Thank you in advance.

Best regards,
Lijun

Hi Yin,

1) Yes correct.

2) If you would like to test for a negative effect, simply multiply your connectivity values by -1.You cannot test for both a negative and a positive effect using one-sample.

3) t-test is for testing other contrasts, such as between-group differences, correlations, etc. The one-sample test only tests the null hypothesis that connectivity values are 0. One-sample test is rarely used in practice.

Andrew




Originally posted by LJ Yin:

Hi Andrew,

Thank you very much for your reply. I still have one follow-up question to my second question. I performed one sample test to test both positive and negative effect as you told me but by using FDR (no significant results were found by using NBS). But I found highly overlapped results from both effects. How could this happen?

Many thanks,
Lijun

Hi Lijun,

When using FDR, the number of permutations must be very large, as described in the manual.

For NBS, 5000 permutations is ok, but for FDR I suggest generating at least 100,000 permutations, otherwise false positives are likely.This will take a longer time to run.


The reason why you have overlapping positive and negative effects is probably due to insufficient number of permutations with FDR.

Andrew

 
Originally posted by LJ Yin:

Originally posted by LJ Yin: Hi Andrew, I have a quite similar question.
I tried the same analysis (one-sample t-test), on 20 matrices from the same group of typically developing controls. I did it to do a first thresholding on the matrices by testing, within this group, which connections were significantly different from 0.
Here the setup. Test: One-sample; Method: NBS, Sig: 0.05. Threshold: 3. 5000 permutations. The design matrix is as LJ Yin showed, contrast set as [1].

However, when I run the analysis, the Max Size Random, Max Size Actual never change from 0.0, and Lowest p-value never changes from 1.0.
Obviously, there is no significant result.
How is this possible?
What did I do wrong?

Many regards,
Alessio

Hi Alessio,

Try lowering the threshold. Try 0.2 or 0.5, or threshold values in this range.The threshold value for the one-sample t-test is expressed in terms of the mean connectivity across all samples.

If this does not work, check that your connectivity values are indeed positive for the majority of samples.

Andrew



Originally posted by Alessio Bellato:

Oh I see, I have just tried and it works with lower thresholds.
Thanks very much Andrew!
Regards,
Alessio
Originally posted by Andrew Zalesky:

Hi,

thanks for elaborating on this topic. What is not clear to me is what the threshold means in a one-sample test? Why is that in the range around the mean connectivity and how would you justify any choice of threshold? I have one group and only want to assess which values are different from zero, corrected for multiple testing. Without NBS I would do a one-sample t-test for every edge over subjects and apply FDR correction, now I want to do the equivalent with NBS.

Many thanks,
Selma

Hi Selma, 

The threshold for the one-sample t-test is applied to the mean connectivity value. 

So if you are considering functional connectivity estimated with Pearson correlation, a threshold of 0.3 will consider clusters/components of connections for any connections for which the mean connectivity value exceeds 0.3. 

The threshold can thus be used to set a minimum meaningful connectivity value. There is no right or wrong threshold. 

I hope that helps. 

Andrew


Originally posted by Selma Lugtmeijer:

  

Dear Andrew

I have came across this discussion and I'm curious about why you are recommending a much lower range of thresholds for a one-sample t-test.

I conducted some one-sample t-test analyses to examine within group patterns of connectivity. 

I have found that using the same t threshold as I have used for unpaired t-test analyses does not yield any significant results.

If I open the NBS.test_stat file generated by the one-sample t-test analysis, the values look like coefficient values, not t-stat values.
 
If I lower the t threshold to just below 1 for the one-sample t-test analyses, I do see significant results.  

Is there something different in the thresholding process for a one-sample t-test compared to an unpaired t-test that I need to consider? Am I getting something wrong here?

All the best , Liam.

Hi Liam, 

There is no "right" or "wrong" threshold. 

For the one-sample test, the test statistic is based on mean connectivity (as opposed to a t-statistic), so depending on your connectivity values, you may want to try a relatively low threshold, compared to the t-statistic threshold used for the two-sample t-test. 

In practice, the one-sample test is rarely used. 

Andrew

 

 

Originally posted by liam nestor:

  

Dear Andrew

I have came across this discussion and I'm curious about why you are recommending a much lower range of thresholds for a one-sample t-test.

I conducted some one-sample t-test analyses to examine within group patterns of connectivity. 

I have found that using the same t threshold as I have used for unpaired t-test analyses does not yield any significant results.

If I open the NBS.test_stat file generated by the one-sample t-test analysis, the values look like coefficient values, not t-stat values.
 
If I lower the t threshold to just below 1 for the one-sample t-test analyses, I do see significant results.  

Is there something different in the thresholding process for a one-sample t-test compared to an unpaired t-test that I need to consider? Am I getting something wrong here?

All the best , Liam.

 

Thanks Andrew. That makes sense now. All the best, Liam.