Hi Andrew,

I am trying to use NBS to analyze data. I have one group of subjects and I would like to test if the means of some connections are greater than or smaller than zero.
1) I chose one sample as the statistic analysis. And the design matrix is listed as follow:
1
1
1
1
...
I set the contrast of 1. Is this the correct way of performing the analysis?
2) According to the manual, "The one sample test randomly flips the sign of each data point for each permutation, which corresponds to a one-sided, one sample t-test. Note that the one sample test always assesses whether the mean is greater than zero, irrespective of the sign of the contrast vector." I guess the one sample test is only to test the positive effect, right? What if I would like to test both the positive and negative effect?
3) What's the difference between "one-sample" and "t-test"? Shouldn't I obtain the same result by using "one-sample" and "t-test"by applying the same design matrix and contrast like I mentioned above?
Thank you in advance.

Best regards,
Lijun

Hi Yin,

1) Yes correct.

2) If you would like to test for a negative effect, simply multiply your connectivity values by -1.You cannot test for both a negative and a positive effect using one-sample.

3) t-test is for testing other contrasts, such as between-group differences, correlations, etc. The one-sample test only tests the null hypothesis that connectivity values are 0. One-sample test is rarely used in practice.

Andrew




Originally posted by LJ Yin:

Hi Andrew,

Thank you very much for your reply. I still have one follow-up question to my second question. I performed one sample test to test both positive and negative effect as you told me but by using FDR (no significant results were found by using NBS). But I found highly overlapped results from both effects. How could this happen?

Many thanks,
Lijun

Hi Lijun,

When using FDR, the number of permutations must be very large, as described in the manual.

For NBS, 5000 permutations is ok, but for FDR I suggest generating at least 100,000 permutations, otherwise false positives are likely.This will take a longer time to run.


The reason why you have overlapping positive and negative effects is probably due to insufficient number of permutations with FDR.

Andrew

 
Originally posted by LJ Yin:

Originally posted by LJ Yin: Hi Andrew, I have a quite similar question.
I tried the same analysis (one-sample t-test), on 20 matrices from the same group of typically developing controls. I did it to do a first thresholding on the matrices by testing, within this group, which connections were significantly different from 0.
Here the setup. Test: One-sample; Method: NBS, Sig: 0.05. Threshold: 3. 5000 permutations. The design matrix is as LJ Yin showed, contrast set as [1].

However, when I run the analysis, the Max Size Random, Max Size Actual never change from 0.0, and Lowest p-value never changes from 1.0.
Obviously, there is no significant result.
How is this possible?
What did I do wrong?

Many regards,
Alessio

Hi Alessio,

Try lowering the threshold. Try 0.2 or 0.5, or threshold values in this range.The threshold value for the one-sample t-test is expressed in terms of the mean connectivity across all samples.

If this does not work, check that your connectivity values are indeed positive for the majority of samples.

Andrew



Originally posted by Alessio Bellato:

Oh I see, I have just tried and it works with lower thresholds.
Thanks very much Andrew!
Regards,
Alessio
Originally posted by Andrew Zalesky:

Hi,

thanks for elaborating on this topic. What is not clear to me is what the threshold means in a one-sample test? Why is that in the range around the mean connectivity and how would you justify any choice of threshold? I have one group and only want to assess which values are different from zero, corrected for multiple testing. Without NBS I would do a one-sample t-test for every edge over subjects and apply FDR correction, now I want to do the equivalent with NBS.

Many thanks,
Selma

Hi Selma, 

The threshold for the one-sample t-test is applied to the mean connectivity value. 

So if you are considering functional connectivity estimated with Pearson correlation, a threshold of 0.3 will consider clusters/components of connections for any connections for which the mean connectivity value exceeds 0.3. 

The threshold can thus be used to set a minimum meaningful connectivity value. There is no right or wrong threshold. 

I hope that helps. 

Andrew


Originally posted by Selma Lugtmeijer: