help > Threshold
Showing 1-2 of 2 posts
Display:
Results per page:
May 10, 2021  05:05 PM | Noemi Meylakh
Threshold
Hi team,

I have a question regarding the meaning of the Threshold number in the NBS GUI.

When I ran a comparison between controls v patients, it only became significant when the threshold was 2.5

However, I ran a different comparison between controls v new patient group, and it wasn't significant at 2.5 but became very significant at 3.

I am confused as to why making the threshold more stringent made this second comparison significant, whilst when the threshold was lower it wasn't significant. And furthermore, why this wasn't the case with the first comparison. What does this threshold mean?

Would really appreciate some guidance!!

Thank you,
Noemi
May 11, 2021  12:05 AM | Andrew Zalesky
RE: Threshold
Hi Noemi, 

What you describe is completely plausible. Increasing the threshold won't necessarily make it harder to find significant findings. In brief, larger thresholds will also make it harder to identify large subnetworks in the null data. So larger thresholds don't necessarily mean that the null hypothesis becomes harder to reject. 

Consider setting the threshold based on a minimum meaningful effect size. Based on you experiment and hypotheses. you might only be interested in effects with a Cohen's d greater than 0.1 (for example), and thus you can set the t-stat threshold at 0.1*sqrt(n), where n is your sample size. THis will ensure that you only detect subnetworks with effects exceeding d=0.1

Andrew

Originally posted by Noemi Meylakh:
Hi team,

I have a question regarding the meaning of the Threshold number in the NBS GUI.

When I ran a comparison between controls v patients, it only became significant when the threshold was 2.5

However, I ran a different comparison between controls v new patient group, and it wasn't significant at 2.5 but became very significant at 3.

I am confused as to why making the threshold more stringent made this second comparison significant, whilst when the threshold was lower it wasn't significant. And furthermore, why this wasn't the case with the first comparison. What does this threshold mean?

Would really appreciate some guidance!!

Thank you,
Noemi