Dear Andrew,

NBS is really great. Thanks for making this public!

Sadly, I am having some issues with the results.

If I understand correctly, a stable network should be present over a certain range of thresholds. (Not sure this is right)

However, when I lower the threshold the network simply keeps getting larger and larger until it covers maybe 30% of the entire network. (see image for an example)

Interestingly, this problem persists even when I change the contrast [1,0] to [-1,0].

I observe this with every variable I test for, mainly sex, age, brain size, etc....


I am working with about 100 subjects. 

If I was testing for sex differences (as in the example)

the design matrix I am using is 

1 1
0 1
1 1
0 1
0 1
0 1   
....
The contrast I am using is either [1,0] or [-1,0]

I am using the T-test for all testing. I simply change the first column to be age, brain size, etc. (Maybe there is something wrong with this)

One additional thing I notice is that the maximum t-statistic is usually about 2.5, which seems low compared to the thresholds others are using.


I have spent a long time trying to resolve this error with no luck.

Any help would be greatly appreciated.

Regards, Tim.

Hi Tim,

It looks like you are not including a column of ones in your design matrix.

I suggest using a t-stat threshold between 2 and 3. A sanity check is to set the threshold very high (say 10), in which case you should find no significant effects.

Andrew

Originally posted by Tim Lyoo:

Dear Andrew,

Thanks for the quick reply.

I am using the constant intercept column already.  

I have double checked two approaches (male/female column, and male column/constant column) and they show exactly the same result, so I am confident that this is not a design matrix issue.

I have begun to wonder whether these results are right after all. (The sanity check of threshold 10 comes out correctly, as the max t statistic is usually about 2.7)

I am wondering
(a) What is the rationale behind setting a certain threshold. If I choose a threshold between 2 and 3(as per your advice) the results look more nice (because they are a lot less dense), as in the figure below.

Maybe the rationale could be I am focusing on the 'strongly effected' edges?

(b) Do you think that these results are wrong?
(No stable network appears, network keeps getting larger and larger as threshold is lowered)

Thank you so much.

Regards, Tim.






Regards, Tim.



Originally posted by Andrew Zalesky:

Hi Tim,

Yes - male and female column versus male and constant column will give the same result. This is not specific to NBS, but a property of the GLM.

Setting an initial threshold is intrinsic to cluster based approaches. When you set a t-threshold of 2, for example, this ensures a t-stat of at least 2 must be seen at a connection for that connection to even have a chance of forming a cluster.

Its perfectly normal for the network to get larger and larger as the threshold is decreased.

I suggest choosing a threshold of 2, 2.5 or 3. It's probably not sensible to select a threshold of 0.5.

One strategy is to set the threshold as high as possible, while still finding a significant effect. This will determine the most strongly affected edges, to reuse your words.

Andrew


Originally posted by Tim Lyoo:

Dear Andrew,

Thanks for the kind explanation.
It really took a lot off my mind.
Again, thanks so much for the great software!


Regards, Tim.