**Subnetwork from dNBS not a subset of significant edges from edge-wise Welch's t-test**

*University of Washington, Seattle*

Hi,

Thank you very much for this great tool. I have a question, as follows.

I performed two analyses after obtaining subject-specific directed functional connectivity: 1) used dNBS with primary threshold 1.8, 10K permuations, and size based on extent, to obtain a subnetwork with lower strength in diseased group compared to healthy group at significance level 0.05. 2) obtained edge-wise p-values for lower strength in diseased group compared to healthy group using Welch's t-test for each edge in the directed functional connectivities.

However, the subnetwork obtained in analysis 1 is not a subset of edges having edge-wise p-value less than 0.05 in analysis 2. Isn't this wierd? I would really appreciate an explanation.

Some more detail: Only two edges in the subnetwork in analysis 1 have edge-wise p-value less than 0.05 in edge-wise Welch's t-test (analysis 2).

Thank you very much for explaining this.

Best regards,

Rahul

Hi Rahul,

this is most likely due to your use of a lenient primary threshold (t=1.8).

If the sample size is not sufficiently large, t=1.8 will not ensure that p<0.05 and thus it is unsurpising that several connections do not have p<0.05.

I recommend increasing the primary threshold. Perhaps try t=3.

Andrew

*Originally posted by Rahul Biswas:*

Hi,

Thank you very much for this great tool. I have a question, as follows.

I performed two analyses after obtaining subject-specific directed functional connectivity: 1) used dNBS with primary threshold 1.8, 10K permuations, and size based on extent, to obtain a subnetwork with lower strength in diseased group compared to healthy group at significance level 0.05. 2) obtained edge-wise p-values for lower strength in diseased group compared to healthy group using Welch's t-test for each edge in the directed functional connectivities.

However, the subnetwork obtained in analysis 1 is not a subset of edges having edge-wise p-value less than 0.05 in analysis 2. Isn't this wierd? I would really appreciate an explanation.

Some more detail: Only two edges in the subnetwork in analysis 1 have edge-wise p-value less than 0.05 in edge-wise Welch's t-test (analysis 2).

Thank you very much for explaining this.

Best regards,

Rahul

*University of Washington, Seattle*

Hi Andrew,

Thanks for your answer. Just a quick follow-up with details on the sample size.

We have a sample size of 34 subjects in the diseased group and 41 subjects in the healthy group. May I double check, if you still think this is the same issue of sample size that you mentioned? By the way, I checked that t = 1.8 would have a right-sided p-value < 0.05 for t-distribution with df = 73. Perhaps the distribution of the t statistic computed by dNBS is different.

Indeed, for t = 2.5, the subnetwork has three edges, for which the edge-wise p-value of two edges are less than 0.05 and that of the third edge is 0.056. t = 3 did not yield any significant outcomes.

Thank you!

Best regards,

Rahul

*Originally posted by Andrew Zalesky:*

Hi Rahul,

this is most likely due to your use of a lenient primary threshold (t=1.8).

If the sample size is not sufficiently large, t=1.8 will not ensure that p<0.05 and thus it is unsurpising that several connections do not have p<0.05.

I recommend increasing the primary threshold. Perhaps try t=3.

Andrew

Originally posted by Rahul Biswas:

Hi,

Thank you very much for this great tool. I have a question, as follows.

I performed two analyses after obtaining subject-specific directed functional connectivity: 1) used dNBS with primary threshold 1.8, 10K permuations, and size based on extent, to obtain a subnetwork with lower strength in diseased group compared to healthy group at significance level 0.05. 2) obtained edge-wise p-values for lower strength in diseased group compared to healthy group using Welch's t-test for each edge in the directed functional connectivities.

However, the subnetwork obtained in analysis 1 is not a subset of edges having edge-wise p-value less than 0.05 in analysis 2. Isn't this wierd? I would really appreciate an explanation.

Some more detail: Only two edges in the subnetwork in analysis 1 have edge-wise p-value less than 0.05 in edge-wise Welch's t-test (analysis 2).

Thank you very much for explaining this.

Best regards,

Rahul

Hi Rahul,

I suspect that this may be caused by slight differences in how you and dNBS are computing the t-test. The computation of the denominator (standard deviations) varies between t-tests. I'm not familiar with how the t-test is computed in dNBS - perhaps others can comment. Of course it could also be a bug in the dNBS code! Let us know if you look into this issue in more detail.

However, your intuition is correct. If you set a primary threshold of p<0.05 (or corresponding t-stat) all edges within a signifciant component should be p<0.05.

Best wishes,

Andrew

*Originally posted by Rahul Biswas:*

Hi Andrew,

Thanks for your answer. Just a quick follow-up with details on the sample size.

We have a sample size of 34 subjects in the diseased group and 41 subjects in the healthy group. May I double check, if you still think this is the same issue of sample size that you mentioned? By the way, I checked that t = 1.8 would have a right-sided p-value < 0.05 for t-distribution with df = 73. Perhaps the distribution of the t statistic computed by dNBS is different.

Indeed, for t = 2.5, the subnetwork has three edges, for which the edge-wise p-value of two edges are less than 0.05 and that of the third edge is 0.056. t = 3 did not yield any significant outcomes.

Thank you!

Best regards,

Rahul

Originally posted by Andrew Zalesky:

Hi Rahul,

this is most likely due to your use of a lenient primary threshold (t=1.8).

If the sample size is not sufficiently large, t=1.8 will not ensure that p<0.05 and thus it is unsurpising that several connections do not have p<0.05.

I recommend increasing the primary threshold. Perhaps try t=3.

Andrew

Originally posted by Rahul Biswas:

Hi,

Thank you very much for this great tool. I have a question, as follows.

Thank you very much for explaining this.

Best regards,

Rahul

*University of Washington, Seattle*

Thank you very much Andrew for your kind and helpful answers!

*Originally posted by Andrew Zalesky:*

Hi Rahul,

I suspect that this may be caused by slight differences in how you and dNBS are computing the t-test. The computation of the denominator (standard deviations) varies between t-tests. I'm not familiar with how the t-test is computed in dNBS - perhaps others can comment. Of course it could also be a bug in the dNBS code! Let us know if you look into this issue in more detail.

However, your intuition is correct. If you set a primary threshold of p<0.05 (or corresponding t-stat) all edges within a signifciant component should be p<0.05.

Best wishes,

Andrew

Originally posted by Rahul Biswas:

Hi Andrew,

Thanks for your answer. Just a quick follow-up with details on the sample size.

We have a sample size of 34 subjects in the diseased group and 41 subjects in the healthy group. May I double check, if you still think this is the same issue of sample size that you mentioned? By the way, I checked that t = 1.8 would have a right-sided p-value < 0.05 for t-distribution with df = 73. Perhaps the distribution of the t statistic computed by dNBS is different.

Indeed, for t = 2.5, the subnetwork has three edges, for which the edge-wise p-value of two edges are less than 0.05 and that of the third edge is 0.056. t = 3 did not yield any significant outcomes.

Thank you!

Best regards,

Rahul

Originally posted by Andrew Zalesky:

Hi Rahul,

this is most likely due to your use of a lenient primary threshold (t=1.8).

I recommend increasing the primary threshold. Perhaps try t=3.

Andrew

Originally posted by Rahul Biswas:

Hi,

Thank you very much for this great tool. I have a question, as follows.

Thank you very much for explaining this.

Best regards,

Rahul

*Nanjing University of Science and Technology*

Hi everyone,

I have recently been introduced to this tool, which greatly interests me. I am sorry if my questions seem silly.

I have directional connectivity matrices with (240 control and 360 MCI) subjects. I want to measure the most significant edges where ( CN > MCI) or (MCI > CN), but I am not very sure how to analyze it. I put all subjects 600 in one folder. I have tried many experiments with varying thresholds from (2.5 to 4.18), sometimes I get a set of only subnetworks, sometimes 2 or 3. I have attached my design matrix. Could you please verify this is correct?

Summary of my questions.

1) Is the design matrix correct for the task I am trying to do?

2) What does it mean if the tool dNBS outputs one, two, or three subnetworks? How could we analyze them?

3) How to measure the edges with CN > MCI or MCI > CN.

Thank you for your valuable time.

More info: In the design matrix, the first 360 are MCI subjects, and the next 240 are Normal controls.

Contrast: [-1,1]

signification 0.05

component size Extent

permutations 10k

statistical test t-test

Hi Saqib,

yes - design matrix and contrasts look correct.

You can use [-1 1] and [1 -1] to test for MCI>CN and MCI<CN respecitvely.

It is possible that more than one subnetwork is found.

I recommend checking out the NBS manual as well as previous papers that have performed post hoc analyses on subnetworks.

Best,

Andrew

*Originally posted by Saqib Mamoon:*

Hi everyone,

I have recently been introduced to this tool, which greatly interests me. I am sorry if my questions seem silly.

I have directional connectivity matrices with (240 control and 360 MCI) subjects. I want to measure the most significant edges where ( CN > MCI) or (MCI > CN), but I am not very sure how to analyze it. I put all subjects 600 in one folder. I have tried many experiments with varying thresholds from (2.5 to 4.18), sometimes I get a set of only subnetworks, sometimes 2 or 3. I have attached my design matrix. Could you please verify this is correct?

Summary of my questions.

1) Is the design matrix correct for the task I am trying to do?

2) What does it mean if the tool dNBS outputs one, two, or three subnetworks? How could we analyze them?

3) How to measure the edges with CN > MCI or MCI > CN.

Thank you for your valuable time.

More info: In the design matrix, the first 360 are MCI subjects, and the next 240 are Normal controls.

Contrast: [-1,1]

signification 0.05

component size Extent

permutations 10k

statistical test t-test

*Nanjing University of Science and Technology*

Got it. Thanks a lot.

*Nanjing University of Science and Technology*

Hi Andrew,

I read the articles and manuals on NBS, but I need some clarification. Here, you stated the [-1 1] and [1 -1] to test for MCI>CN and MCI<CN. Here, the greater sign is towards -1. However, in the NBS manual 1.2, it is stated like:

Design Matrix:

0 1

0 1

0 1

1 0

1 0

1 0

"Given the above design matrix, the following contrast vector specifies a onesided t-test assessing whether the group modeled by the first column is greater than the group modeled by the second column Example contrast [1 -1] with group1 > group 2. Here, the greater sign is towards 1.

Does direct NBS infer the greater smaller direction differently, or it has something to do with design matrix?

Thank you for your valuable time and kind consideration.

Best regards,

Saqib

Hi Saqib,

For the design matrix that you have given, [-1 1] will test whether the group modelled by the first column is LESS THAN the group modelled by the second column. I hope that helps.

Andrew

Originally *posted by Saqib Mamoon:*

Hi Andrew,

I read the articles and manuals on NBS, but I need some clarification. Here, you stated the [-1 1] and [1 -1] to test for MCI>CN and MCI<CN. Here, the greater sign is towards -1. However, in the NBS manual 1.2, it is stated like:

Design Matrix:

0 1

0 1

0 1

1 0

1 0

1 0

"Given the above design matrix, the following contrast vector specifies a onesided t-test assessing whether the group modeled by the first column is greater than the group modeled by the second column Example contrast [1 -1] with group1 > group 2. Here, the greater sign is towards 1.

Does direct NBS infer the greater smaller direction differently, or it has something to do with design matrix?

Thank you for your valuable time and kind consideration.

Best regards,

Saqib

*Nanjing University of Science and Technology*

Thanks a lot for the clarification. Understood.

Regards,

Saqib