help
help > RE: Multiple primary thresholds correction
Nov 17, 2021 10:11 PM | Andrew Zalesky
RE: Multiple primary thresholds correction
Hi Peder,
yes - testing a range of thresholds technically creates a multiple comparisons problem. However, if only a few different thresholds are tested and these are reported in a paper, I would argue that multiple comparisons correction is potentially overkill.
You may want to select a single threshold that corresponds to the minimum meaningful effect size in your experiment.
If you are determined to correct for multiple comparisons across thresholds, FDR is a good option. A more principled approach would be to use NBS-TFCE, which does not require a primary threshold choice, and thus overcomes this problem.
best,
Andrew
Originally posted by Peder Lillebostad:
yes - testing a range of thresholds technically creates a multiple comparisons problem. However, if only a few different thresholds are tested and these are reported in a paper, I would argue that multiple comparisons correction is potentially overkill.
You may want to select a single threshold that corresponds to the minimum meaningful effect size in your experiment.
If you are determined to correct for multiple comparisons across thresholds, FDR is a good option. A more principled approach would be to use NBS-TFCE, which does not require a primary threshold choice, and thus overcomes this problem.
best,
Andrew
Originally posted by Peder Lillebostad:
Hello NBS experts,
In the manual you recommend to test a range of primary thresholds. If I'm not mistaken, this brings about another problem of multiple testing, and ought to be corrected for.
But here is the problem: doing an FDR correction would be too conservative, because the values are not independent for obvious reasons. Does there exist a clever solution to this?
Thanks a lot,
Peder
In the manual you recommend to test a range of primary thresholds. If I'm not mistaken, this brings about another problem of multiple testing, and ought to be corrected for.
But here is the problem: doing an FDR correction would be too conservative, because the values are not independent for obvious reasons. Does there exist a clever solution to this?
Thanks a lot,
Peder
Threaded View
| Title | Author | Date |
|---|---|---|
| Peder Lillebostad | Nov 17, 2021 | |
| Andrew Zalesky | Nov 17, 2021 | |
| Peder Lillebostad | Nov 18, 2021 | |
| Andrew Zalesky | Nov 18, 2021 | |