sdm-help-list > Compare Effect Sizes/SDM Estimates
Showing 1-4 of 4 posts
Display:
Results per page:
Aug 23, 2019  06:08 PM | Jack Rogers - University of Birmingham
Compare Effect Sizes/SDM Estimates
Dear Experts, 

I should preface this query by saying that we used Seed-based d Mapping to carry out our analysis, and not the recently listed SDM-PSI version 6.12 (great see this new version released). 

In a subgroup analysis we demonstrate that the results replicate the findings from the whole group analysis, but with larger effect sizes (SDM estimates) for the subgroup analyses (as might be expected). We would like to use this pattern of results to suggest that the subgroup analysis is essentially 'driving' the findings from the whole group analysis. However, we worry about the validity of this claim without accounting for whether the effect sizes for the subgroup findings significantly differ (or not) from the effect sizes reported in the whole group analysis. Is it appropriate to directly compare the effect sizes/SDM estimates  from these two analyses statistically in this way to see how they differ in a more quantitative manner?

In a not unrelated point, we are also wondering about how best to calculate statistical power for our meta-analysis within the context of the SDM framework (i.e. using the effect sizes/SDM estimates?), as has been described previously for off-line analysis (link below). 

https://towardsdatascience.com/how-to-ca...

Many thanks for your help with this matter. I am conscious of the fact that these queries border on the empirical and not just the methodological. 
Kind regards, 
Jack
Aug 27, 2019  03:08 PM | Joaquim Radua
RE: Compare Effect Sizes/SDM Estimates
Dear Dr Rogers,

I think that maybe you could compare the studies included in the subgroup analysis with the studies not included in the subgroup analysis. To do so, you could extract the values of the peak and plot and/or describe how they are similar or different for the two groups. If the number of studies is large enough, you could statistically compare the two groups using a linear model in SDM.

Re: statistical power, I am indeed working on it but the analyses are still ongoing...

With the best wishes,

Joaquim
Aug 28, 2019  04:08 PM | STEPHANE DE BRITO - University of Birmingham
RE: Compare Effect Sizes/SDM Estimates
  
Originally posted by Joaquim Radua:
Dear Dr Rogers,

I think that maybe you could compare the studies included in the subgroup analysis with the studies not included in the subgroup analysis. To do so, you could extract the values of the peak and plot and/or describe how they are similar or different for the two groups. If the number of studies is large enough, you could statistically compare the two groups using a linear model in SDM.

Re: statistical power, I am indeed working on it but the analyses are still ongoing...

With the best wishes,

Joaquim
Dear Joaquim,

Thank you for your reply to my colleague Jack and the suggestions - much appreciated.

If I may, as an estimate for the power of our meta-analysis, could we take the highest SDM effect size we obtained from our main meta-analysis, the average number of partcipant per group accross all the studies, and the number of effect sizes as inputs for this excel calculator: https://osf.io/juzfg/ also described here at the bottom of the page: https://towardsdatascience.com/how-to-calculate-statistical-power-for-your-meta-analysis-e108ee586ae8

I have attached an exemple for you to consider. If I understand correctly the output, it suggests that our meta-analysis would have 90% power to detect a medium effect size in the context of moderate heterogenity.

Thank you in advance for your comments on this.

Best wishes,

Stephane
Aug 28, 2019  04:08 PM | STEPHANE DE BRITO - University of Birmingham
RE: Compare Effect Sizes/SDM Estimates
Dear Joaquim,

The main reason why I am pushing you a bit on this issue of power is because our manuscript has been rejected several times on the ground that our meta-analysis is small because we only included 8 eligible group comparisons studies with a total sample size of 185 men in the clinical group and 164 control men. We nevertheless obtain 3 effects that are conserved across all the possible study combinations in our jacknife analysis, which identifies them as reliable effects for the first time in this small body of literature.

Best wishes,

Stephane