Dear all,

I have some questions about the bias analysis of SDM-PSI. 

After performing the extraction steps with SDM-PSI software, I used the R program to test the heterogeneity and bias of the extracted data of each imputation. 

The SDM-PSI and R procedures have consistent findings in exploring heterogeneity, although the specific values are different. However, for the issue of bias, the two methods yielded opposite conclusions, with SDM yielding insignificant p and no bias. However, the R procedure obtained p<0.0001 and there was bias.

I used the R program for Egger's test by importing the extracted data for each iteration (if there are 10 studies and 50 iterations, there are 500 studies) with standard errors as regressors. However, I observed that the report after bias analysis using SDM showed only 8 degrees of freedom instead of 498.

I don't know if I'm using the R program to calculate the bias in the wrong way or what's causing this, hoping for help!

I attached the results obtained by both methods below.

Best!

Hi,

I experienced a similar problem.

I think the problem comes from the use of different dispersion metrics to study precision.
SDM Extract bottom on SDM gives you effect size and variance for each study.
Using this formula I obtained Standard Error [https://www.bmj.com/about-bmj/resources-readers/publications/statistics-square-one/5-differences-between-means-type-i-an] but it should be noticed that it applies when the two means come from the same population.

Using "Bias Test" on SDM I obtain low heterogeneity (I²=3.8%), and no publication bias.

Using R (package meta and metagen function) I performed ROI analyses looking at heterogeneity, publication bias, and overall effect size (with significance value which is not reported in SDM).

When I use Standard Error I get a high heterogeneity (I²=67.7%) and the triangle of my funnel plot is relatively narrow.
When I use Standard Deviation I found low heterogeneity (I² = 0.0%) and the triangle is enlarged (more simililar to the one plot in SDM while performing the Bias Test). In both cases the Egger's test is non significant but the statistics values have small changes.

But most importantly the significance of the overall effect changes as well (I get a p-value of 0.0016 when using Standard Error instead of 0.0857 when using Standard Deviation).

As suggested by Sterne and Egger (DOI: 10.1016/s0895-4356(01)00377-8) "Standard error is likely to be the best choice for the vertical axis [of the funnel plot]".

Heterogeneity index and publication bias inspection are important checks to interpret the results of a meta-analysis (and are usually asked by the reviewers). I am therefore confused regarding whether I should state that in my data there is high or low heterogeneity (as well as regarding which representation and metrics I should use for the funnel plot and the Egger's test).
Maybe SDM developers could clarify this point and give further insight regarding the best metrics that should be used?

Thank you in advance (and thanks for developing such a useful software!)
Best,
Anna