Calculating Effect Sizes
The Mathematics of Standardization
Why Standardize?
In meta-analysis, we combine results from different scales (e.g., BDI ranges 0-63, HAM-D ranges 0-52). We need a Common Metric. The most common is the Standardized Mean Difference (SMD).
1. Choosing the Right Effect Size
The choice of Effect Size depends on your research question and variables:
| IV Type | DV Type | Statistical Test | Effect Size Metric |
|---|---|---|---|
| Continuous | Continuous | Correlation / Regression | Correlation (\(\hat{\rho}\) or \(r\)) |
| Dichotomous | Continuous | ANOVA / \(t\)-test | Std Mean Difference (\(\hat{\delta}\) or \(d\)) |
| Dichotomous | Dichotomous | Chi-Square (\(\chi^2\)) | Odds Ratio (\(OR\)) |
Note: * IV = Independent Variable (Predictor) * DV = Dependent Variable (Outcome)
2. Standardized Mean Difference (Cohen’s \(d\))
The standardized mean difference (\(\delta\)) quantifies the improvement in the Treatment group relative to the Control group, standardized by the pooled standard deviation.
2.1 The Formula (Formula 12.11)
\[ d = \frac{M_{Tx} - M_{Ctrl}}{S_{within}} \]
2.2 Pooled Standard Deviation (Formula 12.12)
When sample sizes differ (\(n_1 \neq n_2\)), we weight the variance: \[ S_{within} = \sqrt{\frac{(n_1-1)SD_1^2 + (n_2-1)SD_2^2}{n_1 + n_2 - 2}} \]
2.3 Hedges’ \(g\) Correction (Formula 12.15)
To correct for bias in small samples: \[ J = 1 - \frac{3}{4(df) - 1} \] \[ g = d \times J \]
2.4 Variance of \(d\) (Formula 12.13)
\[ V_d = \frac{n_1+n_2}{n_1 n_2} + \frac{d^2}{2(n_1+n_2)} \] This formula shows that variance depends on Sample Size and the magnitude of the effect (\(d^2\)).
3. Variance & Standard Error
We weight studies by their Variance (\(V_g\)). Small variance = High Precision = High Weight.
\[ V_d = \frac{n_1+n_2}{n_1 n_2} + \frac{d^2}{2(n_1+n_2)} \quad (Formula\ 12.13) \]
\[ V_g = J^2 \times V_d \quad (Formula\ 12.17) \]
\[ SE_g = \sqrt{V_g} \]
4. Building Your Own Function
Before using a package, let’s write a function to calculate \(d\) and \(g\) manually. This mirrors the logic found in compute.es.
5. Interactive Calculation
Do not rely on manual math. Use this tool to standardizing your data.
5. The compute.es Package
For complex conversions (e.g., from \(F\), \(t\), \(\chi^2\), or \(p\)-values), use the package.
Try It: Imputing Effect Sizes
What if a study only reports a sample size of 50 (25 per group) and a p-value of 0.04?