Omnibus Analysis
Quantifying Heterogeneity
Introduction
Before interpreting the Mean Effect Size, we must ask: Do the studies agree? If studies vary wildly (beyond sampling error), the Mean might be meaningless.
1. Visual Inspection (Forest Plot)
The Forest Plot shows the ES and CI for each study. * Overlap: If CIs overlap heavily, Homogeneity is likely. * No Overlap: If CIs are disjointed, Heterogeneity exists.
2. Detailed Interpretation of Omnibus Output
When you run summary(omn.1)$coef, here is what the output means:
| Term | Definition |
|---|---|
estimate |
\(\hat{\mu}\) = Overall Omnibus Effect Size (\(\beta_0\)). |
se |
Standard Error of the Omnibus ES. |
z |
\(z\)-value for the test of significance. |
ci.l / ci.u |
95% Confidence Interval. |
Pr(>|z|) |
The \(p\)-value for the summary effect. |
2.1 Model Fit Statistics
When you run summary(omn.1)$fit:
| Term | Definition |
|---|---|
QE |
\(Q\)-Error: Measure of error in the model. |
QEp |
p-value for Homogeneity (Low p = Heterogeneous). |
QM |
\(Q\)-Model: Measure of model fit. |
I2 |
Percentage of variation due to True Heterogeneity. |
3. Cochran’s \(Q\)
A statistical test for heterogeneity. \[ Q = \sum W_i (ES_i - \bar{ES}_{FE})^2 \] * Weighted sum of squared differences. * Distributed as \(\chi^2\) statistic with \(df = k - 1\). * Problem: Significance depends on \(k\). Low power if few studies; too sensitive if many studies.
3. The \(I^2\) Statistic
Describes the Percentage of variation due to True Heterogeneity (rather than chance). \[ I^2 = \frac{Q - df}{Q} \times 100\% \]
Benchmarks: * \(25\%\): Low Heterogeneity. * \(50\%\): Moderate. * \(75\%\): High.
Interpretation: “75% of the observed variance in these studies is real; only 25% is noise.”
4. Publication Bias
The “File Drawer Problem”. Studies with null results (\(p > .05\)) are less likely to be published. This biases the meta-analysis upwards.
Detection
- Funnel Plot: Scatterplot of ES vs Precision. Asymmetry suggests bias.
- Egger’s Test: Statistical test for funnel plot asymmetry.
Correction
- Trim and Fill: Imputes missing “negative” studies to balance the funnel.