Correcting for Publication Bias in the Presence of Covariates

Publisher Marketing: Meta-analysis has become increasingly popular in the last decade and is now in the top position on most proposed hierarchies of evidence. Meta-analyses are also the most cited study design in the health sciences literature. The traditional role for meta-analysis has been to compile information from diverse studies on the same topic, thus increasing power. However, there is increasing recognition of the challenges that heterogeneity presents in data synthesis. It is important to quantify, assess, and potentially interpret heterogeneity and try to distinguish between genuine between-study heterogeneity and biases. Meta-analytic studies provide a very useful tool for sensitizing researchers, physicians, and public health practitioners to the almost ubiquitous presence of biases in research. The focus in the present report is on one such bias, publication bias, which has been suggested as the most important threat to the validity of meta-analyses. The trim and fill method was developed by Duval and Tweedie to address the issue of publication bias in meta-analysis, and it is now widely used in practice in many areas of public health research. It relies on scrutinizing a funnel plot for asymmetry, assumed to be a manifestation of publication bias (see below). The goal of this research was to extend the trim and fill method to those situations where covariates are available as possible explanations for some of the funnel plot asymmetry. Publication bias is the term used for the bias that may occur when the research on a particular topic does not include the whole population of studies that has been performed. The danger to interpretation under these conditions is that the wrong conclusions may be drawn if the available studies differ systematically from the results of all the research that has been done. Publication bias is a phenomenon that runs counter to the way in which the scientific method has developed over the past century. One of the key historical contributions of statistical thinking has been a move away from a context where possible random observations were acceptable, to one where only those results that are statistically significant (i.e., not due to chance alone) are seen as being established and worth consideration. It is commonly believed that studies are not uniformly likely to be published in scientific journals. Easterbrook et al. suggested that statistical significance is a major determining factor of publication. Some researchers may not submit a nonsignificant result for publication, and editors may fail to publish nonsignificant results even if they are submitted. Therefore, there may be a nonrepresentative proportion of significant studies in the scientific literature. This becomes problematic for a meta-analysis in which data come solely from the published literature, potentially leading to a nonrepresentative proportion of significant studies in the meta-analysis dataset. A standard meta-analysis will then result in a conclusion biased toward significance. This is not just an academic problem: it can ultimately influence the clinical decisions of medical practitioners and public health officials. As Glass noted in 1976, journals that directly or indirectly influence medical practice cannot afford to ignore this problem. He claims, "The potential for publication bias concerns us because physicians now will search systematic reviews to determine the best treatment for patients. If positive results get published more, then the risk of adopting ineffective and even harmful medical practices is greater."