- Log In More
Log in via Institution
Log in via OpenAthens
Log in using your username and password
- Basket
- Search More
Advanced search
- Latest content
- Topic collections
- Archive
- Authors
- About
- Jobs
Advanced search
- CloseMore
Main menu
- Latest content
- Topic collections
- Archive
- Authors
- About
- Jobs
- Log in More
Log in via Institution
Log in via OpenAthens
Log in using your username and password
- BMJ Journals
You are here
- Home
- Archive
- Volume 3,Issue 1
- Assessing allocation concealment and blinding in randomised controlled trials: why bother?
Article Text
Article menu
- Article Text
- Article info
- Citation Tools
- Share
- Rapid Responses
- Article metrics
- Alerts
Assessing allocation concealment and blinding in randomised controlled trials: why bother?
Statistics from Altmetric.com
Request Permissions
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
The scientific community's quest for unbiased research received a strong boost from a recent policy amendment on randomised controlled trials (RCTs) in this journal. Henceforth, the status of allocation concealment will be clearly indicated in the abstracts along with that of blinding. Thus, readers will have additional information by which to judge the internal validity of trials. In this editorial I address the background and rationale for these enhancements.
Background
Random allocation to intervention groups remains the only method of ensuring that the groups being compared are on an equivalent footing at the outset of the study, thus eliminating selection and confounding biases. This has allowed RCTs to play a key part in advancing medical science.
The success of randomisation depends on 2 interrelated processes.1, 2 The first entails generating a sequence by which the participants in a trial are allocated to intervention groups. To ensure unpredictability of that allocation sequence, investigators should generate it by a random process. The second process, allocation concealment, shields those involved in a trial from knowing upcoming assignments in advance.3, 4 Without this protection, investigators and patients have been known to change who gets the next assignment, making the comparison groups less equivalent.5, 6
For example, suppose that an investigator creates an adequate allocation sequence using a random number table. However, the investigator then affixes the list of that sequence to a bulletin board, with no allocation concealment. Those responsible for admitting participants could ascertain the upcoming treatment allocations and then route participants with better prognoses to the experimental group and those with poorer prognoses to the control group, or vice versa. Bias would result. Inadequate allocation concealment also exists, for example, when assignment to groups depends on whether a participant's hospital number is odd/even, or depends on translucent envelopes that allow discernment of assignments when held to a light bulb.
Allocation concealment should not be confused with blinding. Allocation concealment concentrates on preventing selection and confounding biases, safeguards the assignment sequence before and until allocation, and can always be successfully implemented.1, 2 By comparison, blinding concentrates on preventing study personnel and participants from determining the group to which participants have been assigned (which leads to ascertainment bias), safeguards the sequence after allocation, and cannot always be implemented.1, 7
Reporting of methods
Investigators must not only minimise bias but also must communicate those efforts to the reader. Readers should not have to assume or guess the methods employed. Yet assessments of the reporting quality of published trials have consistently found major flaws.3, 8–14 Only 9% of trials in the specialist journals and 15% in the general journals reported both an adequate method of generating random sequences and an adequate method of allocation concealment.3, 8, 15 Of trials reported as double blind, only 45% described similarity of the treatment and control regimens and only 26% provided information on the protection of the allocation schedule.16 Most reports simply provide no information on methods.
With so little relevant information, many of us resort to inappropriate markers of trial quality. Two noteworthy examples illustrate that concern. Firstly, many designate a trial as high quality if it is “double blind,” as if double blinding is the sine qua non of an RCT. Although double blinding can reflect good methods, it is not the sole criterion of quality. As I shall discuss later, adequate allocation concealment actually appears to be the more important indicator. Moreover, many trials cannot be double blinded. Those trials must be judged on other merits and not on an inapplicable standard based on double blinding.
Secondly, some assume that a good quality trial contains arms of equal size while a poor quality trial contains unequal sizes. That standard applies only when the investigators used a restricted randomisation generation scheme that aimed for equality. In fact, exactly equal numbers in treatment groups may mean that some process other than randomisation was used, for example allocation of every second patient to the intervention group or manipulation of assignment to treatment groups to force equality. A simple randomisation method will infrequently yield exactly equal sample sizes. Indeed, expecting equal treatment group sizes in trials is inconsistent with simple randomisation.
Although RCT reporting remains weak, it is improving. Methodologists, editors, and clinicians addressed the prevailing flaws in reporting by publishing The Consolidated Standards of Reporting Trials (CONSORT) statement.17 Already over 70 journals have adopted the standards including such high profile general medical journals as JAMA, Lancet, BMJ, and Annals of Internal Medicine. Yet, even with improvement, readers of RCTs should beware of the information provided in many current trial reports.
Empirical evidence of bias
Recent studies have shown that poor RCTs and poorly reported RCTs yield biased results. For example, in a study of 250 controlled trials from 33 meta-analyses in pregnancy and childbirth, investigators found that alleged RCTs with inadequate and unclear allocation concealment yielded larger estimates of treatment effects (41% and 33%, respectively on average) than trials in which authors reported adequate concealment.5 Investigators found similar results for trials in digestive diseases, circulatory diseases, mental health, and stroke.18 Those trials that used inadequate or unclear allocation concealment, compared with those that used adequate concealment, yielded 37% larger estimates of effect on average.
These exaggerated estimates of treatment effects reveal meaningful levels of bias. If a study is designed to detect a decrease in mortality of 25% or 50% from a particular treatment, biases of 30–40% would overwhelm estimates of the treatment effect. The elimination of bias is crucial in trials designed to detect moderate effects.
Double blinding also appears to reduce bias. Trials that were not double blinded yielded larger estimates of treatment effects compared with trials in which authors reported double blinding (odds ratios exaggerated, on average, by 17%).5 Another recent analysis has also noted the importance of double blinding.19 Although double blinding appears to prevent bias, its effect appears weaker than allocation concealment. Indeed, Moher et al found little impact from double blinding.18
Summary
As users of RCT results, we must understand the potential for humans to interject bias. By including assessments of allocation concealment and double blinding, abstracts in this journal will help readers to discern those trials that have made superior efforts to minimise bias. Judging the quality of allocation concealment and blinding reflects current empirical research and demonstrates the commitment of the editors of this journal to applying the principles of evidence-based medicine to the practice of reporting.
References
- ↵
Schulz KF. Subverting randomization in controlled trials. JAMA 1995;274:1456–8.
OpenUrlCrossRefPubMedWeb of Science
- ↵
Schulz KF. Randomized trials, human nature, and reporting guidelines. Lancet 1996;348:596–8.
OpenUrlCrossRefPubMedWeb of Science
- ↵
Schulz KF, Chalmers I, Grimes DA, et al. Assessing the quality of randomization from reports of controlled trials published in obstetrics and gynecology journals. JAMA 1994;272:125–8.
OpenUrlCrossRefPubMedWeb of Science
- ↵
Chalmers TC, Levin H, Sacks HS, et al. Meta-analysis of clinical trials as a scientific discipline. I: Control of bias and comparison with large co-operative trials. Stat Med 1987;6:315–25.
OpenUrlPubMedWeb of Science
- ↵
Schulz KF, Chalmers I, Hayes RJ, et al. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408–12.
OpenUrlCrossRefPubMedWeb of Science
- ↵
Poco*ck SJ. Statistical aspects of clinical trial design. Statistician 1982;31:1–18.
- ↵
Schulz KF. Unbiased research and the human spirit: the challenges of randomized controlled trials. CMAJ 1995;153:783–6.
- ↵
Altman DG, Doré CJ. Randomization and baseline comparisons in clinical trials. Lancet 1990;335:149–53.
OpenUrlCrossRefPubMedWeb of Science
Moher D, Fortin P, Jadad AR, et al. Assessing the completeness of reporting of trials published in languages other than English: implications for the conduct and reporting of systematic reviews. Lancet 1996;347:363–6.
OpenUrlCrossRefPubMedWeb of Science
Williams DH, Davis CE. Reporting of assignment methods in clinical trials. Controlled Clin Trials 1994;15:294–8.
OpenUrlCrossRefPubMedWeb of Science
Sonis J, Joines J. The quality of clinical trials published in the journal of family practice. J Fam Pract 1994;39:225–35.
OpenUrlPubMedWeb of Science
Mosteller F, Gilbert JP, McPeek B. Reporting standards and research strategies for controlled trials: agenda for the editor. Controlled Clin Trials 1980;1:37–58.
DerSimonian R, Charette LJ, McPeek B, et al Reporting on methods in clinical trials. N Engl J Med 1982;306:1332–7.
OpenUrlCrossRefPubMedWeb of Science
- ↵
Tyson JE, Furzan JA, Reisch JS, et al. An evaluation of the quality of therapeutic studies in perinatal medicine. J Pediatr 1983;102:10–3.
OpenUrlCrossRefPubMedWeb of Science
- ↵
Schulz KF, Chalmers I, Altman DG, et al. The methodologic quality of randomization as assessed from reports of trials in specialist and general medical journals [article]. Online J Curr Clin Trials [serial online] 1995 August 26;4(doc no 197);[81 paragraphs].
- ↵
Schulz KF, Grimes DA, Altman DG, et al. Blinding and exclusions after allocation in randomised controlled trials: survey of published parallel group trials in obstetrics and gynaecology. BMJ 1996;312:742–4.
OpenUrlAbstract/FREE Full Text
- ↵
Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA 1996;276:637–9.
OpenUrlCrossRefPubMedWeb of Science
- ↵
Moher D, Pham B, Jones A, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998;352:609–3.
OpenUrlCrossRefPubMedWeb of Science
- ↵
Khan KS, Daya S, Collins JA, et al. Empirical evidence of bias in infertility research: overestimation of treatment effect in crossover trials using pregnancy as the outcome measure. Fertil Steril 1996;65:939–45.
OpenUrlPubMedWeb of Science
Coordinating Editor's note
Although we are happy to take credit for incorporating more information on blinding and concealment in our abstracts, the credit for stimulating us to do so belongs elsewhere. Firstly, Ken Schulz and others have shown us that randomisation, blinding, and concealment of allocation make a difference to the accuracy of trial reports. Secondly, Phillip Devereaux and others have taken us to task for failing to report these important features of clinical trials. Readers will find that abstracts of trials now include whether the randomisation was concealed from those responsible for entering patients into trials, and who was blinded to treatment allocation during the trial. This information will be provided, under the “design” heading in abstracts, whenever it is possible to obtain from the report of the study or, failing that, directly from the investigators.
Unfortunately, our experience to date shows that it is not always possible to acquire an unequivocal answer from authors about blinding or allocation concealment. For example, the trial may be billed as “double blind” but the regimen appears to have adverse effects that might reveal to patients or investigators who was taking it, and authors did not check to see if the blinding was maintained. Or the investigator indicates using sealed envelopes to conceal randomisation but there was no indication that the envelopes were opaque. If, in our judgment, there is reason to suspect that blinding or concealment were not secure, we will indicate that by rating the appropriate feature as “unclear.” In doing so, we do not mean to offend investigators who have done their best to protect their trials from bias; rather, we wish to protect readers from us—the editors—conveying a sense of false security about studies where we remain uncertain about what was done about concealment or blinding.
The definitions that we will use for the categories of allocation concealment and blinding can be found in the glossary (inside back cover).
Read the full text or download the PDF:
Subscribe
Log in via Institution
Log in via OpenAthens