International Journal of Medical Informatics
The use of a registry database in clinical trial design: Assessing the influence of entry criteria on statistical power and number of eligible patients
Introduction
Prospective empirical studies are commonly used to investigate the effect of a particular medical intervention on an outcome of interest in a specific patient population. In this paper we will focus on Randomized Controlled Trials (RCTs), although the method we present can also be applied for other types of studies. In an RCT the study population is randomly divided into two parts; in one part of the population the intervention is performed, whereas the other part of the population receives the treatment that is considered to be the standard or a placebo treatment. The goal of the RCT is to show whether there is a statistically significant difference in the outcome of interest between the two groups. However, many trials fail to show a significant effect. This is often interpreted as the intervention not having a significant effect on the outcome. However, detecting no significant difference can also be caused by the use of a study design that is not able to detect a statistically significant difference between the groups, even if it exists [1], or by including an insufficient number of patients. The design of an RCT is a delicate decision process. Decisions are made for example on whether or not to cluster physicians or sites, how to randomly assign patients to each of the arms of the trial, how to blind patients, physicians or both, etcetera. The goal of these decisions is to reduce the risk of failure to show an effect of the intervention. A fundamental concept in this stage of the trial is the statistical power that can be achieved (i.e., the probability to correctly reject the null hypothesis that the difference in outcome between the study groups equals zero), given the specified difference in outcome that is to be detected and the chosen level of significance [2].
One way to achieve a higher statistical power is to include more patients into the trial, but this induces higher costs and implies that the duration of the trial will be longer or the number of sites has to be increased. Therefore, in the decisions made in the study design of the trial both statistical and economic arguments are taken into account. The goal is to maximize statistical power given the constraints of the budget, or likewise, to minimize the costs to obtain a specific level of statistical power [3].
One of the important steps in the design of a clinical trial is the choice of the entry criteria. These criteria determine which patients are eligible for the trial and thereby influence the characteristics of the study population and the number of patients that are eligible. However, in the process of finding a balance between statistical power and costs, often the influence of the entry criteria on these two aspects is not quantified.
We believe that, given the considerable influence of entry criteria on size and characteristics of study populations, explicit estimates of the statistical power and number of eligible patients should be made in the process of choosing entry criteria. Especially in entry criteria that leave room for variation (e.g., the lower threshold for age to select patients for a trial in middle-aged men could vary between 50 and 55), the influence of different choices on statistical power is to be investigated.
This paper presents the projection method, a method that enables the trial designer to quantify the influence of the choice of entry criteria on the statistical power and on the expected sample size (and thereby on the expected duration of the trial) using a registry database. In the last decade registries have been set up in specific clinical domains or in relation to specific interventions to collect information on the process of care and outcome of the patients on a continuous basis during a long period of time, often with the aim to measure the quality of care [4]. The patients described in these registries represent a sample of the patient population within the specific clinical domain.
The core of the projection method consists of mimicking the patient selection process by applying the proposed entry criteria onto this patient sample.
In this paper we present the projection method and indicate how it is embedded in the process of trial design. The use of the method is illustrated with an example from the area of severe sepsis. The paper is structured as follows: Section 2 provides more background information on the choice of entry criteria and the calculation of the statistical power. In Section 3 we explain the use of the projection method and indicate how it can be used in the practice of clinical trial design. Section 4 illustrates the use of the projection method in the area of severe sepsis and in Section 5 we discuss the advantages and limitations of the projection method and relate our work to existing literature.
Section snippets
Background
This paper focuses on the influence of the choice of entry criteria on statistical power and number of eligible patients. This section provides background information on the use of entry criteria and on factors that play a role when choosing these criteria (Section 2.1) and describes the use and calculation of statistical power (Section 2.2).
The projection method
Nowadays in more and more clinical domains registries are being set up to monitor the process of care on a continuous basis [4]. An important reason to set up and maintain these registries is to obtain more information on the characteristics of the group of patients that is included into the registry and to measure and monitor quality of care provided to these patients. The information is collected prospectively and routinely as part of the clinical care process, and in many registries at
Choosing entry criteria for a severe sepsis trial
Sepsis is a quite common condition which can be cured when it is timely detected. However, if the disease progresses to severe sepsis, it can be life-threatening, as organ systems start to fail. In this stage of the disease treatment at the Intensive Care Unit is necessary to stabilize the function of the organ systems. Mortality among patients with such a severe form of sepsis is high, reported to be 20–50% [6]. Curative interventions are hardly available and treatment is very expensive. This
Discussion
This paper presents the projection method which enables the RCT designer to investigate the influence of the choice of entry criteria on statistical power, required sample size, and expected duration of the trial. In current practice in trial design these estimates are often made implicitly, based on related literature and experience. However, the patient populations from which these values are derived may be different from the population in which the future trial is to be conducted (e.g., with
Acknowledgements
The authors would like to thank Jeremy Wyatt for his valuable input in our discussion on the possibilities of using a registry database in choosing entry criteria. NICE/L.P. received an educational grant from Eli Lilly Netherlands BV. N.P. receives a grant from the Netherlands Organisation for Scientific Research (NWO) under project number 634.000.020.
References (17)
Balancing the number and size of sites: an economic approach to the optimal design of cluster samples
Control. Clin. Trials
(2003)- et al.
Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis
Chest
(1992) - et al.
Decision support for clinical trial design
Artif. Intell. Med.
(2003) - et al.
Absence of evidence is not evidence of absence
BMJ
(1995) Fundamentals of Biostatistics
(1990)- H. Reitsma, Registers in cardiovascular epidemiology. Ph.D. thesis, Faculty of Medicine, University of Amsterdam, 1999....
- et al.
Fundamentals of clinical trials
(1998) - et al.
Treating patients with severe sepsis
N. Engl. J. Med.
(1999)
Cited by (10)
Multicenter Clinical Trials in Sepsis: Understanding the Big Picture and Building a Successful Operation at Your Hospital
2011, Critical Care Nursing Clinics of North AmericaCitation Excerpt :The industry sponsor has two important goals that may in fact compete with each other. First is to enroll quality patients to enhance the chance of observing a treatment effect and the second is avoiding being so precise in trying to achieve this goal that enrollment is difficult and the study is prolonged.17 When a study is prolonged because of difficulty in enrolling, the fixed costs of sponsoring the study also is extended and may become untenable.
Automated matching software for clinical trials eligibility: Measuring efficiency and flexibility
2010, Contemporary Clinical TrialsCitation Excerpt :Further, low recruitment in underserved populations leads to the appearance of inequalities in access to the latest technology and treatments. [9,10]. Determining eligibility in every patient is critical to assuring adequate and unbiased enrollment rates and represents the first step in addressing the issue of under representation in clinical trials [11–13]. Despite this, not all eligible patients are evaluated or invited to participate in a clinical trial even though patients who are offered a trial are likely to participate [14].
Multicenter Clinical Trials in Sepsis: Understanding the Big Picture and Building a Successful Operation at Your Hospital
2009, Critical Care ClinicsCitation Excerpt :The industry sponsor has two important goals that may in fact compete with each other. First is to enroll quality patients to enhance the chance of observing a treatment effect and the second is avoiding being so precise in trying to achieve this goal that enrollment is difficult and the study is prolonged.17 When a study is prolonged because of difficulty in enrolling, the fixed costs of sponsoring the study also is extended and may become untenable.
An automated communication system in a contact registry for persons with rare diseases: Scalable tools for identifying and recruiting clinical research participants
2009, Contemporary Clinical TrialsCitation Excerpt :Disease registries have long been used to assist the clinical research process. Registries have been used to estimate achievable sample size for prospective clinical trials [20] and to explore research questions on incidence [21,22], screening [23,24], growth rates [23,25], co-morbidities [26–29], and mortality [23,30–32] in hundreds of diseases, including heart disease [33] and cystic fibrosis (in multiple countries) [26,30,32,34–36]. A major challenge in clinical trials is identifying potential research subjects.
Important issues in the design and reporting of clinical trials in severe sepsis and acute lung injury
2008, Journal of Critical CareCitation Excerpt :Acquiring the ability to personalize subject enrollment in a fashion that identifies patients who have biology that matches the proposed treatment is likely to be a major step forward in producing positive clinical trials, for example, enrolling septic patients with altered sublingual flow to receive a therapy targeting improvement in flow or administering antiendotoxin therapy to a patient testing positive for endotoxemia. The inability to enroll suitable patients that allow at least an opportunity for the new therapy to work has major negative effect on power calculation [8]. Clinical coordinating centers may assist in limiting some of the variables intrinsic to human sepsis studies and ensuring that the subject enrolled meets the true intent of the protocol [9].
Clinical trial registration-Now and thereafter
2008, Acta Anaesthesiologica Taiwanica