The use of a registry database in clinical trial design: Assessing the influence of entry criteria on statistical power and number of eligible patients

https://doi.org/10.1016/j.ijmedinf.2006.05.036Get rights and content

Abstract

Randomized clinical trials (RCTs) are prospective empirical studies used to investigate the effect of a particular medical intervention. The design of a clinical trial is a delicate decision process, as each of the decisions that are taken in this process influences the eventual result of the clinical trial. Despite the efforts that are put into trial design, many trials fail to show an effect of the intervention. In some of these situations the intervention may be truly ineffective, however, more often this is caused by problems with the inclusion of patients and a resulting lack of statistical power to show the effect. To avoid this problem, in the design of a trial, the statistical power that can be achieved with the current design choices is calculated and balanced with economic considerations. In the choice of the entry criteria however, an important step in the design process, the influence of the chosen criteria on statistical power and number of eligible patients is not quantified. As these criteria influence the characteristics of the study population and the number of patients that will be eligible for the trial, and thereby the chances of finding an effect of the intervention, we believe that also in the choice of entry criteria explicit estimates of the number of eligible patients should be made.

This paper presents a method to arrive at precise, objective estimates of statistical power and the number of eligible patients, using a registry database. Furthermore, we describe how this method is incorporated in the process of choosing entry criteria for a clinical trial. We illustrate the method with an example in the area of severe sepsis.

Introduction

Prospective empirical studies are commonly used to investigate the effect of a particular medical intervention on an outcome of interest in a specific patient population. In this paper we will focus on Randomized Controlled Trials (RCTs), although the method we present can also be applied for other types of studies. In an RCT the study population is randomly divided into two parts; in one part of the population the intervention is performed, whereas the other part of the population receives the treatment that is considered to be the standard or a placebo treatment. The goal of the RCT is to show whether there is a statistically significant difference in the outcome of interest between the two groups. However, many trials fail to show a significant effect. This is often interpreted as the intervention not having a significant effect on the outcome. However, detecting no significant difference can also be caused by the use of a study design that is not able to detect a statistically significant difference between the groups, even if it exists [1], or by including an insufficient number of patients. The design of an RCT is a delicate decision process. Decisions are made for example on whether or not to cluster physicians or sites, how to randomly assign patients to each of the arms of the trial, how to blind patients, physicians or both, etcetera. The goal of these decisions is to reduce the risk of failure to show an effect of the intervention. A fundamental concept in this stage of the trial is the statistical power that can be achieved (i.e., the probability to correctly reject the null hypothesis that the difference in outcome between the study groups equals zero), given the specified difference in outcome that is to be detected and the chosen level of significance [2].

One way to achieve a higher statistical power is to include more patients into the trial, but this induces higher costs and implies that the duration of the trial will be longer or the number of sites has to be increased. Therefore, in the decisions made in the study design of the trial both statistical and economic arguments are taken into account. The goal is to maximize statistical power given the constraints of the budget, or likewise, to minimize the costs to obtain a specific level of statistical power [3].

One of the important steps in the design of a clinical trial is the choice of the entry criteria. These criteria determine which patients are eligible for the trial and thereby influence the characteristics of the study population and the number of patients that are eligible. However, in the process of finding a balance between statistical power and costs, often the influence of the entry criteria on these two aspects is not quantified.

We believe that, given the considerable influence of entry criteria on size and characteristics of study populations, explicit estimates of the statistical power and number of eligible patients should be made in the process of choosing entry criteria. Especially in entry criteria that leave room for variation (e.g., the lower threshold for age to select patients for a trial in middle-aged men could vary between 50 and 55), the influence of different choices on statistical power is to be investigated.

This paper presents the projection method, a method that enables the trial designer to quantify the influence of the choice of entry criteria on the statistical power and on the expected sample size (and thereby on the expected duration of the trial) using a registry database. In the last decade registries have been set up in specific clinical domains or in relation to specific interventions to collect information on the process of care and outcome of the patients on a continuous basis during a long period of time, often with the aim to measure the quality of care [4]. The patients described in these registries represent a sample of the patient population within the specific clinical domain.

The core of the projection method consists of mimicking the patient selection process by applying the proposed entry criteria onto this patient sample.

In this paper we present the projection method and indicate how it is embedded in the process of trial design. The use of the method is illustrated with an example from the area of severe sepsis. The paper is structured as follows: Section 2 provides more background information on the choice of entry criteria and the calculation of the statistical power. In Section 3 we explain the use of the projection method and indicate how it can be used in the practice of clinical trial design. Section 4 illustrates the use of the projection method in the area of severe sepsis and in Section 5 we discuss the advantages and limitations of the projection method and relate our work to existing literature.

Section snippets

Background

This paper focuses on the influence of the choice of entry criteria on statistical power and number of eligible patients. This section provides background information on the use of entry criteria and on factors that play a role when choosing these criteria (Section 2.1) and describes the use and calculation of statistical power (Section 2.2).

The projection method

Nowadays in more and more clinical domains registries are being set up to monitor the process of care on a continuous basis [4]. An important reason to set up and maintain these registries is to obtain more information on the characteristics of the group of patients that is included into the registry and to measure and monitor quality of care provided to these patients. The information is collected prospectively and routinely as part of the clinical care process, and in many registries at

Choosing entry criteria for a severe sepsis trial

Sepsis is a quite common condition which can be cured when it is timely detected. However, if the disease progresses to severe sepsis, it can be life-threatening, as organ systems start to fail. In this stage of the disease treatment at the Intensive Care Unit is necessary to stabilize the function of the organ systems. Mortality among patients with such a severe form of sepsis is high, reported to be 20–50% [6]. Curative interventions are hardly available and treatment is very expensive. This

Discussion

This paper presents the projection method which enables the RCT designer to investigate the influence of the choice of entry criteria on statistical power, required sample size, and expected duration of the trial. In current practice in trial design these estimates are often made implicitly, based on related literature and experience. However, the patient populations from which these values are derived may be different from the population in which the future trial is to be conducted (e.g., with

Acknowledgements

The authors would like to thank Jeremy Wyatt for his valuable input in our discussion on the possibilities of using a registry database in choosing entry criteria. NICE/L.P. received an educational grant from Eli Lilly Netherlands BV. N.P. receives a grant from the Netherlands Organisation for Scientific Research (NWO) under project number 634.000.020.

References (17)

There are more references available in the full text version of this article.

Cited by (10)

  • Multicenter Clinical Trials in Sepsis: Understanding the Big Picture and Building a Successful Operation at Your Hospital

    2011, Critical Care Nursing Clinics of North America
    Citation Excerpt :

    The industry sponsor has two important goals that may in fact compete with each other. First is to enroll quality patients to enhance the chance of observing a treatment effect and the second is avoiding being so precise in trying to achieve this goal that enrollment is difficult and the study is prolonged.17 When a study is prolonged because of difficulty in enrolling, the fixed costs of sponsoring the study also is extended and may become untenable.

  • Automated matching software for clinical trials eligibility: Measuring efficiency and flexibility

    2010, Contemporary Clinical Trials
    Citation Excerpt :

    Further, low recruitment in underserved populations leads to the appearance of inequalities in access to the latest technology and treatments. [9,10]. Determining eligibility in every patient is critical to assuring adequate and unbiased enrollment rates and represents the first step in addressing the issue of under representation in clinical trials [11–13]. Despite this, not all eligible patients are evaluated or invited to participate in a clinical trial even though patients who are offered a trial are likely to participate [14].

  • Multicenter Clinical Trials in Sepsis: Understanding the Big Picture and Building a Successful Operation at Your Hospital

    2009, Critical Care Clinics
    Citation Excerpt :

    The industry sponsor has two important goals that may in fact compete with each other. First is to enroll quality patients to enhance the chance of observing a treatment effect and the second is avoiding being so precise in trying to achieve this goal that enrollment is difficult and the study is prolonged.17 When a study is prolonged because of difficulty in enrolling, the fixed costs of sponsoring the study also is extended and may become untenable.

  • An automated communication system in a contact registry for persons with rare diseases: Scalable tools for identifying and recruiting clinical research participants

    2009, Contemporary Clinical Trials
    Citation Excerpt :

    Disease registries have long been used to assist the clinical research process. Registries have been used to estimate achievable sample size for prospective clinical trials [20] and to explore research questions on incidence [21,22], screening [23,24], growth rates [23,25], co-morbidities [26–29], and mortality [23,30–32] in hundreds of diseases, including heart disease [33] and cystic fibrosis (in multiple countries) [26,30,32,34–36]. A major challenge in clinical trials is identifying potential research subjects.

  • Important issues in the design and reporting of clinical trials in severe sepsis and acute lung injury

    2008, Journal of Critical Care
    Citation Excerpt :

    Acquiring the ability to personalize subject enrollment in a fashion that identifies patients who have biology that matches the proposed treatment is likely to be a major step forward in producing positive clinical trials, for example, enrolling septic patients with altered sublingual flow to receive a therapy targeting improvement in flow or administering antiendotoxin therapy to a patient testing positive for endotoxemia. The inability to enroll suitable patients that allow at least an opportunity for the new therapy to work has major negative effect on power calculation [8]. Clinical coordinating centers may assist in limiting some of the variables intrinsic to human sepsis studies and ensuring that the subject enrolled meets the true intent of the protocol [9].

  • Clinical trial registration-Now and thereafter

    2008, Acta Anaesthesiologica Taiwanica
View all citing articles on Scopus
View full text