Clinical Research Designs
Epidemiology and Biostatistics
Dennis C. Stevens, M.D.
I. Randomized Clinical Trial -
- True Experimental Design
- "Gold Standard" of Clinical Research
II. Epidemiologic Designs
A. Case-Control Study
B. Cohort Study
III. Quasi-Experimental Designs
A. Case Report
B. Case Series
C. Other
IV. Outcome Based Research
A. Economic Analysis
1. Cost/Benefit
2. Cost/Effectiveness
B. Quality of Life
C. Decision Analysis
D. Other
V. Survey Research
A. Questionnaires
B. Polls
C. Surveys
VI. Meta Analysis
- Pooled analysis of previous clinical trials
- Generally includes randomized controlled trials
VII. Survival Analysis
- Actuarial Analysis
- Cancer Studies
- e.g. Plot of hazard function curve
Clinical Trials
I. Definition: A prospective study comparing the effect and value of interventions
against a control in human subjects.
II. Essential features:
A. Subjects are followed forward in time - prospective
B. Employ one or more interventions - may be prophylactic, diagnostic,
therapeutic agent, devices, regimens or procedures
C. Must have a control group which must be similar to the intervention
group at baseline
- Phase III Clinical Trials - comparative trial
- Phase I - evaluates dose and toxicity
- Phase II - assesses drug activity
- The control group is selected to be as similar to the study group as is
possible in virtually all respects
D. Human Subjects
- concerns regarding subject safety.
- issues of research ethics and informed consent
E. The ideal clinical trial includes both the randomization of subjects and
blinding of subjects and care providers
1. Randomization allows for the equal allocation of potential confounders
and effect modifiers between the two study groups. These are factors
which are possibly unknown or unpredictable at the onset of the study
2. Blinding attempts to eliminate bias which might be introduced by either
the participating subject or care providers
- Single Blinding
- Double Blinding
III.Other Characteristics of clinical trials:
A. Need:
1. Most definitive method of determining whether an intervention has the
postulated effect
2. Uncertainty regarding diseases and their natural history and therapy and
its positive and negative effects
3. The consequences of not conducting appropriate clinical trial can be
serious and costly in the long run.
Examples:
- Use of CPAP in COPD patients
- Use of digitalis in congestive heart failure
- Uncontrolled use of high concentrations of O2 in premature neonates
B. Timing:
1. Must be performed before drugs or interventions have become part of
routine medical practice
2. Early in the development of new therapies, but
3. Only after sufficient knowledge is available concerning efficacy and safety
4. Should not be conducted if therapy will be outmoded before or shortly
After the trial has concluded
C. Ethics:
1. Three fundamental ethical principles regarding research:
a. Respect for Persons - individuals should be treated as autonomous.
Those with diminished autonomy need protection.
b. Beneficence -
- Secure the well being of the individual
- Benefit for society/class of patients
c. Justice - treat persons fairly. Equally share the risks and benefits.
2. Ethical Norms dictate that there should be:
a. Good research design
(1) Randomization
- may be a problem if the treatment is known (or perceived ) to be
superior to placebo
- trial may be unethical
(2) Placebo control
- problems of an acceptable placebo
- deprivation of treatment
(3) Monitoring of the trial
- how to handle available data as it accrues
- Safety monitoring committee
b. Competent investigators
c. Favorable balance of harm and benefit
- Welfare of the subject/physician's obligation to his patient
- Societal good
d. Informed consent - cannot always be obtained
e.g. minor (infants), comatose subjects, mentally incompetent,
prisoners, emergency procedures, pregnant women/fetus
e. Equitable selection of subjects
f. Compensation for research related injury
IV. Essential Components of a Clinical Trial:
Generally, these criteria may be applied to virtually all clinical investigations
A. Review of the scientific background for the study
- Previous animal investigations/laboratory work
- Preliminary human evidence from case reports or case series
(Phase I and II Trials)
B. Development of specific written hypothesis/hypotheses to be tested
- Random testing for statistical significance is unjustifiable (data dredging).
- By chance 1 in 20 comparisons will achieve p<.05
- Specific methods are available if multiple testing is to be used
C. Study Design
1. What is the basic study design?
a. Randomized/controlled trial
b. Non-randomized concurrent controlled study
c. Historical controls - non-randomized, non-concurrent
d. Crossover designs - subject serves as own control
e. Withdrawal studies - assesses response to withdrawal of intervention
or a reduction of dosage
f. Factorial design - assesses the response to more than one
intervention
2. Study population
a. Specific inclusion and exclusion criteria are necessary
b. Calculation of sample size - power calculations/curves
3. Planned statistical analysis/data needs
a. What is/are the dependent and independent variables?
How will these be measured/evaluated?
b. How will bias be controlled?
c. Are there specific effect modifiers and/or confounders which
need to be considered in the planning of the study and analysis
of the data?
d. What measurements are needed and how is the validity and
accuracy of the measure to be confirmed?
e. What are the preliminary plans for statistical analysis?
f. On the basis of sample size calculations can the study be
completed?
-Alpha error, beta error, effect size and estimate of variance
-What about the loss of subjects and how will this be considered in
the final analysis?
-If a significant difference exists between groups can it in fact be
demonstrated - does the study have adequate power?
g. How will attrition/loss to follow-up be handled?
h. Did the investigators employ the services of a biostatistician
or epidemiologist - check authors and acknowledgments
4. Enrollment of subjects
a. Informed consent
b. Assessment of eligibility
c. Baseline studies/examinations
d. Allocation to study group
- random allocation involves more than alternate assignment
- use of randomization scheme (random numbers)
- blocked randomization schemes are sometimes used to assure
equal distribution of study subjects between groups
5. Intervention
a. Description and schedule
b. Measures of compliance
c. Measures of the reliability/validity of the tools used
d. Criteria for termination of intervention
e. Criteria for withdrawal or loss of subject
6. Follow-up of subjects
7. Ascertainment of response variables
a. Training
b. Data collection
c. Data monitoring and quality control
d. Data analysis
e. Are there criteria to determine when the study should be terminated?
D. Organization
1. Participating investigators
2. Study administration
3. Data monitoring committee
4. Human subjects review
RESEARCH ERROR AND THREATS TO VALIDITY
I. Research Error - Two types:
A. Random error - handled with the use of statistical tests and methods
B. Systematic error - uncontrolled error which may change the results and/or
interpretation of research
C. Specific types of error:
1. Bias - any systematic error that results in an incorrect estimate of the
association between exposure (intervention) and the risk of disease
e.g. selection bias, recall bias, lead time bias
2. Confounding - when the effect of the exposure (intervention) upon disease
is altered by some other unaccounted for factor
a. e.g. in a study of the effect of exercise on the occurrence of coronary
artery disease, age could be a confounder
b. Confounding may be adjusted for in the study design or in the final
analysis of the data. Controlled by:
-Randomization-assures equal distribution of confounders between
study and control groups
-Restriction-subjects are restricted by the levels of a known
confounder
-Matching-potential confounding factors are equally distributed
between the study groups
-Stratification in the Analysis-risk estimates (RR) are computed
for the various levels of potential confounders
-Can control for confounding in the analysis of the data
c. Illustration:
E-------------------D E---------CF?-----------D
CF
NOT A CONFOUNDER
CONFOUNDER CF IS IN THE CAUSAL PATHWAY
3. Effect Modification - when the association between exposure
(intervention) and disease varies by the level of a third factor.
a. This represents an inconsistent distortion or nuisance effect.
b. Cannot adjust for effect modification
-can compare risk estimates by levels of the effect modifier, but
-cannot control for effect modification in the analysis
II. Validity- four types:
A. Internal Validity
Is there in fact a causal relationship between the experimental treatment
(Independent variable) and the observed effect (dependent variable)?
B. Construct Validity
of Cause - infers that the observed effect is attributable to the specific
experimental intervention and not other variables
of Effect - infers that the conceptual dependent variable is accurately
reflected by the dependent variable as operationalized and
measured
C. External Validity - could the observed effect be produced by in other
settings, with other populations, at other times...
D. Statistical Conclusion Validity - Are the conclusions reached justifiable on
statistical grounds?
E. Threats to Validity
1. Construct Validity of Cause
- Knowledge by subject that he/she is part of a study may cause an increase in the change
(Hawthorne effect).
- Single operationalization of the independent variable.
- Single measurement of the outcome.
Multiple measures improve strength of study
- Guessing the purpose of the study (hypothesis guessing).
- Subject apprehension.
(people like to be seen in a favorable light - act as they perceive they
should)
- Expectations of the experimenter.
- Weak treatment (small effect size)
- Implementation of treatment (Integrity)
2. Construct Validity of Effect
- Inadequate conceptual analysis of the variables/concepts studied.
- Small number of effects measured.
3. Internal Validity
- Co-occurring events (history) may cause the observed change.
- Things change on their own (maturation).
- Testing on multiple occasions may change the results.
- Instruments change.
- Regression to the Mean - extreme observations may be only random events.
- Selection Error (Bias) - control by randomization
- Loss of subjects before the end of the study - is there a reason for
losses/dropouts?.
- Interaction of above errors.
- Introduction of experimental treatment for all patients (compensatory equalization of treatment).
- Subjects who perceive that they are receiving a less desirable treatment
may work harder (compensatory rivalry).
- Subjects who perceive that they are receiving a less desirable treatment
give up effect (resentful demoralization).
4. External Validity
- Treatment does not generalize to other situation.
- Treatment does not generalize to other population.
- Treatment does not generalize to other settings.
- Treatment does not generalize to other time periods.
- One part of a treatment package cannot be generalized to other circumstances when used alone.
5. Statistical Conclusion Validity
- Low statistical power
- Violations of assumptions of statistical tests
- Multiple tests of significance
- Low reliability of measures
Introduction to Decision Analysis
I. Definition:
Decision Analysis is a mathematical tool designed to facilitate
complex (clinical) decisions in which many variables must be considered
simultaneously
II. Properties:
A. Systematic framework organizing all data relevant to the decision
B. Clearly define the relationship between possible courses of action
and their outcomes
C. Assigns numerical values to outcomes, simplifying comparisons
III.Steps in Decision Analysis:
A. Create a decision tree
1. Formulate the Decision problem
- Written statement
- Must be clearly stated
2. Identify Decision Alternatives
3. List the clinical outcomes of each decision alternative
4. Represent the sequence of events leading to the clinical outcomes by a series of chance nodes (circle) and decision nodes (box)
- Events at a chance node are unlimited, but must be mutually exclusive
and exhaustive
5. Choose time horizon for the problem
6. Determine the probability of each chance outcome
- may be from the past literature
- may be and educated guess from personal experience
7. Assign a value to each clinical outcome
- Assessment of individual value of the outcome
- Utility Assessment - Standard Reference Gamble technique
- Cost
- Length of Life
- May be a multidimensional valuation
- Quality Adjusted Life Years (QALY's) - Time trade-off method
- Cost /Benefit, Cost/Effectiveness
B. Calculate the Expected Utility of each decision alternative
C. Choose the decision alternative with the highest Expected Utility
D. Use Sensitivity Analysis to test the conclusions of the analysis NORMAL ÚÄ¿
IV. Example - will follow the above steps:
Return to the Biostats/Epidemiology Page