Shadish, Cook, Campbell: TABLE 4.3 Threats to Validity in Case-Control Studies

In Uncategorized

TABLE 4.3 Threats to Validity in Case-Control Studies

1.1n reading up on the field·

a.The biases of rhetoric Any of severa! techniques used to corw!nce te appealirg to reason.

b,The all's well literature bias. Scientific or professional societies !T'ay publish  reports or editorials that omit or play down controversies or disparatresults.

c. One-sided reference bias. Authors may restrict their references to only those works. that support their position: a literature review with a single starting point risks confinement  to a single side of the lssue.

d. Positive results  bia:s. Authors are more  likely to submit, and Nitors accept, positive than null results.

e. Hot stuff bias. When a topic lS hot, neither investigators nor editors may be able to resist the temptation to publish additional r :Sults, no matter how preliminary or shaky.

2. In specifying and selecting the study sample:

a. Popularity bias, The admission of patients to some practices, institutions, or procedures (surgery, autopsy)  is influenced  by the interest shrred up by the presenting conditions and its possible causes.

b. Centripetal bias. The reputations of certain  diniciar1s and institutions cause individuals with specific disorders or exposures to gravitate toward them.

c. Referral fitter  bias. As a group of ill are referred from primary to secondary to tertiary care, the concentra1!on of rare causes, multiple diagnoses, and "hopeless cases" may Increase.

d. Diagnostic access bias. Individuals differ tn their geographic, temporal, and economic access to the diagnostic procedures that label them as having a given disNse.

e. Diagnostic .suspicion  bia:s. A knoWredge ot the subject's prior exposure to a putative cause (ethnicity, tak:ing a certain drug, having a second disorder, being exposed in .:ln epidemic) may influence both the intensity and the outcome of the diagnostic process.

t,Unmasking (detection signal} bias. An innocent exposure  may bec.ome suspect it, rather

than causing a disease, it causes a sign or symptom that precipitates a search for the disease.

g. MimicJY bias. An innocent exposure may become suspect if, rather than causing a dis£-ase,

it causes a (benign} disorder that resembles th(' disease.

h. Previous opinion bias. The tactics and results of a previous diagnostic process on a patient, if known, rray affect the tactics and results of a subsequent diagnostic process on the same patient.

i. Wrong sample size bia.'t. Samples that are too small can prove nothing; samples that are

too large can prove anythir.g.

j.Admission rate (Berkson) bias. If  hospitalization rates differ for different exposure/disease groups, the relation betw'een exposure afld disease wi!! become distorted in hospital-based


k. Prevalence-incidence (Neyman) bias, A late look: at those exposed (or affected) Ndy wl!!

rniss fatal ar.d other short episodes, plus mild or "silent" cases ar.d cases in which evidence of exposure disappears  'Niih disease onset

I. Diagnostic vogue bias. Tr.e sarre illness rr:ay receive different diagnostic !abels at different

points in space or time

m. Diagnostic purity bias. Wher. "pure" diagnostic groups exdude co-morbidity, they may

become non-representative.






























































TABLE 4.3 Continued



n. Procedure selection bias, Certain dinica! procedures may be preferentially offered to those who are poor risks.

a. Missing clinical data bias. Missing clinical data rilay be missing because they are norma!,

negative, never measured, or measured but never recorded.

p. Non-contempOldneovs control bias. Secular changes in definitions, exposures, diagn05is, dise.Jses, and treatments may render non-<ontempo aneous

q. Starting time bias. The failure to identify a common starting time for exposure or illness

may lead to systematic mjsdass!ficat!ons. ·

r. Unacceptable disease bias. VVhen d1sorders  are socially unacceptable (V.D., suicide,

Insanity), they tend to be undeHeported.

s. Migrator bias. Migrants may differ systematicalfrom those who stay home.

t. Membership bias. Membership ln a group (the employed, joggers, etc) may imply a degree

of health that differs systematically from that of the general population.

u.MJn--respondent bia:s. Non-respondents (or "late comers") from a specified sample may

Fixhibit exposures or outcomes that differ from those of respondents (or "early comers").

v. Volunteer bias. Volunteers or "early comers" from a specified sample may exhibit

exposures or outcomes (they tend to be healthier} that differ from those of non-volunteers

or "late comers."

3. In executing the experimental manoeuvre {or exposure):

a. Contamination  bla$, ln an expenment when members of the control group inadvertently

receive the experimental manoeuvre, the difference in outcomes between experimental and control patients may be systematically uced.

b.Withdrav.raf bias. Patients who are withdra'Ml from an experiment may differ systematically

from those who remain.

c Compliance bias, Jn experiments requiring patient adherence to therapy, issues of efficacy

become confounded with those of compliance.

d. Thf!rapeutic peBonality bias. When treatment is not "blind," the therapist's convictions about efficacy may systematically  influence both outcomes (positive personality) and their measurement (desire for positive results).

e. Bogus control bias. Whefl patients who are allocated to an experimental manoeuvre die or sicken before or during its administration and are omitted or re-allocated to the control group, the experimental manoeuvre wil! appear spuriously superior.

4. !n measuring exposures and outcomes:

a.Insensitive measure bias. When outcome measures are incapable of detecting dinica!

significant changes or differences, Type ll errors occur.

b. Underlying cause bias (rumination bias),  Cases. may ruminate about possible causes for their illnesses and thus exhibit diffl'!rent recall or prior exposuns

c. End digit

terminal digits v.rith an unusual freque-ncy.

d. Apprehension bias. Certain measures (pulse, blood pressure) may alter systematically from their usual levels If  the subject is apprehensive.

e. Unacceptabl'lity bias. Measurements that hurt, embarrass, or invade privacy may be

systematically refused or evaded.

f. Obsequiousness bias. Subjects may systematically a!ter questionnaire responses in the

direction they perr:elve desired by the- investigator.


TABLE 4.3 Continued



g. Expectation bias. Observers may systematically err in measurig

so that they concur with pr;or exp ctations.

h, Substitution gamEl. The substitution of a risk iactor that has not been cstabiJshed as cau al

for its associated  outcome.

L Family information bias. The flow o7 family information about exposure and illness is.

stimulated  by, and directed :o,  a new case in its midst.

j. Exposure suspicion bias. A knowledge of the subject's  disease status  may influence bo:h the intrms;ry ard  outcome of a search for exposure 10 the  putative cause.

k. Reca!f bias.  Que-stions  aboL.t specific  exposures fllay be asked severatimes  of cases but

only once ocontrols. (See also the underlying cause bias.)

!.Attention bias. S1udy subje:cts  may sy5tematic.a;ly alter their behavior when they know they

are being observed.

m_ Instrument bias. Defects Jn  the calibration or maintenane

may le-ad to systematic deviat,on from true values.

5.1n analyzing  the data:

a.Post-hoc significance bias. VVhen decisio;-1 levels or "tails" for x a!"1d fJ are selected after t e

data rave been examined, conclusions may be biased,

b.Data dredging bias (looking for rhe pony). When data are reviewed for aU  possible associations without prior hypothcs:s, the rcs lts

act1vifes o"ly.

c5ca/e degradation bias. The degradation  and collapsing of measurement scales te ds obscurt: differences between groups under compwison.

d. Ttd}ring-up bias. rhe exc!Jsion of outliers or other unt:dy resu:ts cannot be justified on

statistical grounds and may ,ead to bias,

c. Repeated peeks bias. Repeated peeks al accumulating data in a random1zcd uial annat dependent afld may h:od to inappropriate termination.

6.1n interpreting the analysis:

a,Misraken identity bias, !n compliance trial,

patient's COIT'piiance rnay, instead  or in addition, cause the treating clinician to prescribe

more vigorously; the effect uoon achievement of the troatme;,t goal may be misinterpreted.

h, Cognitive dissonance bias. The be:ief in a given mecPa.nrsrn may increase rather than decrease in the face of contradictory  evidence.

c. Magnitude bias. In intemret1ng a finding, the selection of a s::aie of mea5ure111ent m.:Jy markedly affecthe int.C>rpretatlon.

d.Significance bias. The conksion  of .statistltalsig::if1cance, on the one hand, witT- biology  m

din:ca! or heolth care significance, on the other ha:" d, om lead -to fruitless  stt.:dtes and

useless conclusions.

e, Correlation bias. Equating correlat;on with causation leads to errors of both k;nds.

f.Under-exhaustion bias. The failure to exhaust the hypothesis space may lead to

authoritarian rather than authoritative interpretation.