Download (PDF, Unknown)

Problems that occur early in experiments

ethical and legal issues

a few key ethical and legal questions that: philosophers, lawyers, and scientists have raised about both experimentation and randomization:

• the ethics of experimentation on human beings
• the ethics of withholding a potentially effective treatment from control or comparison participants
• the ethics of random assignment compared with alternatives such as assignment based on need
• the conditions under which experiments might be discontinued for ethical reasons
• some legal problems that bear on experiments

Cites the Nazi medical experiments as well as the U.S. Tuskegee syphillis study

three sources of help to avoid ethical problems in experimentation

1. ethical codes
2. informed consent
3. institutional review boards

Belmont Report

1. respect for persons
2. beneficence
3. justice

pros and cons of withholding treatment in experimentation

imposing undesirable treatments

ethics of random assignment
the case for assignment by need is strongest when a treatment is known to be the most effective in meeting that need

regression discontinuity design can yield unbiased estimates of effects when assignment is made by need or merit, though less powerfully than the randomized experiment and with its own implementation issues.

some say randomization is ethical only if the conditions being compared may be therapeutically equivalent and if no better treatment exists. The ethical principle of autonomy contained in the Belmont Report requires informed consent so participants can judge for themselves the relevance and weight of risks.

Other arguments versus randomization deal with practical realities.

Physicians' Health Study Research Group study on aspirin had an effect that was large enough to end the study early on grounds that it was unethical to withhold such an effective treatment from controls. For reasons like this, preliminary analysis of early results at fixed intervals is advised.

Of course, ending a study early may prevent researchers from seeing longer-term effects.

Legal guarantees of confidentiality
researchers should consider using research procedures that can ensure confidentiality, such as using randomized response methods or determining not to gather any data about potential identifiers, and,  just as important, they should ensure that such procedures are followed.

Federal Judicial Center Advisory Committee on Experimentation
in the Law (Federal Judicial Center, 1981)

• present conditions need improvement
• proposed improvement is of unclear value
• only an experiement could provide the necessary data to clarify the question
• the results of the experiment would be used to change the practice or policy
• the rights of individuals would be protected in the experiment

recruiting

To ensure that they can be located, experimenters can use:

1. preexperimental surveys to locate and characterize potential participants,
2. pipeline studies to follow what happens to them over time,
3. pilot tests of the solicitation procedures to see who will learn about the experiment and who will attend if eligible,
4. trained outreach specialists skilled in communicating and generating interest about a program,
5. intake specialists who aggressively recruit potential participants, and (
6. efforts to learn what features of the intervention cause people to decline to enroll, such as lack of child care or finding the training to be in a career that they do not ultimately want to pursue

If the number of eligible participants is lower than anticipated despite thesebefforts, the researcher can

1. extend the time frame for the experiment if time and resources are available and if program staff are willing;
2. divert additional resources to intensified outreach efforts;
3. alter eligibility requirements so that more participants are eligible, though this may require testing interactions between eligibility requirements and treatment;
4. reduce the proportion assigned to treatment if power analyses suggest it would still be feasible to find an effect; or
5. terminate the experiment, which is sometimes better than spending funds on an experiment that cannot find the expected treatment effect, especially if the decision is made early in the life of the experiment.

implementation of random assignment

Simple random assignment

These procedures have considerable public relations value
when the decision that random assignment makes is sensitive and public, but two key practical drawbacks:

1. physical structure of random assignment mechanism (2 sides of a coin, six sides of a die)
2. biased by systematic experimenter behavior

The main alternative is to use a table of random numbers. Another alternative is to use a computer to generate one's own list of random
numbers. The general principle in using any of these methods is to make all the allocation decisions as random as possible.

It is not a "failure of randomization" if some observed means are significantly different across conditions at pretest after random assignment. Indeed, from sampling theory we expect that the fewer the number of participants assigned to conditions, the larger are the differences in pretest means that may be found.

Matching and Stratifying
Units can be matched or stratified before being randomly assigned to conditions. First match participants on a relevant variable and then randomly assigning from these matches. stratifying into five strata is often sufficient to remove over 90% of the bias due to the stratification variable. Both matching and stratifying greatly increase the likelihood that conditions will have similar pretest means and variances on the matching/stratifying variable
and on any variables correlated with it.

Matching and Analysis of Covariance
The analysis of covariance (ANCOVA) can be used as an alternative or as a complement to matching or stratifying