Introduction to Research Design and Statistics

Research Validity

Internal Validity

Internal Validity -- A research study or experiment has internal validity if the outcome is a function of the variables that are measured, controled or manipulated in the study. Did in fact the experimental treatments make a difference in this specific instance?

Artifact -- If the results of a research study or experiment are due to some factor or factors outside of the study then the results are said to be an artifact.

Threat Definition Associated Terms

History Unanticipated events occurring while the experiment is in progress. counfounding, teacher effect
Maturation processes within the subjects operating as a function of time.
Testing The effect of taking one test upon the scores of a subsequent test. pretest/posttest, practice effect
Instrumentation An effect due to changes in a measuring instrument or changes in observers or scorers. instrument drift, fatigue effect
Statistical
regression An effect operating where subjects selected on the basis of extreme scores regress toward the mean of that variable.
Differential
selection
of subjects Biases resulting from selection or creation of groups that are not equivalent. random sampling, random assignment, intact groups
Experimental
mortality The differential loss of subjects from one or more groups on a nonrandom basis.
Selection-maturation
interaction Nonequivalent groups of different ages creating a bias such that selection and maturation interact. interactive combinations of factors
Expectancy A bias caused by the expectations of either the experimenter or the subjects or both. experminter bias, demand characteristics

External Validity

External Validity -- A research study or experiment has external validity if the results obtained would apply to other similar programs or approaches.

Generalizability -- The degree to which the results of a research study or experiment can be generalized to other groups, settings or situations.

Threat Definition Associated Terms

Reactive effect
of testing Pretest interacts with the treatment resulting in an effect that will not generalize. sensitization
Interaction effects
of selection biases
and the experimental
treatment An effect of some selection factor of intact groups interacting with the treatment that would not have occurred in randomly formed groups.
Reactive effects
of experimental
arrangements An effect due to subjects knowing that they are participating in an experiment. Hawthorne effect
Multiple
treatment
interference In subjects receiving multiple treatments there may be carry-over effects between treatments such that the results cannot be genearalized to single treatments.

Adapted from: Campbell, Donald T. & Stanley, Julian C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally & Company.

Intro Home Page

Phil Ender, 30Jun98

Threat	Definition	Associated Terms
History	Unanticipated events occurring while the experiment is in progress.	counfounding, teacher effect
Maturation	processes within the subjects operating as a function of time.
Testing	The effect of taking one test upon the scores of a subsequent test.	pretest/posttest, practice effect
Instrumentation	An effect due to changes in a measuring instrument or changes in observers or scorers.	instrument drift, fatigue effect
Statistical regression	An effect operating where subjects selected on the basis of extreme scores regress toward the mean of that variable.
Differential selection of subjects	Biases resulting from selection or creation of groups that are not equivalent.	random sampling, random assignment, intact groups
Experimental mortality	The differential loss of subjects from one or more groups on a nonrandom basis.
Selection-maturation interaction	Nonequivalent groups of different ages creating a bias such that selection and maturation interact.	interactive combinations of factors
Expectancy	A bias caused by the expectations of either the experimenter or the subjects or both.	experminter bias, demand characteristics