Factors Threatening Internal and External Validity YouTube Lecture Handouts

Get top class preparation for IAS right from your home: Get complete video lectures from top expert with unlimited validity: cover entire syllabus, expected topics, in full detail- anytime and anywhere & ask your doubts to top experts.

Factors Threatening Internal & External Validity | Standardized Tests & Research

Factors Threatening Validity


Internal and External Validity
  • Internal validity refers specifically to whether an experimental treatment/condition makes a difference or not, and whether there is sufficient evidence to support the claim.
  • External validity refers to the generalizability of the treatment/condition outcomes.

Factors Threatening Internal Validity

Factors Threatening Internal Validity
  • History – the specific events that occur between the first and second measurement.
  • Maturation – the processes within subjects, which act as a function of the passage of time. i.e.. if the project lasts a few years, most participants may improve their performance regardless of treatment.
  • Testing – the effects of taking a test on the outcomes of taking a second test.
  • Instrumentation – the changes in the instrument, observers, or scorers that may produce changes in outcomes.
  • Statistical regression – It is also known as regression to the mean. This threat is caused by the selection of subjects based on extreme scores or characteristics. Take worst students and guarantee that they will show immediate improvement right after my treatment.
  • Selection of subjects – the biases that may result in selection of comparison groups. Randomization (Random assignment) of group membership is a counter-attack against this threat. However, when the sample size is small, randomization may lead to Simpson Paradox, (in which a trend appears in several different groups of data but disappears or reverses when these groups are combined) .
  • Experimental mortality – the loss of subjects. For example, in a Web-based instruction project entitled Eruditio, it started with 161 subjects and only 95 of them completed the entire module. Those who stayed in the project all the way to end may be more motivated to learn and thus achieved higher performance.
  • Selection-maturation interaction – the selection of comparison groups and maturation interacting which may lead to confounding outcomes, and erroneous interpretation that the treatment caused the effect.
  • John Henry effect – John Henry was a worker who outperformed a machine under an experimental setting because he was aware that his performance was compared with that of a machine.

Factors Threatening External Validity

Factors Threatening External Validity
  • Reactive or interaction effect of testing – a pretest might increase or decrease a subject՚s sensitivity or responsiveness to the experimental variable.
  • Interaction effects of selection biases and the experimental variable
  • Reactive effects of experimental arrangements – it is difficult to generalize to non-experimental settings if the effect was attributable to the experimental arrangement of the research.
  • Multiple treatment interference – as multiple treatments are given to the same subjects, it is difficult to control for the effects of prior treatments

Developed by: