Welcome
Syllabus
Schedule
Assignments
Theory
Research
Practice
Communication (Links to Calendar,
Class Email and Forums)
WEEK
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
Useful
Additional
Resources

Threats to Internal Validity in Quantitative Studies

Threat What does this mean? What kind of study is most likely to be effected?

History

 

An event that occurs during a study that can effect the responses of the participants. It could be something the participants due like start exercising all on their own or it could be an event such as an article in the newspaper or national publicity.

Longitudinal. A study with repeated measures.

Maturation

 

The participants get older, wiser, more depressed, more hungry, more tired. They change all on their own! Longitudinal., studies where the participants are more likely to change such as adolescents, infants or people who are quite ill.

Testing

Sometimes the effect being measured changes because of the number of times the participant is tested. The test itself may add to the person's knowledge or change their attitudes, hence the test becomes part of the intervention! Studies with repeated measures or a pre-test-post test design.

Instrumentation

This means changes in the instrument during the study. It can also mean the data collectors get better or worse at what they are doing. To me this always seems like a reliability issue, but it is always listed under internal validity.

Physiologic instruments or those in which researchers are collecting the data in person.

Statistical regression In general, participants who score really high or really low on a test or a questionnaire will have a more moderate score the next time they are tested. This is called regressing towards the mean. Pretest-posttest designs. A study with a small sample size will have more of an effect since with less participants, one person's scores has a greater effect on the average score.
Selection Pre-existing differences between the participants selected for a study or those who volunteer for a study and those not in the study or differences between or among the study groups. Quasi-experimental studies and convenience samples. Doing statistical comparisons between groups and help address this threat.
Mortality or attrition No, this doesn't mean the sicker people die! Participants drop out of s study or are lost to follow-up. Longitudinal. studies. It helps to have several contact phone numbers or addresses for each participant. It is important to analyze the existing data to determine if there are differences between those who dropped out and those who continued. If more than 10% of participants are lost to follow-up this seriously effects generalizability.
Interaction effect One or more of these threats can combine and have have a compounding effect. The three most likely interactions are history, maturation and instrumentation.
Nonadherence/failure to complete protocol This in essence could water down the effect of the intervention

Studies with an intervention. Especially at risk would be longitudinal studies and studies with a complex intervention.

Diffusion of treatment "Contamination" of the no-treatment group by the treatment group. Somehow the no-treatment finds out what the treatment is and start doing it! It may also mean that the researchers change one of the intervention groups or control group to contain treatments not originally in their intervention. Studies with an intervention when the intervention groups and control groups are being studied at the same time in the same location (a clinic, small town). This means there is essentially no control group. I would consider this a fatal flaw! The trade off is the effects of history and selection bias.
Compensatory rivalry of the no-treatment group Participants know they are in the no-treatment group and so attempt to improve on their own. Good for them, but bad for the study! Intervention studies where the participants are aware which group they are in. It can also occur if the health care providers know the participants are in the no-treatment group so they give better care. Blinded and double blinded studies help.
Resentful-demoralization of the no-treatment group The opposite of above. The control group gives up because they aren't getting the treatment. The reverse of the above. above.

Note: One sited web site lists these as internal validity threats and they are usually included under threats to external validity.

 

Experimenter bias, Placebo Effect, Hawthorne Effect.

 

 

 

 

Jeanette Koshar RN, NP, PhD
Office: (707) 664-2649 | Office Hours: Wed 10-12, email and by appointment | Email: jeanette.koshar@sonoma.edu
Deb Kindy RN, PhD
Email: klaas@sonoma.edu | Office Hours: Tues 1-3