||What does this mean?
||What kind of study
is most likely to effected?
|Small sample size
||Bigger is not always better, but you want enough people in the study so you feel comfortable believing they represent other people like them.
||Convenience samples where there are a smallish (< 30) number of participants.
Reactive effects of testing
|The study results may not be
generalizable to people who didn't undergo a pretest.
This would be related to the internal validity threat
of testing. That alone may have raised awareness.
Any study with a pre-test. For example if you were in a study related to nutrition then a pre-test might be what you had eaten in the last couple of days. The "pre-test" might have reminded you to eat healthier before you even started the study!
Interaction effects of selection
and the independent variable
The effects of the internal
threats of selection bias and receiving the treatment
may compound each other making it difficult to discern
how much is treatment and how much is the participants'
Refusal rates: This is the percentage of people who
agreed to be in the study as opposed to those asked
to be in the study. The researcher needs to report the
refusal rate. Anything less than 80% makes the generalizability
Studies that require a major
time commitment or inconvenience to the participants.
Those who chose to participate may for example, be hardier,
more gullible or have more time on their hands than
the general population! Also if choosing participants
was difficult and there was a high refusal rate than
the participants may be different.
Studies requiring a big time commitment by the participants
even if it is only a one shot deal. It helps if the
researcher has collected some basic demographics on
those who refused and then comparisons can be run between
the groups to see if they are different or similar.
Also if researchers get a low response rate, they need
to asked themselves "why has this occurred?"
Is the questionnaire too long? Is it a subject of little
interest to the population being studied?
Interaction effects of setting and the
Where the study took place may effect
how well the intervention worked. For example some hospitals
or clinics or schools might be more willing to allow
nurse researchers to conduct studies and the types of
patients going to those hospitals may be different than
hospitals less willing to participate in studies. Think
UCSF and Stanford!
The trade-off is if the study is done in the same hospital and the same time there is a major risk of contamination (an internal validity threat)!
This would be another plug for replication studies.
The same study design could be sued with different populations.
Studies related to the same phenomenon
which used similar types of hospitals.
In a study I conducted with pregnant adolescents and
their intention to continue in school after the birth
of their children, I used schools specifically for pregnant
and parenting teens. I had sound reasons for doing this
decision, but it meant study results could only be generalized
to girls in similar schools.
Interaction of history and the independent
|When the study took place is always a consideration
and results can't be generalized across time. State of
the art equipment changes as do the understandings about
adverse physiologic conditions. I am such a long time
nurse that I can remember when we treated all 9 pound
babies as term babies! Oops! Also think of the political
climate. Women who had abortions 35 years ago might have
very different outcomes then women having abortions today.
Studies in which the intervention is
likely to have changed or the targeted population
may have changed. Studies related to domestic violence
conducted 20 years ago often included only poor
women in shelters. Now there is a bit more diversity
in the targeted population.
Again, replication studies would be beneficial!
Reactive effects of experimental
|Participants may react to being
in a study and that alone can influence the study results.
Factors such as novelty, the Hawthorne effect (people
change their behavior because they are in a study) or
the placebo effect are included.
Studying participants over
a short period of time. Of course this has trade-offs.
The longer the study goes on, the more likely people
will return to their previous behaviors when the novelty
has worn off. This could be a factor in the internal
threat of statistical regression.
Also participants in an experimental study may be followed much more closely then people would be in general (think adherence to medication regimes)