Reliability
- precision of measurements
- reliability = absence of measurement error
Measure Reliability
- Test-Retest method
- split-half method
- inter-class correlation
Validity
- do we measure what we want to measure
- validity = fit between underlying construct and the measure
Internal vs External Validity
- internal … truth in the study
- are we doing the right thing in our experiment?
- external … truth in real life
- can we generalize the findings to the whole population?
- usually a tradeoff between internal and external validity
Threats
- internal
- failure to randomize
- failure to follow treatment protocol → IV Regression
- attrition (dropping out of experiment)
- Hawthorne effect
- external
- non-representative sample (cannot infer from sample to population)
- general equilibrium effects (factors/interaction which are not included in the model)
- manna from heaven (earned vs given money)
Visual
