Motivation

  • test hypothesis
    • test and idea or model, find out how things work
  • search for facts
    • study causal effects without having a model in mind first
  • whisper in the ears of princess
    • make suggestions for changes based on data
    • test small & cheap interventions to decide which one to scale
  • get right what others got wrong
    • disprove a wrong/weak work
    • gather further evidence for an insight

Rules

  • no deception
    • unless you are in marketing
  • real financial incentives, whenever possible
    • sometimes impossible, too expensive, not ethically OK

Terminology

Important

  • control: the change between 2 groups
    • e.g. heels vs no heels
  • factor levels:
    • 2 levels: heels vs no heels
    • 3 levels: <20, 20<22, >22
    • n levels
  • full factorial design
    • testing all combinations of factor levels
      • no theoretical limitation, but a practical limitation
  • between-subject vs within-subject design
    • between: each treatment on different subjects
    • within: each subject gets different treatments

Tradeoffs

  • possibility to study Interaction Effects
    • how do different independent variables interact with one another
      • are heels more effective in summer than in winter?
  • cost/feasibility
    • cannot add too many independent variables

Repetition

  • one-shot vs repeated game