Sampling and Experiments

IMS, Ch. 2

Prof Randi Garcia

2026-02-02

Sampling

Null distribution

Simple random sampling

http://en.wikipedia.org/wiki/Simple_random_sampling

Stratified sampling

http://en.wikipedia.org/wiki/Stratified_sampling

Cluster sampling

http://en.wikipedia.org/wiki/Cluster_sampling

Group problem set, page 1

Experiments

Confounding

  • Confounding
    • variable is correlated with both the response and explanatory variable
  • Common response
    • variable causes change in both the response and explanatory variables

Causality

  • correlation does not imply causation!
  • Be especially wary of time as a confounder
  • Causality generally cannot be inferred from observational data – only from randomized, controlled experiments.
  • Causal inference is one of the hottest fields in statistics!

https://twitter.com/causalkathy

Causal diagram (generic)

  • How does the treatment affect the outcome?

Causal diagram (example)

  • How does distance affect the number of tickets sold (\(Flights\))?

Key terms in experimental design

  • Controlling
    • eliminating any differences except the explanatory variable (e.g. using a control group to simulate absence of the treatment)
  • Randomization
    • create equivalent groups by random assignment
  • Blocking
    • dividing groups proactively to compare like with like
  • Blinding and double-blinding
    • prevent bias on the part of researchers and patients

Reproducibility vs. replicability

  • Replication
    • a study performed multiple times should get the same results!
  • How are these different?
    • replication: different people get the same results with different data
    • reproducibility: different people get the same results with the same data

Group problem set, page 2