Correlation

Maths of Correlation

• covariance

  • Covariance
  • Covariance and Correlation
  • reminder: independent (unrelated) → covariance = 0
    • BUT covariance = 0 does not imply independence
  • estimation $\hat{\sigma }=\frac{1}{n-1}\sum _{i=1}^n(x_i-\bar{x})(y_i-\bar{y})$

• correlation coefficient (Karl Pearson)

  • for population: $p_{XY}=\frac{Cov(X,Y)}{{\sigma }_X{\sigma }_Y}$
  • for sample: $r_{XY}=\frac{\sum (x_i-\bar{x})(y_i-\bar{y})}{\sqrt{\sum (x_i-\bar{x}{)}^2\ast \sum (y_i-\bar{y}{)}^2}}$

• notation $S_{XY}=\sum (x_i-\bar{x})(y_i-\bar{y})=\sum x_i\ast y_i-\frac{\sum x_i\ast \sum y_i}{n}$ $S_{XX}=\sum (x_i-\bar{x}{)}^2=\sum x_i^2-\frac{(\sum x_i{)}^2}{n}$

  • $S$ … sum of squared standard deviations
  • $S$ is to be minimized in method of least squares

• equivalent representation $r_{XY}=\frac{\sum x_i{y}_i-n\bar{x}\bar{y}}{\sqrt{\sum (x_i^2-n{\bar{x}}^2)}\sqrt{\sum (y_i^2-n{\bar{y}}^2)}}=\frac{S_{XY}}{\sqrt{S_{XX}}\sqrt{S_{YY}}}$

• correlation is measure of linear relation

• correlation does not mean causality

• $-1\le r_{XY}\le 1$ … between -1 and 1

  • sign indicates direction of the correlation

• $|r_{XY}|$ shows the strength of the correlation

• $r_{XY}=r_{YX}$ … symmetrical

Proof of $Y=a+bX$

• given $Y=a+bX$
• short form: $r_{XY}=\frac{Cov(X,Y)}{{\sigma }_X{\sigma }_Y}=\frac{b}{|b|}\to -1\text{or}1$
• long from: Proof of Correlation Coefficient
• $r_{XY}$ is -1 or 1, depending on sign of $b$

Non-Linear Correlation

• if the data has non-linear correlation (e.g. quadratic, exponential) it is impossible to measure with $Y=a+bX$
  • but might be possible with e.g $Y=a+b{X}^2$
• a scatter plot is always helpful to understand the problem and choose the correct correlation function

Example Correlation from Table