Proof of Correlation Coefficient

Proof of Linear Regression Theory

Given: $r_{XY}=\frac{Cov(X,Y)}{{\sigma }_X{\sigma }_Y}$

Let’s break this down further:

Covariance and Standard Deviations

$Cov(X,Y)=E[(X-{\mu }_X)(Y-{\mu }_Y)]$ ${\sigma }_X=\sqrt{E[(X-{\mu }_X{)}^2]}$ ${\sigma }_Y=\sqrt{E[(Y-{\mu }_Y{)}^2]}$

Linear Relationship

For a perfect linear relationship between (X) and (Y), we can write: $Y=a+bX$

Expected Values

${\mu }_Y=E[Y]=a+bE[X]=a+b{\mu }_X$

Substituting Y

$Cov(X,Y)=E[(X-{\mu }_X)(Y-{\mu }_Y)]$ Substitute $Y=a+bX$ and ${\mu }_Y=a+b{\mu }_X$: $Cov(X,Y)=E[(X-{\mu }_X)(bX+a-b{\mu }_X-a)]$ Simplify the expression inside the expectation: $Cov(X,Y)=E[(X-{\mu }_X)(b(X-{\mu }_X))]$ $Cov(X,Y)=bE[(X-{\mu }_X{)}^2]$ $Cov(X,Y)=b{\sigma }_X^2$

Correlation Coefficient

Substitute $Cov(X,Y)=b{\sigma }_X^2$ into $r_{XY}$: $r_{XY}=\frac{b{\sigma }_X^2}{{\sigma }_X{\sigma }_Y}$ Simplify: $r_{XY}=\frac{b{\sigma }_X}{{\sigma }_Y}$

Standard Deviations Relationship

From the relationship $Y=a+bX$: ${\sigma }_Y^2=E[(Y-{\mu }_Y{)}^2]=E[(b(X-{\mu }_X){)}^2]=b^{2}E[(X-{\mu }_X{)}^2]=b^2{\sigma }_X^2$ ${\sigma }_Y=|b|{\sigma }_X$

Final Expression

Substitute ${\sigma }_Y=|b|{\sigma }_X$ into $r_{XY}$: $r_{XY}=\frac{b{\sigma }_X}{|b|{\sigma }_X}$ Simplify: $r_{XY}=\frac{b}{|b|}$

Thus, the value of $\frac{b}{|b|}$ is either $1$ if $b$ is positive, or $-1$ if $b$ is negative.

Therefore: $r_{XY}=\frac{b}{|b|}\to -1\text{ or }1$

This shows that for a perfect linear relationship between (X) and (Y), the correlation coefficient $r_{XY}$ is (-1) or (1), indicating a perfect negative or positive linear relationship, respectively.