Addendum: The Meaning of Regression

STATISTICS FOR BRAIN AND COGNITIVE SCIENCES

9.07

Lecture 1

Addendum: The Meaning of Regression

Emery N. Brown

December 7, 2016

Bivariate Gaussian Density The Meaning of Regression

Bivariate Gaussian (Joint) Density of (X, Y)

f x y( , ) 1 ₁  2 _{x y} (1 ²)²

1 (x _x)² (y_y)2 2(x_x)(y_y)

exp{  ₂ [ ₂  ₂  ]}

2(1  )  _y  _y  _x y Pr( A B)

Conditional Density of Y Given X _{Pr(B A| ) }

Pr( A )

( , ) f x y ( | ) f y x

f y ( )

Conditional Expectation: Theoretical Regression Line

_y ( |  x) _y

 ^(x_x) E Y X

x

Conditional Variance

( |  x)   Var Y X _{y xy y}

The Geometry of the Regression Line

The Meaning of Regression

Conditional Expectation: Theoretical Regression Line

_y

( | x)   (x )

E Y X   _y  _x

_x

The Empirical Regression Line Variance

ˆ_y ˆ   ˆ (xx) y y 

ˆ_x y ˆ

  (xx)

ˆ ˆ

y _x 

   x

  ˆ ˆx

Y

X

Front (Y) and Back (X) MEG Sensor Background Noise Recordings

The Meaning of Regression

The Meaning of Regression

The geneticist Sir Francis Galton (1822-1911) observed that the sons of fathers who were taller than average tended to be shorter than average and that the sons of fathers who were shorter than average tended to be taller than average. He termed the phenomenon “regression towards mediocrity”.

More recently it has been termed “regression to the mean”.

This relation is given exactly by the regression line.

uu

ˆ 68.7 y 

Heights of 1078 Pairs of Fathers and Sons

major axis

x  67.7

 ^ˆ

 2.74 y  68.7

 ^ˆ

 2.81

 ^ˆ  0.5

 0.5  2.81 ( x  67.7) 2.74

x  72   y ˆ 70.9

Rice (2007)

x  64   y ˆ 66.8

Summary

Simple regression is our most basic technique for relating one variable to another.

It is useful to be able to think of simple regression in terms of likelihood, method-of-moments and least squares

analyses.

The geometry of the Pythagorean relations carries over to the case of multiple regression.

The concepts discussed here form the basis for our

MIT OpenCourseWare https://ocw.mit.edu

9.07 Statistics for Brain and Cognitive Science

Fall 2016

For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.