+ - 0:00:00
Notes for current slide
Notes for next slide

Gauss Markov Theorem

Part 2

Dr. D’Agostino McGowan

1 / 9

Gauss Markov Theorem

  • Least Squares is the Best Linear Unbiased Estimator (BLUE)
2 / 9

Gauss Markov Theorem

  • Least Squares is the Best Linear Unbiased Estimator (BLUE)

What does this mean?

2 / 9

Gauss Markov Theorem

  • Least Squares is the Best Linear Unbiased Estimator (BLUE)

What does this mean?

  • Best: has the smallest variance
2 / 9

Gauss Markov Theorem

  • Least Squares is the Best Linear Unbiased Estimator (BLUE)

What does this mean?

  • Best: has the smallest variance
  • Linear: it is linear in the observed output variables
2 / 9

Gauss Markov Theorem

  • Least Squares is the Best Linear Unbiased Estimator (BLUE)

What does this mean?

  • Best: has the smallest variance
  • Linear: it is linear in the observed output variables
  • Unbiased: it is unbiased
2 / 9

Gauss Markov Theorem

  • Least Squares is the Best Linear Unbiased Estimator (BLUE)

What does this mean?

  • Best: has the smallest variance
  • Linear: it is linear in the observed output variables
  • Unbiased: it is unbiased
  • Estimator
2 / 9

Gauss Markov Theorem

  • Least Squares is the Best Linear Unbiased Estimator (BLUE)

What does this mean?

  • Best: has the smallest variance
  • Linear: it is linear in the observed output variables
  • Unbiased: it is unbiased
  • Estimator

Let's prove best now.

2 / 9

What do we need to do?

☝️ Come up with another linear unbiased estimator of β, let's call it β~ ✌️ Show that this estimator has a variance that is no smaller than var(β^|X)

What is the var(β^|X)?

3 / 9

☝️ A linear, unbiased estimator of β

  • β~=Cy
4 / 9

☝️ A linear, unbiased estimator of β

  • β~=Cy

Why Cy? This is linear.

4 / 9

☝️ A linear, unbiased estimator of β

  • β~=Cy

Why Cy? This is linear.

  • C=(XTX)1XT+D
4 / 9

☝️ A linear, unbiased estimator of β

When is β~ unbiased?

5 / 9

☝️ A linear, unbiased estimator of β

When is β~ unbiased?

Show that E[Cy|X]=β

5 / 9

☝️ A linear, unbiased estimator of β

When is β~ unbiased?

Show that E[Cy|X]=β E[Cy|X]=E[((XTX)1XT+D)(Xβ+ϵ)|X]

5 / 9

☝️ A linear, unbiased estimator of β

When is β~ unbiased?

Show that E[Cy|X]=β E[Cy|X]=E[((XTX)1XT+D)(Xβ+ϵ)|X]

Try It

Solve for E[Cy|X].

02:00
5 / 9

☝️ A linear, unbiased estimator of β

When is β~ unbiased?

E[β~|X]=E[((XTX)1XT+D)(Xβ+ϵ)|X]=DXβ+β

6 / 9

☝️ A linear, unbiased estimator of β

When is β~ unbiased?

E[β~|X]=E[((XTX)1XT+D)(Xβ+ϵ)|X]=DXβ+β

We need E[Cy|X]=β

  • For β~ to be unbiased DX must be 0.
6 / 9

Now let's calculate the variance

For our estimate β^ to be best what needs to be true?

7 / 9

Now let's calculate the variance

For our estimate β^ to be best what needs to be true?

var(β^|X)<var(β~|X)

7 / 9

Now let's calculate the variance

For our estimate β^ to be best what needs to be true?

var(β^|X)<var(β~|X) var(β~|X)=var(Cy|X)

7 / 9

Now let's calculate the variance

For our estimate β^ to be best what needs to be true?

var(β^|X)<var(β~|X) var(β~|X)=var(Cy|X)

What is constant?

7 / 9

Now let's calculate the variance

For our estimate β^ to be best what needs to be true?

var(β^|X)<var(β~|X)

var(β~|X)=var(Cy|X)=Cvar(y|X)CT

Try It

Finish solving for var[Cy|X]. Remember DX=0 for this to be unbiased.

03:00
8 / 9

Now let's calculate the variance

For our estimate β^ to be best what needs to be true?

var(β^|X)<var(β~|X)

var(β~|X)=var(Cy|X)=Cvar(y|X)CT=var(β^)+σ2DDT

9 / 9

Now let's calculate the variance

For our estimate β^ to be best what needs to be true?

var(β^|X)<var(β~|X)

var(β~|X)=var(Cy|X)=Cvar(y|X)CT=var(β^)+σ2DDT

  • DDT is a positive semidefinite matrix, therefore

var(β^) is always <var(β~)

9 / 9

Gauss Markov Theorem

  • Least Squares is the Best Linear Unbiased Estimator (BLUE)
2 / 9
Paused

Help

Keyboard shortcuts

, , Pg Up, k Go to previous slide
, , Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
Number + Return Go to specific slide
b / m / f Toggle blackout / mirrored / fullscreen mode
c Clone slideshow
p Toggle presenter mode
t Restart the presentation timer
?, h Toggle this help
Esc Back to slideshow