Notesale: Turn your study into money

Document Preview

Extracts from the notes are below, to see the PDF you'll receive please use the links above

---

ES30027 Egg Timer!
Topic

Sub-Topic

Summary of Key Points & Formulae

Key Takeaways!

Matrix Algebra
Review

Vectors &
Matrices; Matrix
Operations;
Transpose

 A 𝑚𝑥𝑛 matrix ⟶ bold uppercase letters
...

 2 matrices are said to be conformable if they have the appropriate dimensions for the same
operation
...

o Addition/Subtraction: same number of rows & columns
...
of columns for 1st matrix should equal no
...

 Key properties & facts!
o 𝑨+𝑩 =𝑩+𝑨
o (𝑨 + 𝑩) + 𝑪 = 𝑨 + (𝑩 + 𝑪)
o (𝐴𝐵)𝐶 = 𝐴(𝐵𝐶) = 𝐴𝐵𝐶
o 𝐴(𝐵 + 𝐶) = 𝐴𝐵 + 𝐴𝐶 𝑎𝑛𝑑 (𝐵 + 𝐶)𝐴 = 𝐵𝐴 + 𝐶𝐴
o 𝐴𝐵 not necessarily equal to 𝐵𝐴
o 𝐴𝐵 = 0 possible even if 𝐴 ≠ 0 and 𝐵 ≠ 0
o 𝐶𝐷 = 𝐶𝐸 possible even if 𝐶 = 0 and 𝐷 ≠ 𝐸
 Transpose of a 𝑚𝑥𝑛 matrix A is the 𝑛𝑥𝑚 matrix B such that 𝑏 = 𝑎 , ∀ 𝑖 = 1, … , 𝑛 𝑎𝑛𝑑 𝑗 =
1, … , 𝑛
 Key properties!
o (𝑨 ) = 𝑨
o (𝑨 + 𝑩) = 𝑨 + 𝑩
o (𝑨𝑩) = 𝑩 𝑨
 Square Matrix: 𝑚𝑥𝑛 matrix where 𝑚 = 𝑛
...
e
...
𝑎 =
1 ∀ 𝑖 = 1, … , 𝑛 𝑎𝑛𝑑 𝑎 = 0 ∀ 𝑖 ≠ 𝑗
...

 Linear Dependence: A set of m-vectors 𝑥 , … , 𝑥 is linearly dependent if there exist numbers
𝛾 , … , 𝛾 such that ∑ 𝛾 𝑥 = 0
...
(can express one vector as a weighted sum of all other

 Key properties of matrices:
o 𝑨+𝑩 =𝑩+𝑨
o (𝑨 + 𝑩) + 𝑪 = 𝑨 + (𝑩 +
𝑪)
o (𝑨𝑩)𝑪 = 𝑨(𝑩𝑪) = 𝑨𝑩𝑪
o 𝑨(𝑩 + 𝑪) = 𝑨𝑩 +
𝑨𝑪 𝑎𝑛𝑑 (𝑩 + 𝑪)𝑨 =
𝑩𝑨 + 𝑪𝑨
o 𝑨𝑩 not necessarily equal to
𝑩𝑨
o 𝑨𝑩 = 𝟎 possible even if
𝑨 ≠ 0 and 𝑩 ≠ 0
o 𝑪𝑫 = 𝑪𝑬 possible even if
𝑪 = 0 and 𝑫 ≠ 𝑬
 Properties of Transposes
o (𝑨 ) = 𝑨
o (𝑨 + 𝑩) = 𝑨 + 𝑩
o (𝑨𝑩) = 𝑩 𝑨
 Properties of Inverses:

Some Special
Matrices

Rank; Inverse;
Trace

vectors)
 When considering linear dependencies of matrices, can either look at the rows/columns of
vectors that make up the matrix, and see if they are linearly dependent, i
...


o 𝑨 𝟏
o (𝑨𝑩)

𝟏
𝟏

=𝑨
= 𝑩 𝟏𝑨

𝟏

𝟏 𝑻

𝟏

o 𝑨𝑻
= 𝑨
 Properties of Traces (where A,B &
C are nxn matrices and 𝛾 is a
scalar)
o 𝒕𝒓(𝑨 + 𝑩) = 𝒕𝒓(𝑨) +
𝒕𝒓(𝑩)
o 𝒕𝒓(𝜸𝑨) = 𝜸𝒕𝒓(𝑨)
o 𝒕𝒓(𝑨𝑩𝑪) = 𝒕𝒓(𝑩𝑪𝑨) =
𝒕𝒓(𝑪𝑨𝑩)
 Key results from matrix
differentiation!


𝒂𝒃
𝒃

=

𝒃𝒂
𝒃

=𝒂

𝑋=

1
3

2
→ 𝑥 = [1
4

2] 𝑜𝑟

1
2
& 𝑥 = [3 4] 𝑜𝑟
3
4

 Rank (of a set of vectors): The maximum number of linearly independent vectors that can be
chosen from the set
...

o For any matrix, rank of row vectors = rank of column vectors
...

o Singularity: An mxm matrix A is singular if the rank of A is less than m
...

 Key properties!
 (𝐴 ) = 𝐴
 (𝐴𝐵) = 𝐵 𝐴
 (𝐴 ) = (𝐴 )
 Inverse of a 2x2 matrix:
1
𝑎 𝑏
𝑑 −𝑏
𝐴=
→𝐴 =
𝑐 𝑑
𝑎𝑑 − 𝑏𝑐 −𝑐 𝑎
 Inverse of a diagonal matrix
1 0 0
⎡
⎤
1
1 0 0
0⎥
⎢0
𝐴= 0 4 0 →𝐴 =⎢
4
⎥
1⎥
0 0 2
⎢
⎣0 0 2⎦
 Trace [tr()]: The sum of the elements on the principal diagonal of a square matrix
...
r
...

 Scalar w
...
t column vector → mxn matrix
...
r
...

 Key results!





𝑨𝒃
𝒃𝑨
= 𝑨 𝒂𝒏𝒅
=𝑨
𝒃
𝒃
𝒃 𝑨𝒃
= (𝑨 + 𝑨 )𝒃
𝒃
𝒃 𝑨𝒃
If A is symmetric,
=
𝒃

𝟐𝑨𝒃

o

=

o

= 𝐴 𝑎𝑛𝑑

o

=𝑎
=𝐴

= (𝐴 + 𝐴 )𝑏

o If A is symmetric,
Ordinary Least
Squares with
Matrices

Probability
Distributions

Random
Variables;
Random Vectors

= 2𝐴𝑏

 Convert a scalar relationship into a vector matrix relationship by stacking all the rows for different
observations
...

 Multivariate true linear model: 𝑦 = 𝑋𝛽 + 𝑢
𝑦
𝑦= ⋮
𝑦
𝑋 = [1 𝑥 … 𝑥 ]
𝛽
𝛽= ⋮
𝛽
𝑢 … 𝑢 ]
𝑢 = [𝑢
 Multivariate Linear Fitted Model: 𝑢 ≡ 𝑦 − 𝑋𝛽
 Concept of Ordinary Least Squares: min ∑ 𝑢 = min 𝑢 𝑢, given 𝑢 𝑢 = ∑ 𝑢
 𝛽
= (𝑋 𝑋) 𝑋 𝑦 (know how to prove this too!)
 𝑋 𝑋 should be a positive definite matrix if 𝛽
works
...

 A random vector x is a vector whose elements are themselves random variables
...

𝐸(𝑥 )
⋮
 Expected Value: 𝐸(𝑥) =
⋮
𝐸(𝑥 )
 Variance: A matrix containing variances of each element of x along the principal diagonal, and the
covariances between different elements of x off the diagonal
...

o X is a fixed, non-stochastic matrix with rank k
...

U is a random vector with 𝐸(𝑢) = 0 and 𝑣𝑎𝑟(𝑢) = 𝐸(𝑢𝑢 ) = 𝜎 𝐼
𝑣𝑎𝑟(𝑢) is the error covariance matrix, which gives the covariance between any 2 elements in u
...

o Assume that all observations have zero covariance (no autocorrelation)
...
e
...

o Linearity: 𝛽
= 𝐶𝑦, where 𝐶 denotes a matrix
o Unbiasedness: 𝐸 𝛽
= 𝛽 = 𝐸(𝐶𝑦)
Residuals: 𝑢 ≡ 𝑦 − 𝑋𝛽 = (𝐼 − 𝑋(𝑋 𝑋) 𝑋 )𝑢
Expected Sum of Squared Residuals: 𝐸( 𝑢 𝑢) = 𝜎 (𝑛 − 𝑘)
𝑢 𝑢
⟹𝜎
=
𝑛−𝑘
Since 𝜎
is an unbiased estimator, 𝐸(𝜎 ) = 𝐸
=𝜎

 Normal Distribution, N 𝝁, 𝝈𝟐
o 𝜇 is the mean, controls center point
...
e
...

 Linearity: 𝛽
= 𝐶𝑦, where 𝐶
denotes a matrix
o Unbiasedness: 𝐸 𝛽
=𝛽=
𝐸(𝐶𝑦)

 Key relationships between
distributions:
o If 𝑧 ~ 𝑁(0,1), 𝑖 = 1, … , 𝑛 and
𝑧 are all independent, then
∑ 𝑧 ~ 𝜒 (𝑛)

o If 𝑧 ~ 𝑁(0,1), 𝑦 ~ 𝜒 (𝑛) and 𝑧
& 𝑦 are independent, then
√
√

~ 𝑡(𝑛)

 Key properties on joint PDFs:
 𝑓 (𝑥, 𝑦) ≥ 0, for all x & y
∫
𝑓 (𝑥, 𝑦) 𝑑𝑦 𝑑𝑥 =
∫
1
∫

∫

𝑓 (𝑥, 𝑦) 𝑑𝑦 𝑑𝑥 =

𝑃(𝑎 ≤ 𝑋 ≤ 𝑏, 𝑐 ≤ 𝑌 ≤ 𝑑)

 Student’s t-Distribution, 𝒕(𝒗)
o Has a single parameter
...

o As 𝑣 → ∞, 𝑡(𝑣) → 𝑁(0, 1) [standard normal distribution]

 Key properties of multivariate
normal distributions:
 An affine transformation (a type
of geometric transformation that
preserves collinearity) of a
normal vector is a normal
vector
...
e
...
v’s
...

o Defined only for positive values of x
...


 Chi-square Distribution, 𝝌𝟐 (𝒑)
o Right-skewed distribution
...

o Need one degree of freedom 𝑝
...

 Joint P
...
v’s move together
...
v’s,
their PDF gives the probability of X and Y simultaneously lying in given ranges
...

𝑓 (𝑥) =

𝑓 (𝑥, 𝑦) 𝑑𝑦

𝑓 (𝑦) =

𝑓 (𝑥, 𝑦) 𝑑𝑥

 Independence: X and Y are said to be independent iff: 𝑓 (𝑥, 𝑦) = 𝑓 (𝑥)
...
Not independent, in general
...

𝐴𝑥 + 𝑏 ~ 𝑁(𝐴𝜇 + 𝑏, 𝐴Σ𝐴 )

Distribution of
OLS Estimates






Testing A Single
Hypothesis

 If covariance between 2 jointly normal random variables is 0, the random variables are
independent, i
...
, if the off-diagonal terms in 𝑣𝑎𝑟(𝑥) are 0, elements of x are independent
r
...
[useful for proving independence]
𝑢 ~ 𝑁(0, 𝜎 𝐼)
𝑢 are independent r
...
’s, variance matrix is a diagonal matrix [zero autocorrelation]
If 𝑢 is multivariate normal, each of 𝑢 will be univariate normal, where 𝛽
is an affine
transformation of 𝑢, and 𝑥 𝐴𝑥 is a scalar
...
e
...

 The respecified statistic:


~ 𝑁(0,1) ⟹

~ 𝑡(𝑛 − 𝑘), which now follows a t-distribution and has 𝑛 − 𝑘 degrees of freedom
...

1
...

2
...

3
...
Decide on a significance level 𝑐
...
i
...

5
...
If |𝑡 | > 𝑡 , reject 𝐻
...

P-values: They capture the probabilities of observing OLS estimates that are greater in absolute
value than the estimates actually obtained, under the assumption that the true parameter value is
0
...

Using a two-sided F-test!
Here, you can jointly test a set of 𝑞 linear restrictions on the model parameters and express the
restrictions as 𝑅𝛽 = 𝑟
...

If 𝐻 : 𝑅𝛽 = 𝑟 is true, 𝑅𝛽 − 𝑟 ~𝑁(0, 𝜎 𝑅(𝑋 𝑋) 𝑅 )

 F-statistic (learn to derive this!): 𝐹(𝑞, 𝑛 − 𝑘)~

Heteroskedastici
ty

OLS Under
Heteroskedasticity

Generalised Least
Squares (𝜷𝑮𝑳𝑺 )

Feasible General
Least Squares

 Heteroskedasticity: Occurs when 𝑣𝑎𝑟(𝑢) is diagonal, but has different terms along the main
diagonal, i
...

 Under heteroskedasticity, 𝐸 𝛽
= 𝛽
...

 How do you choose the method?
 If we know the structure of the variance matrix, use feasible GLS
...

 Both provide unbiased estimates of 𝛽 and standard errors in large samples, but perform poorly
in small samples
...

 By standardization, we can re-estimate a new variance matrix, and the transformed model
would be homoskedastic
...
OLS is a type of GLS estimator
...

 The special case of GLS under heteroskedasticity only is aka Weighted Least Squares (𝛽
)
...
Rather, we only need to know the entries of Ω up to a constant
...

𝑣𝑎𝑟(𝑢) = 𝜎 Ω
 This is how GLS can be used to find the IV estimator, given that 𝑣𝑎𝑟(𝑢) = 𝜎 (𝑍 𝑍)
 Key question: Problem Set 3, Q1(d)
...

𝜎 𝐼
0
Ω=
0
𝜎 𝐼
 How to perform feasible GLS?
1
...

2
...

𝜎 =
𝜎 =

∑

1
𝑛 −𝑘

𝑢

𝑢 (the first observation from group B is 𝑛 + 1!)

 How do you choose the method?
 If we know the structure of the
variance matrix, use feasible
GLS
...

 Both provide unbiased
estimates of 𝛽 and standard
errors in large samples, but
perform poorly in small
samples
...

 The special case of GLS under
heteroskedasticity only is aka
Weighted Least Squares (𝛽
)
...

Rather, we only need to know the
entries of Ω up to a constant
...

𝑣𝑎𝑟(𝑢) = 𝜎 Ω
 This is how GLS can be used to
find the IV estimator, given that
𝑣𝑎𝑟(𝑢) = 𝜎 (𝑍 𝑍)

Ω=

𝜎 𝐼

0

0

𝜎 𝐼

3
...


White Standard
Errors
Instrumental
Variable
Estimation

The Endogeneity
Problem

𝛽
= 𝑋 Ω 𝑋 𝑋 Ω 𝑦
𝛽
is not unbiased, but the bias disappears in large samples
...

 Given that 𝑣𝑎𝑟 𝛽
= (𝑋 𝑋) 𝑋 ΩX(𝑋 𝑋)
...

 As the sample size tends to infinity, 𝑋 ΩX tends to converge to 𝑋 ΩX
...
Even if X is
random, OLS is still an unbiased estimator
...

 𝐸 𝛽
≠ 𝛽, OLS is biased!
 What if you could increase the sample size? Even if there is bias in a finite sample, will this bias
tend to 0 as sample size tends to infinity?
 Consistency: An estimator 𝜃 is said to be consistent if it converges in probability to the true
parameter value 𝜃, which requires that:
lim 𝑃 𝜃 − 𝜃 ≤ 𝜀 = 1, for all 𝜀 > 0
...

 What does convergence of probability require?
 Limit 𝑛 → ∞ of the probability that the absolute difference between our estimator and our true
value is less than some value 𝜀 going to 1 for any positive 𝜀
...

 An estimator is consistent if 𝑝𝑙𝑖𝑚 𝛽 = 𝛽
...

 Using these rules (also need to calculate!): 𝑝𝑙𝑖𝑚 𝛽

= 𝛽 + 𝑝𝑙𝑖𝑚

𝑝𝑙𝑖𝑚

 If endogeneity occurs, OLS is
biased and 𝐸 𝛽
≠ 𝛽
...

 If 𝑝𝑙𝑖𝑚

=Σ

, X is well-

behaved
...

 Key conditions for validity of
instruments:
 Relevance
 Exogeneity

 For specification of IVs, k is the
number of variables and l is the
If 𝑝𝑙𝑖𝑚
= 0, 𝛽
is consistent and will provide good estimates as long as we have a large
number of instruments
...
If not, (if equal to Σ ≠ 0) then 𝑝𝑙𝑖𝑚(𝛽 ) ≠ 𝛽 and 𝛽
is inconsistent even in large  The IV Estimates:
 𝛽 =
samples
...
(linearly dependent instruments
 𝑝𝑙𝑖𝑚 𝛽 = 𝛽, hence 𝛽 is
must be ruled out!)
consistent!
Key conditions for validity of instruments!
 𝑣𝑎𝑟(𝑍 𝑢) = 𝜎 𝑍′𝑍
 2SLS Estimation:
 Relevance: 𝑝𝑙𝑖𝑚
= Σ ≠ 0: variables in Z are correlated with those in X
...

variable 𝑥 𝜖 𝑋 on Z and
“Exogenous” variables are not correlated with u, and can be used as instruments for themselves
...

Endogenous Variables: Variables correlated with u
...
e
...
If 𝑙 > 𝑘, over-identification
...

under-identification
...
e
...




 𝑣𝑎𝑟(𝑍 𝑢) = 𝜎 𝑍′𝑍
 2SLS Estimation: Need to perform OLS in 2 stages
...
Regress each endogenous variable 𝑥 𝜖 𝑋 on Z and obtain predicted values
...

𝑥 = 𝑍(𝑍 𝑍) 𝑍 𝑥 for 𝑗 = 1, … , 𝑘
𝑥 = 𝑥 for all exogenous variables
...
Regress y on 𝑋 and obtain the OLS slope estimate 𝛽
= 𝑋 𝑋 𝑋 𝑦 [also need to prove
that 𝛽 = 𝛽
]
 Need to use the GLS variance estimator 𝑣𝑎𝑟 𝛽
, since estimated variance matrix of OLS
is wrong
...

 Are instruments uncorrelated with U? Use Sargan test
...

 Key Characteristics:
 The IV must be correlated with economic growth, but not directly affect the probability of
conflict
...


𝑋 𝑋 𝑋 𝑦 [also need to
prove that 𝛽 = 𝛽
]
 Standard errors may change when
using either approach, but IV
estimation gives better standard
errors
...

 Is the IV relevant?
 It should have a positive relationship with economic growth
...

 Switching to the IV from OLS regressions should not result in loss of significance of the
coefficient, but should ideally be larger than the coefficient
...

 Step 2: Regress dependent variables on predicted values from Step 1
...
But the coefficients shouldn’t change a
lot, and they should still be significant
...

 Use an F-test to regress the dependent variable on the dummy variables
...

1
...

 Likelihood function takes y as given and tells the likelihood (probability) of obtaining some
parameter value 𝜃
...

𝐿(𝜃; 𝑦) = 𝑓 (𝑦 ; 𝜃)
...
Calculate the log likelihood (In 𝐿(𝜃; 𝑦)) by taking logs of the likelihood function
...
Set the derivative of the log likelihood function w
...
t 𝜃 (the “score”) equal to 0
...

4
...

These are the ML estimates, 𝜃
...
Write down the likelihood function
...

𝐿(𝛽, 𝜎 ; 𝑦) = 𝑓 (𝑢; 𝛽, 𝜎 )
Since individual 𝑢 are independent normal r
...
𝑓 (𝑢 ; 𝛽, 𝜎 )
Since error terms have same distribution,
𝐿(𝛽, 𝜎 ; 𝑦) =
2
...


𝑓 (𝑢 ; 𝛽, 𝜎 )



Key steps:
 Write down the likelihood
function
...

 Set the derivative of the log
likelihood function (the
“score”) to 0
...


𝑛
𝑛
1
(𝑦 𝑦 − 2𝑦 𝑋𝛽 + 𝛽 𝑋 𝑋𝛽)
ln 𝐿(𝛽, 𝜎 ; 𝑦) = − ln(2𝜋) − ln 𝜎 −
2
2
2𝜎
3
...

𝜕 ln 𝐿(𝛽, 𝜎 ; 𝑦|𝑋)
−1
⎡
⎤
(−𝑋 𝑦 + 𝑋 𝑋𝛽)
𝜕𝛽
0
⎢
⎥=
𝜎
=
⎢𝜕 ln 𝐿(𝛽, 𝜎 ; 𝑦|𝑋)⎥
−𝑛
1
0
⎢
⎥
(𝑦 − 𝑋𝛽) (𝑦 − 𝑋𝛽)
+
2𝜎
2𝜎
⎣
⎦
𝜕𝜎
4
...

𝛽 = (𝑋 𝑋) 𝑋 𝑦
1
𝜎 = 𝑦 − 𝑋𝛽
𝑦 − 𝑋𝛽
𝑛
 𝛽 is unbiased, but 𝜎 is not
...
e
...

→

 Estimator variance disappears as sample size increases, i
...
, lim 𝑣𝑎𝑟 𝜃
→

Regression with
Binary Dependent
Variables

=0

 𝜎 is consistent
...

 Major drawbacks of using regular OLS estimates:
 OLS predictions of 𝑦 can be less than 0 or greater than 1, which are hard to interpret when
dependent values are binary variables
...
This
problem may disappear in large samples
...
𝑣𝑎𝑟(𝑢 ) is a function of 𝑥 , the model exhibits heteroskedasticity
...
But can use White Standard Errors
...


Probit

 An ML alternative to LPM
...

 Define 𝑦 ∗ , a latent variable (which is unobserved) such that 𝑦 ∗ = 𝑥 𝛽 + 𝑢
...

 Similarly, 𝑃(𝑦 = 0) = 1 − Φ
 These equations are non-linear in parameters, OLS can’t be used
...

 Order the data such that 1st m observations have 𝑦 = 0 & n-m observations have 𝑦 = 1
...
i
...

𝑥𝛽
𝑥𝛽
𝐿(𝛽, 𝜎 ; 𝑦|𝑋) =
1−Φ
Φ
𝜎
𝜎
 Taking log likelihood, which needs to be maximised numerically (since there is no closed form
expression for the standard cdf):
𝑥𝛽
𝑥𝛽
ln 𝐿(𝛽, 𝜎 ; 𝑦|𝑋) =
(𝑦 ln Φ
) + (1 − 𝑦 )ln ( 1 − Φ
)
𝜎
𝜎
 Probit estimates 𝛽 & 𝜎 obtained
...


Marginal Effects

 Helps to interpret 𝛽 , as a derivative indicating how much a one unit increase in each X variable
will increase the expected value of y by
...

( )
 In probit, marginal effects:
=
Φ
=𝜙
 Can be evaluated at any value of 𝑥 , however, it is more common to use the mean values of each
X variable, denoted 𝑥̅ , a row vector
...
Then the estimated marginal effect just uses the probit coefficients in
place of the true model parameters
...




---