Notesale: Turn your study into money

Document Preview

Extracts from the notes are below, to see the PDF you'll receive please use the links above

---

Linear regression
&
Correlation

1

Types of
Regression Models
1 Explanatory
Variable

Simple

Regression
Models

2+ Explanatory
Variables

Multiple

2

Types of
Regression Models
1 Explanatory
Variable

Regression
Models

2+ Explanatory
Variables

Multiple

Simple

Linear

NonLinear

Linear

NonLinear

3

Simple Linear Regression
(SLR)

4

5

Linear Regression and Correlation


Linear regression and correlation are
aimed at understanding how two variables
are related



The variables are called Y and X



Y is called the dependent variable



X is called the independent variable



We want to know how, and whether, X
influences Y
6

Simple linear regression

dependent Variable (Y)

Simple linear regression describes the linear relationship
between a predictor variable, plotted on the x-axis, and a
response variable, plotted on the y-axis

Independent Variable (X)

7

Types of regression

Positive regression

Negative regression

No regression

8

How to fit data to a linear model?

The Least Square Method

9

Simple Linear Regression
Model

Population
Y intercept
Dependent
Variable

Population
Slope
Coefficient

Independent
Variable

Random
Error
term

Yi  β0 + β1Xi + ε i
Linear component

Random Error
component
10

Simple Linear Regression
Model

(continued)

Y

Yi  β0 + β1Xi + ε i

Observed Value
of Y for Xi

εi
Predicted Value
of Y for Xi

Slope = β1
Random Error for
this Xi value

Intercept = β0

Xi

X
11

Simple Linear Regression Model
Prediction Equation

ˆ ˆ
ˆ
y i  b0 + b1x i
Sample Slope

ˆ
b1 

Sxy
Sxx

 x  x y  y

 x  x  where
i

i

2

i

Sample Y-intercept

ˆ
ˆ
b 0  y  b1x
12

Interpretation of the
Slope and the Intercept
ˆ
 b0

is the estimated average value of Y when
the value of X is zero
...


13

Simple Linear Regression Model
(Example)
You’re an economist for the county cooperative
...
) Yield (lb
...
0
6
5
...
5
12
9
...
Fertilizer*
Yield (lb
...
)
15

Simple Linear Regression Model
(Example)
Parameter Estimation Solution Table

Xi

Yi

Xi2

Yi2

XiYi

4

3
...
00

12

6

5
...
25

33

10

6
...
25

65

12

9
...
00

108

32

24
...
50

218

Simple Linear Regression Model
(Example)
Parameter Estimation Solution
n

ˆ
b1 

 X  n

  i   Yi 
n
 i 1  i 1 
 X i Yi 
n
i 1
n

 X
  i
n
2
 i 1 
 Xi  n
i 1

2

32 24 
218 


4
32 2
296 
4

 0
...
658  0
...
Slope ( b1 )
 Crop Yield (Y) Is Expected to Increase by
...
for

Each 1 lb
...


18

Coefficient Interpretation
(Example)


^

1
...
65 lb
...
Increase in Fertilizer (X)
...
Y-Intercept ( b 0)
 Average Crop Yield (Y) Is Expected to Be 0
...


When No Fertilizer (X) Is Used
...






If x and y are directly related, r > 0
...


Strength of the linear relationship between x
and y
...


22

Coefficient of Correlation


Pearson’s Correlation Coefficient:

r 



S xy
S xx S yy

1 r 1

To determine strength you look at how closely the
dots are clustered around the line
...


23

Coefficient of Correlation
Values
No
Correlation

-1
...
5

0

+
...
0

24

Coefficient of Correlation
Values
No
Correlation

-1
...
5

0

+
...
0

Increasing degree of
negative correlation
25

Coefficient of Correlation
Values
Perfect
Negative
Correlation

-1
...
5

0

+
...
0

26

Coefficient of Correlation
Values
Perfect
Negative
Correlation

-1
...
5

0

+
...
0

Increasing degree of
positive correlation
27

Coefficient of Correlation
Values
Perfect
Negative
Correlation

-1
...
5

0

+
...
0

28

Coefficient of Correlation Values

29

30

Coefficient of Determination,R2








The coefficient of determination, R2, for a simple
regression is equal to the square of the correlation
The square of the correlation, (r) 2, Measures amount
of variation in Y explained by variation in X
...

2
 The larger the value of R , the more the value of y
depends in a linear way on the value of x
...
(Ie: 22% of variation is explained by
something else
...
x

Y 2  aY  bXY

n2

Correlation analysis
Testing the Coefficient of Correlation
H0: r = 0 There is no correlation between x and y
...

Reject H0 if:
t > t/2,n-2 or t < -t/2,n-2
Test Statistic:

t 

r

(n  2)
(1  r 2 )

follows a Student’s t Distribution with (n-2) degrees of
freedom
...
f
...

H1: b1 ≠ 0 There is a linear relationship between x and y
...
He tracks the grams
of raisins used per bun and
the number of buns sold per
day over a 10 day period
...


Raisins per Bun
(grams)

Number of
Buns Sold

30

20

25

30

80

45

100

61

50

33

30

33

25

43

37

36

75

50

28

20
35

Having established which variable is the independent variable and
which is the dependent we can proceed with our computation of the
formula
...

Raisins per Bun
(grams) x

Number of
Buns Sold y

xy

x2

y2

30

20

600

900

400

25

30

750

625

900

80

45

3600

6400

2025

100

61

6100

10000

3721

50

33

1650

2500

1089

30

33

990

900

1089

25

43

1075

625

1849

37

36

1332

1369

1296

75

50

3750

5625

2500

28

20

560

784

400

Σx = 480

Σy = 371

Σxy= 20,407

Σx2= 29,728

Σy2= 15,269
36

Scatter plot
Scatterplot of No
...
sold (y)

50

40

30

20
20

30

40

50

60
70
Raisin (x)

80

90

100

37

In the context of this problem, this equation could also be
written as follows:
ˆ
ˆ
no
...
of Raisins in grams)
The regression calculation consists of determining the values
ˆ
ˆ
of b0 & b1
...

ˆ  S xy
b1
S xx

ˆ
ˆ
b0  y  b1x

38

ˆ  Sxy
b1
Sxx
ˆ  2599  0
...

ˆ
b0  37
...
3886)(48)  18
...
446 + 0
...
446 + 0
...
446 + 0
...
446 + 0
...
This analysis is shown below
...
38 buns
...
39 buns

40

the correlation coefficient (r)
...
82 This coefficient indicates that there is
a strong and positive relationship
between grams of raisins per bun
and the number of buns sold
...

41

This coefficient represents the percentage change in the
dependent variable explained by the independent variable
...
82 x100
 67
...
24% of
the number of buns sold can be
explained by the number of raisins per
bun
...
9  (0
...
857

se 

The standard error of estimate
indicates that when the regression
equation is used to predict the
number of buns sold we expect it to
be inaccurate by ±7
...


43

The following is a summary of all that we have found in our
Correlation and regression analysis
...

The relationship was a moderate and positive one which meant that as
the number of raisins per bun increased so did the number of buns
sold
...

Regression analysis found that when the baker puts no raisins in the
buns that he can expect to sell 18
...
It also found that for every
additional gram of raisins per bun an additional 0
...

The standard error of estimate showed that predictions made by the
regression equation are expected to be inaccurate by an average of
±7
...

44

MINITAB OUTPUT
Regression Analysis: No
...
sold (y) = 18
...
389 Raisin (x)
Predictor
Constant
Raisin (x)

Standard error

estimate

Coef
18
...
38861

S = 7
...
244
0
...
1%

T
3
...
04

DF
1
8
9

Regression
coefficient

P
0
...
004

R-Sq(adj) = 63
...
0
494
...
9

MS
1010
...
9

F
16
...
004

45

Confidence Interval on Regression
Coefficients
ˆ
Confidence interval on the slope b1

ˆ
ˆ
ˆ
ˆ
b1  t  / 2,( n  2 ) se(b1 )  b1  b1 + t  / 2,( n  2 ) se(b1 )
ˆ
Confidence interval on the slope b 0

ˆ
ˆ
ˆ
ˆ
b 0  t  / 2,( n  2) se(b 0 )  b 0  b 0 + t  / 2,( n  2 ) se(b 0 )

46

Assumptions of Regression
1
...


47

Assumptions of Regression

Y

1
...


X
48

Assumptions of Regression
2
...
For any given value of X, the sampled Y
values are independent
4
...

5
...


50

Multiple Linear
Regression (MLR)

51

Multiple Regression Model
The linear model with a single predictor
variable X can easily be extended to two or
more predictor variables
...
+ b p X p + e

52

Multiple Regression Model


Probabilistic Model
yi = b0 + b1x1i + b2x2i +
...
, xki = individual values of the independent variables,
x1, x2,
...
, bk = the partial regression coefficients for the
independent variables, x1, x2,
...
+bkxki

i

where

ˆ
y = the predicted value of the dependent variable, y, given the
i
values of x1, x2,
...
, xki = individual values of the
independent variables, x1, x2,
...
, bk = the partial regression coefficients for the
independent variables, x1, x2,
...
+ b p X p + e
intercept

Partial Regression
Coefficients

residuals

Partial Regression Coefficients (slopes): Regression
coefficient of X after controlling for (holding all other
predictors constant) influence of other variables from
both X and Y
...
31 + 2
...
0153 Die Height



Predictor
Constant
wire length
Die Height



S = 1
...
3114
2
...
015254

SE Coef
0
...
1260
0
...
2%

DF
2
10
12

SS
1258
...
58
1281
...
33
19
...
63

Regression
equation

Analysis of Variance



P
0
...
000
0
...
9%

MS
629
...
26

F
278
...
000

56

END

57



---