Notesale: Turn your study into money

Document Preview

Extracts from the notes are below, to see the PDF you'll receive please use the links above

---

Introduction

Quantitative Research Methods

Teaching and Assessment
• Lectures (2 hours) – Formative quiz after each one
...

• Assessment:
th
1
...

th
2
...

st
3
...


Variables
• A variable is always something that can be measured and can vary, and is sometimes
something that we can manipulate and control for
...

• Dependant: sometimes called the outcome variable, its value does depend on the value of
another variable
...

• Dichotomous: only two categories
...


Research Design: Non-experimental
• Correlational designs: look for relationship between variables
...

• Longitudinal design: look for changes over time
...


Research Design: Experimental
• Looks for group differences when groups have been created by the experimenter
...

• Within subjects: Group A and Group B contain the same people, group difference now lacks
the ‘noise’ of people differences
...

• Sampling error: minimise by selecting more randomly and increase the amount of people in
your sample
...

• Mode: most frequently occurring value
...


Variation
• Range: from minimum value to maximum value
...

• Standard deviation: standardised measure of variability
...
Leptokurtic – peaked
...
Mesokurtic – normal, bell-shaped
...
Platykurtic – flat
...



Probability and Null-Hypothesis Significance Testing (NHST)
• Probability: mathematical probability is a decimal fraction between 0 and 1, 0 means it
cannot happen, 1 mean it must happen, and anywhere in between means it may happen
...

• Hypothesis testing: H0 (null hypothesis) looks for the likelihood of finding your pattern of
data under the assumption that there is no pattern in your data
...

• Two tailed: there will be an effect but you cannot estimate which of the means will be
higher
...


Types of Error
• Type 1 error: falsely rejecting null hypothesis, probability of making this error is the alpha
level
...


Statistical Tests
Looking for:
Test:
Associations
Correlations
Differences between 2 groups/conditions
T Tests
Does A predict B?
Regression
Do A and B predict C?
Multiple Regression
Differences between three or more levels of a
One way ANOVA
single factor
Differences between multiple factors with many Multi level ANOVA
levels



Issues of Significance
Confidence interval – what is the population mean?
• A statistic that estimates a range within which the mean of a population is probably located
(e
...
mean of 20 degrees Celsius, plus or minus 2 degrees
...

• Can be completed by hand but SPSS will compute as well
...

• p(probability) value
...
“Yes, my effect is likely to be true” (p <
...

2
...
05)
...


Effect size – how big, in standard terms, is a difference or relationship?
• Any difference or relationship statistic represents an effect size e
...
r =
...

• Unadjusted statistics like r and t are not very portable
...

• Cohen’s d (1988) is a standardised measure of difference
...


•

Expressed in standard deviations i
...
the amount of separation between group data – small =
0
...
5 and large = 0
...



Power – if an effect exists, how likely are you to detect it with your design?
• “Too many studies are underpowered” (Maxwell, 2004) – we are not good enough at
detecting effects
...

• Statistical design should minimise Type II errors, that is, avoiding false negatives or failing to
see an effect when there is one
...
e
...

• In other words, how likely are you to find your effect?
• Like p, power is measured from 0-1:
1
...

2
...

• Cohen (1988) suggests aiming for 80%
...


Maximising power
• Effect size – choose a large one
...

• Statistical test – parametric tests are more powerful than non-parametric
...

• One vs
...


Calculating power
• Calculating power is useful in estimating, for example, how many participants you might
need
...


Two-sample between-subjects T-test
• Method:
1
...

2
...

3
...


Summary
• Confidence interval allows you to estimate where the population mean will be on 95% of
occasions
...

• Effect size is a measure of how large a population relationship or difference actually is
...




One-way ANOVA (Analysis of Variance)
What is ANOVA?
• Analysis of variance – testing for differences between groups when you have more than two
groups
...

H0 – no group differences
...



Why not do lots of T-tests?
• Design with 3 groups – three comparisons – 1 v 2, 2 v 3 and 1 v 3
...

• Familywise error is the likelihood of a Type 1 error (false positive) in a set of tests where n ix
the number of tests
...
95)n

Assumptions of ANOVA
• Dependant measure residuals are normally distributed
...

• Variance should be the same across conditions; termed homogeneity of variance
...

• This is because individual differences could cause correlations
...

• ANOVA does not tell us which means differ from which
...

• Post hoc comparisons: pairwise comparisons to examine where the difference is, once we’ve
established there is an effect
...

• Either:
1
...
05/n; n = number of t-tests
...
Or choose a post hoc test (already corrected to take into account the family-wise
error rate): Bonferroni’s – very conservative, Tukey’s – good if equal sample sizes or
Scheffé
...

• Level: condition
...

• Grand mean: overall mean for all observations
...
g
...


One-way ANOVA – between groups
• Is at least one pair of experimental means truly different?
• Variation between groups – variation between participants in whole study
...

• Variation within groups - variation between participants in each level
...

• Both between and within group variance are influenced by individual differences,
experimenter error and other sources of variation
...

• F = between group variance/within group variance OR F = variance explained by the
factor/unexplained variable
...



How does ANOVA calculate variance?
• It starts by calculating raw variance between scores, called the Total Sum of Squares
...
This is between-groups variance
...
This is within-groups variance
...


Assumptions (within-groups)
• Just like between groups ANOVA but requires sphericity
...

• In other words, it is similar to homogeneity of variance in one-way ANOVA
...

• Tested using Mauchly’s test: H0 – sphericity present; H1 – sphericity absent
...

• Variation between participants is how much variability is there for each person compared to
the other people
...

• ANOVA tells us there is a difference somewhere between groups, but not which groups,
hence we need follow-up analyses
...

• ANOVA tells us whether there is a difference between the means of our experimental
groups
...


More language of ANOVA
• Cell – distinctive levels in factorial design resulting from combination of its factors
...

• Interaction effect – effect of factors combined
...


Two-way between-groups ANOVA
• Two factors with at least two levels each where each factor is an independent variable and
each level within a factor is a condition
...
e
...

• More than one factor means your analysis and design is factorial
...

We can look at how factors interact
...



Reporting the design
• Do not mention ANOVA in the design; ANOVA is a form of analysis, not a form of design
...
How many factors?
2
...
Which factors are within participants (repeated) and which are between
participants (independent)?

Variance
• Sources of variance in a two-way between groups ANOVA:
1
...

2
...

3
...

4
...

• Model now means ‘all factors and interactions’
...

• Two-way ANOVA produces 3 F-ratios: one for each main effect and one for the interaction
...

• For interactions – interaction effects are uneven or asymmetric paired cell mean differences;
sometimes called ‘difference of differences’
...

• If your factor has more than two levels (groups), you will need to do pairwise comparisons to
see where the differences lie
...

• Investigating interactions is more complicated – a significant interaction tells us that the
combination of both factors has an effect but it doesn’t tell us which cell means differ
significantly from which
...


Summary
• Two-way ANOVA works in a similar way to a one-way ANOVA but permits interaction
analysis
...




Two-way Repeated and Mixed Groups ANOVA
The story so far
• One-way independent ANOVA – one IV (three or more groups) and one DV (interval level of
measurement)
...

• Two-way independent ANOVA – two IVs (two or more groups each) and one DV (interval
level of measurement)
...
repeated measures designs
• Between-groups designs: individual differences may vary between groups meaning there is
relatively more individual differences
...


Repeated measures assumptions
• Same as between-groups assumptions (except independence)
...

• Between participants variation contains individual differences and other error
...

• In SPSS – paired samples t-tests (with Bonferroni’s correction)
...

• When our factors are repeated, we can do one-way ANOVAs of Factor X within each level of
Factor Y
...


Investigating the interaction
• When the design is mixed, we must be careful with the Simple Main Effects
...

• SMEs of independent factor within each level of an independent factor (requires by-hand
analysis)
...

• Two-way mixed ANOVA – investigate with simple main effects where appropriate
...

• Workflow (effect sizes)
...

• ANCOVA
...


Why use assumptions?
• Why are they important? – when violated our Type 1 error and Type 2 error rates change:
1
...

2
...

Assumptions of ANOVA
1
...

2
...


3
...


Equality of variances – in between groups factors
...

• If correlation is r = 0
...

• How to keep to this assumption – make sure your observations are independent
...

• As a rule of thumb, distributions within groups should be normally distributed
...

• How to keep to this assumption – test for normality
...

• Apply tests whose H0 is normality, so p < 0
...


Q-Q plots
• These plot expected normal values (predicted) against observed values
...


Shapiro-Wilk
• Compares your data with normal distribution computed from your data
...

• Remember, distributions within factor levels should be normally distributed
...

• Use Levene’s Test, whose H0 is that variances are equal
...
05 indicates variance inequality
...

• Applies to all repeated measures factors
...

• p < 0
...


Coping with broken assumptions
1
...

2
...

3
...
Again if group sizes are
equal, there may not be an issue
...

4
...



ANOVA workflow
1
...

2
...


3
...

5
...

Run follow-up analyses where necessary
...



Outliers
• Remember that ANOVA is a test or means
...

• Produce a boxplot and check for extreme cases
...

• Have these people fallen asleep or decided to sabotage your experiment? Remove them
...

• Pairwise comparisons always compare two levels
...

• A level marked 0 is excluded from the comparison
...


Beyond ANOVA – extensions to the paradigm
• Analysis of covariance (ANCOVA) is identical to ANOVA but it assigns another source of
variance to the covariate and removes it
...

• Multivariate analysis of variance (MANOVA) is an ANOVA applied to more than one DV
...

• The MANOVA will look for overall mean differences for the Factor in all DVs combined
...


Summary
• ANOVA is quite robust in violation of assumption, but not always, so be careful
...




Advanced Nonparametric Statistics
Outline
• Parametric vs
...

• Mann-Whitney test
...

• The Kruskal-Wallis one-way analysis of variance
...


Parametric vs
...

• Parametric tests: these estimate the parameter of the population based on a sample e
...
a ttest which uses sample variance to estimate population variance
...

• Nonparametric tests: sometimes called distribution-free tests, do not require the population
to be normal
...


•
•

Yet the shape of the distribution doesn’t affect the ranks much
...



Medians
• Remember, the median is the value separating the higher half of a number set from the
lower half
...


Comparing the two

Parametric
Nonparametric
Central tendency sensitivity
Sensitive to mean differences
Sensitive to median differences
Sensitivity to outliers
High
Low
Power
Higher
Lower
Calculation difficulty
Harder
Easier

Mann-Whitney test – testing for differences between two groups of independent data
• Rationale: if you take all the data points in your sample, then rank them by size, then put
them back into their original groups, you can conclude a significant group difference if the
ranks are unevenly distributed between your groups
...

• Mann-Whitney – rank all scores then calculate how many times one group is ranked higher
than the other
...

1
...

2
...

3
...

4
...

• Report Mann-Whitney U as U
...

• Use exact significance
...


Wilcoxon’s matched pairs signed-ranks test – testing for differences between two groups of related
data
• Rationale: when you have two related groups, a person can either increase (+), lower (-) or
maintain their scores
• If no true difference between all pairs, the signs of the differences will be random
...

• Wilcoxon – obtain differences for all pairs of data, rank the differences, ignoring signs, and
obtain the sum of ranks for the least occurring sign
...

1
...

2
...

3
...

• Sum of ranks = Wilcoxon t
...

• Reporting – z = ?, p < ?
...

Mann-Whitney vs
...

SPSS output – H is reported as Chi-square, remember like ANOVA, Kruskal-Wallis is always
two-tailed
...

Follow up analyses – paired comparisons using Mann-Whitney
...

• If no true difference within a person across groups, the signs of the group differences will be
random throughout the groups
...

• SPSS output – remember, like ANOVA, Friedman is always two-tailed
...

• Follow up analyses – paired comparisons using Wilcoxon
...

• They convert data to ranks, then looks to see if ranks differ
...




Correlation and Regression
Correlation – describing a linear relationship
• Descries the relationship between two variables
...

• Precisely, it is a coefficient measure of the linear dependency between two variables
...

• H1 – there is a linear relationship
...

• Covariation is a good measure of whether two variables are associated
...

• If we standardised them, we can compare them, going from simple covariation to
correlation
...


•
•
•
•

Magnitude by the number
...

0 = no correlation
...



What are nonlinear relationships?
• Exponential – recall for number of items (x) over time (y)
...


Other ways of visualising correlations
• What is the overlap between two variables?
• In other words, what is their shared variance?
• When we multiply r by itself, we obtain the proportion of variation in variable A accounted
for by variation in variable B
...


Regression – describing and predicting a linear relationship
• Like correlation, regression examines the relationship between variables
...

• Like correlation, it uses paired data
...

• IV = predictor
...

• A regression line is a straight line that best fits the plotted data (line of best fit)
...

• y is the value of the variable on the vertical y axis
...

• x is a value of the variable on the horizontal x axis
...


Describing the line of best fit in regression
• y = b0 + b1x
...

• b1 – the gradient shows the direction and strength of the relationship
...


The line of best fit is not quite enough
• The line of best fit approximates the relationship between the two variables
...

• The further away they fall, the greater the error
...


Representing error
1
...

2
...

3
...

Writing up regression equations
2
• F(?) = ?, p < ?, with an R of ?
...

• Regression is the direction, magnitude and prediction of a linear relationship
...

• Multiple regression examines the relationship between three or more variables while still
allowing prediction of one variable
...


Dimensions
• A line of best fit has two dimensions
...

• When we add a variable, we add a dimension
...


R-square
• In the case of multiple regression, our measure of association between the outcome variable
and predictors is called the Multiple R
...

• The more predictor variables added to a model, the more variability in the dependent
measure is “explained”
...


R-square adjusted
• R-square adjusted is a modified version of R-square that accounts for the number of
predictors
...

• In other words, the unadjusted R-square is biased by your sample
...


Correlations: relationships within the model
• Zero-order correlation (r) – simply a correlation between the DV and an IV
...

• First order semi partial correlation (sr) – a correlation between the DV and an IV using only
the variability in the DV that the IV uniquely explains
...
71)
...
64, MSE = 70
...
05
...

• The unstandardised regression coefficient for IQ was
...
18,
...
49, p <
...
52 (95% CI [
...
98], t = 2
...
05)
...
54 and
...
IQ explained more unique
variance in exam scores (27%) than hours revision (13%)
...
Estimates of unique variance
are sensitive to order
...

• Stepwise/forward – predictors are selected statistically (using their correlation with the
outcome)
...


Outliers
• May have a strong effect on your regression
...
Data input error
...
People who didn’t understand the questionnaire/test
...
People deliberately entering wrong answers
...
Genuinely usual
...


Outliers: what to do
• If, in good conscience, you’re able to remove a case, remove it but note in the results
section
...


Assumptions of multiple regression
1
...
Testing this in SPSS – ask for a correlation matrix of the predictors
...
Normality – the residuals (variance) should be normally distributed
...

3
...
Testing this in SPSS – ask for a scattergram plotting standardised
predicted outcome v standardised residuals
...
Linearity – the relationships we’re modelling should be linear
...

5
...
Testing this in SPSS
– ask for the Durbin-Watson statistic
...

• What to do – in discussion, comment on the fact that assumptions have been broken and
results may not be reliable
...

• Outliers – have a big influence, maybe delete them
...





















---