Advanced Statistics
Simple Linear Regression
In simple linear regression there is only one predictor variable and takes the form
y = b0 + b1x + e.
Where y is the response variable,
x the predictor variable
and e the error or residual.
Other names for the
predictor variable include independent variable, explanitory variable or right-hand variable.
The response variable is also know as the dependent variable, criterion variable,
outcome variable or left-hand variable.
Variance
Covariance
Standard Deviation
Sum of Squared Deviations (SS)
Sum of Cross Products (SSCP)
Conditional Expectation
The conditional expectation of y given x is written as, E(y | x).
This expectation indicates that we are interested in in the effect of variable x on
the expected value of y.
Population Regression Model
Yi = β0 +
β1Xi
+ εi
where:
Yi is the value of the dependent, response or outcome variable for the ith case
β0 and β1 are parameters
Xi is the value of the predictor, independent or explanitory variable for the ith case
εi is a random eror term with:
expected value (mean) equal to zero
equal variances for each &epsiloni
εi and εj are
uncorrelated for all i, j; i not equal j
Regression Equations
or 
Scatterplot with Regression Line
Partitioning the Sums of Squares
Residuals Illustrated
Correlation Coefficient
Degree of association between variables.
Squared Correlation Coefficient
aka -- Coefficient of Determintion
Proportion of variance accounted for by the predictor.
Coefficient of Alienation
Proportion of variance not accounted for by the predictor.
Residual
Sum of Squares Residuals
Variance of Estimate
Standard Error of Estimate
Alternative formula
Regression Coefficients
Standard Error of Regression Coefficient
Test of Regression Coefficient
Test of Regression Model
Standardized Regression Coefficients
where β = standardized regression coefficient
Standardized regression coefficients are what you would get if all the variables in the regression
were first converted to standard scores (z-scores).
Sums of Squares Regression
alternatively
Sums of Squares Residual
More Partitioning
This time partitioning variances
Residuals Illustrated
Testing the Regression
In general:
In Simple Regression
Confidence Interval for Regression Coefficient
Factors Affecting Precision
Sample Size, n
The amount of scatter about the regression line, i.e., the standard error of estimate
The range of values in the independent variable, X
Advanced Statistics
Phil Ender, 5Jan98
