Variance

Covariance

Standard Deviation

Sum of Squared Deviations (SS)

Sum of Cross Products (SSCP)

Population Regression Model

where:
Y_i is the value of the dependent, response or outcome variable for the ith case
β₀ and β₁ are parameters
X_i is the value of the independent or predictor variable for the ith case
ε_i is a random error term with:

expected value (mean) equal to zero
equal variances for each ε_i
ε_i and ε_j are uncorrelated for all i, j; i not equal j

Partitioning the Sums of Squares

Deriving the Least Squares Regression Coefficients:

The Regression Equation

Computational Simplification

From Calculus...

• By differential calculus...
• Find the derivative; set it to 0; solve for the unknown variable.
• First time, differentiate with respect to a.

Derivation of the Constant

More Calculus...

• Next differentiate with respect to b.
• Find the derivative; set it to 0; solve for the unknown variable.

Deriving the Regression Coefficient

In Deviation Score Form

Correlation Coefficient

Squared Correlation Coefficient
aka -- Coefficient of Determination

Coefficient of Alienation

Variance of Estimate

Standard Error of Estimate

Alternative formula

Standard Error of Regression Coefficient

Test of Regression Coefficient

Test of Regression Model

Standardized Regression Coefficients
where β denotes standardized regression coefficient

Sums of Squares Regression

alternatively

Sums of Squares Residual

More Partitioning
This time partitioning variances

Residuals Illustrated

Testing the Regression
In general:

In Simple Regression

Confidence Interval for Regression Coefficient

Factor Affecting Precision