Multivariate Analysis

Simple Linear Regression



Standard Deviation

Sum of Squared Deviations (SS)

Sum of Cross Products (SSCP)

Population Regression Model

Yi is the value of the dependent, response or outcome variable for the ith case
β0 and β1 are parameters
Xi is the value of the independent or predictor variable for the ith case
εi is a random error term with:

Regression Equations

Partitioning the Sums of Squares

Deriving the Least Squares Regression Coefficients:

The Regression Equation

Computational Simplification

From Calculus...

Derivation of the Constant

More Calculus...

Deriving the Regression Coefficient

In Deviation Score Form

Correlation Coefficient

Squared Correlation Coefficient
aka -- Coefficient of Determination

Coefficient of Alienation

Variance of Estimate

Standard Error of Estimate

Alternative formula

Standard Error of Regression Coefficient

Test of Regression Coefficient

Test of Regression Model

Standardized Regression Coefficients
where β denotes standardized regression coefficient

Sums of Squares Regression


Sums of Squares Residual

More Partitioning
This time partitioning variances

Residuals Illustrated

Testing the Regression
In general:

In Simple Regression

Confidence Interval for Regression Coefficient

Factor Affecting Precision

  • Sample Size, n
  • The amount of scatter about the regression line, i.e., the standard error of estimate
  • The range of values in the independent variable, X

    Assumptions in Regression Analysis

  • The independent variable, X, is measured without error.
  • The means of the dependent variable, Y, at each level of X, are linear.
  • The mean of the residuals is zero.
  • The εi are uncorrelated (Independence Assumption).
  • The variances of the εi are equal at all levels of the independent variable, X (Homogeneity of Variance Assumption).
  • The errors are not correlated with the independent variable, X.
  • The εi are normally distributed (Normality Assumption) -- Needed for tests of significance.

    Multivariate Course Page

    Phil Ender, 5Jan98