## Multivariate Analysis

### Simple Linear Regression

**Variance**

**Covariance**

**Standard Deviation**

**Sum of Squared Deviations (SS)**

**Sum of Cross Products (SSCP)**

**Population Regression Model**

where:

Y_{i} is the value of the dependent, response or outcome variable for the ith case

β_{0} and β_{1} are parameters

X_{i} is the value of the independent or predictor variable for the ith case

ε_{i} is a random error term with:

expected value (mean) equal to zero

equal variances for each ε_{i}

ε_{i} and ε_{j} are uncorrelated for all i, j; i not equal j

**Regression Equations**

**Partitioning the Sums of Squares**

**Deriving the Least Squares Regression Coefficients:**

**The Regression Equation**

**Computational Simplification**

**From Calculus...**

- By differential calculus...
- Find the derivative; set it to 0; solve for the unknown variable.
- First time, differentiate with respect to a.

**Derivation of the Constant**

**More Calculus...**

- Next differentiate with respect to b.
- Find the derivative; set it to 0; solve for the unknown variable.

**Deriving the Regression Coefficient**

**In Deviation Score Form**

**Correlation Coefficient**

**Squared Correlation Coefficient**

aka -- Coefficient of Determination

**Coefficient of Alienation**

**Variance of Estimate**

**Standard Error of Estimate**

Alternative formula

**Standard Error of Regression Coefficient**

**Test of Regression Coefficient**

**Test of Regression Model**

**Standardized Regression Coefficients**

where β denotes standardized regression coefficient

**Sums of Squares Regression**

alternatively

**Sums of Squares Residual**

**More Partitioning**

This time partitioning variances

**Residuals Illustrated**

**Testing the Regression**

In general:

**In Simple Regression**

**Confidence Interval for Regression Coefficient**

**Factor Affecting Precision**

Sample Size, n
The amount of scatter about the regression line, i.e., the standard error of estimate
The range of values in the independent variable, X
**Assumptions in Regression Analysis**

The independent variable, X, is measured without error.
The means of the dependent variable, Y, at each level of X, are linear.
The mean of the residuals is zero.
The ε_{i} are uncorrelated (Independence Assumption).
The variances of the ε_{i} are equal at all levels of
the independent variable, X (Homogeneity of Variance Assumption).
The errors are not correlated with the independent variable, X.
The ε_{i} are normally distributed (Normality Assumption)
-- Needed for tests of significance.

Multivariate Course Page

Phil Ender, 5Jan98