Partial and multiple correlation definitions, properties

Partial correlation

Partial correlation measures the linear relationship between two variables when the linear effect of one or more other variables is removed (held constant). For three variables, the partial correlation of X₁ and X₂ keeping X₃ constant is

r₁₂.₃ = (r₁₂ − r₁₃r₂₃) / √[(1 − r₁₃²)(1 − r₂₃²)]

Example: the correlation between a student's marks in Mathematics (X₁) and Physics (X₂) is partly due to their general intelligence/study hours (X₃). r₁₂.₃ gives the correlation between Mathematics and Physics marks after removing the effect of study hours.

Multiple correlation

Multiple correlation measures the degree of linear relationship between one variable (dependent) and the combined effect of two or more other variables (independent). For three variables,

R₁.₂₃ = √[(r₁₂² + r₁₃² − 2r₁₂r₁₃r₂₃) / (1 − r₂₃²)]

Example: the correlation between crop yield (X₁) and the combined effect of rainfall (X₂) and fertilizer (X₃).

Properties of partial correlation

1. It lies between −1 and +1: −1 ≤ r₁₂.₃ ≤ 1.
2. It is symmetric: r₁₂.₃ = r₂₁.₃.
3. Its order equals the number of variables held constant (r₁₂.₃ is of first order; r₁₂ is of order zero).
4. It is a simple correlation of the residuals of X₁ and X₂ after regressing each on X₃.
5. Its sign depends on the numerator r₁₂ − r₁₃r₂₃.
6. If X₃ is uncorrelated with both X₁ and X₂ (r₁₃ = r₂₃ = 0), then r₁₂.₃ = r₁₂.

Properties of multiple correlation

1. It is never negative: 0 ≤ R₁.₂₃ ≤ 1.
2. R₁.₂₃ ≥ |r₁₂| and R₁.₂₃ ≥ |r₁₃|; it is at least as large as any simple correlation of X₁ with X₂ or X₃.
3. R₁.₂₃ = 1 means X₁ is an exact linear function of X₂ and X₃; R₁.₂₃ = 0 means X₁ is uncorrelated with both.
4. R₁.₂₃² is the coefficient of multiple determination: the proportion of variation of X₁ explained by X₂ and X₃.
5. It is the simple correlation between X₁ and its estimate X̂₁ from the multiple regression of X₁ on X₂, X₃.
6. R₁.₂₃ ≠ R₂.₁₃ in general (it depends on which variable is dependent).