## Covariance

Given sets of variates denoted , ..., , a quantity called the Covariance Matrix is defined by

 (1) (2) (3)

where and are the Means of and , respectively. An individual element of the Covariance Matrix is called the covariance of the two variates and , and provides a measure of how strongly correlated these variables are. In fact, the derived quantity
 (4)

where , are the Standard Deviations, is called the Correlation of and . Note that if and are taken from the same set of variates (say, ), then
 (5)

giving the usual Variance . The covariance is also symmetric since
 (6)

For two variables, the covariance is related to the Variance by
 (7)

For two independent variates and ,

 (8)

so the covariance is zero. However, if the variables are correlated in some way, then their covariance will be Nonzero. In fact, if , then tends to increase as increases. If , then tends to decrease as increases.

The covariance obeys the identity

 (9)

By induction, it therefore follows that
 (10) (11) (12) (13) (14)