## Entropy

In physics, the word entropy has important physical implications as the amount of disorder'' of a system. In mathematics, a more abstract definition is used. The (Shannon) entropy of a variable is defined as

where is the probability that is in the state , and is defined as 0 if . The joint entropy of variables , ..., is then defined by

See also Kolmogorov Entropy, Kolmogorov-Sinai Entropy, Maximum Entropy Method, Metric Entropy, Ornstein's Theorem, Redundancy, Shannon Entropy, Topological Entropy

References

Ott, E. Entropies.'' §4.5 in Chaos in Dynamical Systems. New York: Cambridge University Press, pp. 138-144, 1993.

© 1996-9 Eric W. Weisstein
1999-05-25