entropy: Details

“Ludwig Boltzmann defined entropy as being directly proportional to the natural logarithm of the number of microstates yielding an equivalent thermodynamic macrostate (with the eponymous constant of proportionality). Assuming (by the fundamental postulate of statistical mechanics), that all microstates are equally probable, this means, on the one hand, that macrostates with higher entropy are more probable, and on the other hand, that for such macrostates, the quantity of information required to describe a particular one of its microstates will be higher. That is, the Shannon entropy of a macrostate would be directly proportional to the logarithm of the number of equivalent microstates (making it up). In other words, thermodynamic and informational entropies are rather compatible, which shouldn't be surprising since Claude Shannon derived the notation 'H' for information entropy from Boltzmann's H-theorem.”