“Despite the similarities, it is very doubtful that the second law holds forShannon entropies.”

The term "Shannon entropy", aside from the commonly used term information entropy, is also referred to as "information theoretic entropy" or "Shannon-Weaver entropy", the latter signifying the efforts of American mathematician Warren Weaver, co-author of the the follow-up 1949 book

Statistical entropy

Many will argue that Shannon entropy is equivalent to statistical entropy used in physics (Boltzmann entropy). [2] In this sense, the term Boltzmann-Shannon entropy is often used as well as Gibbs-Shannon entropy or Boltzmann-Gibbs-Shannon (BGS) entropy. [3]

Clausius entropy

A rare few will go so far as to argue that Shannon entropy is equivalent to Clausius entropy; although connection is difficult to find. In 1948, for instance, American engineer Myron Tribus was examined for his doctoral degree, at UCLA, he was asked to explain the connection between entropy defined by Claude Shannon, and the entropy defined by Rudolf Clausius (1865). In retrospect, in 1998 Tribus comment on this question that: [4]

“Neither I nor my committee knew the answer. I was not at all satisfied with the answer I gave. That was in 1948 and I continued to fret about it for the next ten years. I read everything I could find that promised to explain the connection between the entropy of Clausius and the entropy of Shannon. I got nowhere. I felt in my bones there had to be a connection; I couldn’t see it.”

References

1. Shannon, Claude E. (1948). "A Mathematical Theory of Communication",

2. Wachter, Armin and Hoeber, Henning. (2006).

3. Mitra, Partha and Bokil, Hemant. (2007).

4. Tribus, M. (1998). “A Tribute to Edwin T. Jaynes”. In

5. Dȳke, Charles. (1988).

See also

● Fisher entropy

● Havrda-Charvat entropy

● Kolmogorov entropy

● Kullback-Leibler entropy

● Rényi entropy

● Tsallis entropy

External links

● Shannon entropy – KnowledgeRush.com.