Key facts
- Rudolf Clausius defined entropy in 1865 as a heat flow divided by temperature, dS = dQ_rev / T, and named it from the Greek for transformation.
- Ludwig Boltzmann gave it a statistical meaning in 1877: entropy counts the microscopic arrangements, S = k log W, which sits on his tombstone in Vienna.
- Gibbs, von Neumann and Shannon wrote closely related formulas; Shannon's 1948 paper applies the same mathematics to information, measured in bits.
- The second law says the entropy of an isolated system cannot decrease; local decreases are paid for elsewhere.
- Calling entropy disorder is a metaphor that misleads, and the arrow of time rests on low-entropy conditions in the past.
Entropy is a measure of how many microscopic arrangements of a system are consistent with what is observed on the large scale, and in thermodynamics it is also the quantity that never decreases in an isolated system. The word covers three related ideas: a heat-based definition from the nineteenth century, a statistical count of microstates, and an information measure from 1948. This note follows the concept from Clausius to Boltzmann to Shannon, explains the second law and the arrow of time, and corrects the common picture of entropy as disorder. It belongs to the science notes.
Clausius and the word entropy
The path to entropy begins with heat engines. Carnot published his analysis of the ideal engine in 1824, and Joule’s work of the 1840s, including his 1843 results, established that heat and work are interchangeable. Rankine wrote about a “thermodynamic function” in 1850. In 1865 Rudolf Clausius gave the quantity a name and a definition: for a reversible process, the change in entropy is the heat absorbed divided by the absolute temperature, dS = dQ_rev / T. He coined the word from the Greek tropē, “transformation”, and it was adopted into English in 1868.
Entropy is a state variable. Its value depends on the state of the system, not on the path taken to reach it, and it is extensive, so it adds up when systems are combined.
Boltzmann and the count of microstates
Ludwig Boltzmann asked what entropy is at the level of atoms. In 1877 he related it to the logarithm of the number of microscopic arrangements. The formula in its present form, S = k log W, was put in that shape by Planck around 1900. Here W is the multiplicity, the number of microstates compatible with the observed macrostate, and k is the Boltzmann constant. Since the 2019 redefinition of SI units, k is fixed at exactly 1.380649 times 10 to the power minus 23 joules per kelvin.
Boltzmann died on 5 September 1906 in Duino, and he is buried in the Zentralfriedhof in Vienna, where S = k log W is carved on his tombstone.
The logarithm has a practical effect. Multiplicities multiply when two systems are combined, and logarithms turn products into sums, which is why entropy is additive. A macrostate with many microstates, such as a gas spread through a room, is overwhelmingly more probable than one with few, such as the same gas squeezed into a corner. The tendency of entropy to increase is therefore a statement about probability, made certain by the huge numbers of particles involved.
The formulas side by side
| Name | Formula | Applies to |
|---|---|---|
| Clausius | dS = dQ_rev / T | Thermodynamic processes |
| Boltzmann | S = k log W | Equally likely microstates |
| Gibbs | S = -k_B Σ p_i ln p_i | Microstates with probabilities p_i |
| Von Neumann | S = -k_B tr(ρ ln ρ) | Quantum states with density matrix ρ |
| Shannon | H(X) = -Σ p(x) log p(x) | Information sources |
The Gibbs formula reduces to Boltzmann’s when all microstates are equally likely. Gibbs is also associated with the phrase “mixedupness” for entropy. The von Neumann formula extends the Gibbs form to quantum mechanics, replacing the probabilities with a density matrix. Wikipedia notes that Gibbs’ formula is formally identical to Shannon’s, which is the bridge between physics and information.
Shannon and information
In 1948 Claude Shannon published “A Mathematical Theory of Communication”, which defined the entropy of an information source as H(X) = -Σ p(x) log p(x). The unit depends on the base of the logarithm: bits for base 2, nats for base e and hartleys for base 10. Shannon’s source coding theorem shows that this entropy is the limit for lossless compression: a source cannot on average be compressed below its entropy per symbol, and it can be brought close to that limit.
The two entropies share a mathematical form but answer different questions. Thermodynamic entropy is measured in joules per kelvin and describes a physical system. Shannon entropy is measured in bits and describes the uncertainty of a message. The link is that both count possibilities weighted by probability.
The second law
The second law of thermodynamics has several equivalent statements. One is that heat flows spontaneously from hot to cold and never the reverse. Another is that the entropy of an isolated system cannot decrease. A cup that falls and breaks is allowed by this law, and the reverse event, shards leaping together into a cup, is not.
The law does not forbid local decreases of entropy. Living systems, refrigerators and growing crystals all reduce entropy locally, and the second law is satisfied because the entropy increase elsewhere, in the surroundings, is at least as large. The law applies to the total, not to the parts.
Entropy also enters chemistry. The Gibbs free energy change, ΔG = ΔH - TΔS, combines the enthalpy change with the entropy change at temperature T, and it tells whether a process can proceed spontaneously under constant temperature and pressure.
Why disorder is a misleading word
Entropy is often described as disorder. The picture helps with a messy room and fails in other cases. A gas in equilibrium has a uniform temperature and density, which looks orderly, yet it has the highest entropy available. The accurate statement is that entropy measures the number of microstates accessible to the system, and that count has to be tied to what an observer can distinguish.
For this reason textbooks increasingly prefer wording about energy spreading among available states or about missing information, and treat disorder as an informal image.
The arrow of time
The microscopic laws of mechanics do not distinguish past from future. If the motion of every molecule were reversed, the new motion would also obey the laws. Loschmidt raised this as a paradox against Boltzmann’s statistical account: how can a time-symmetric dynamics give a one-way increase of entropy?
The accepted answer is about boundary conditions. Boltzmann first proposed the link between entropy and the direction of time in 1877. The increase of entropy toward the future reflects that the universe began in a state of very low entropy, and from such a state almost every evolution leads to higher entropy. The direction of time, in this view, is set by the initial condition and not by the laws.
Connections to other notes
The history of heat and pressure begins in the laboratories described in the physicists of Florence, where thermometers and barometers were first made systematic. The information side of entropy is relevant to quantum technology: whether a quantum computer can beat classical machines is discussed in has quantum advantage been achieved? and in part 2, considering the evidence. Sensors that use cold atoms to measure gravity, described in quantum cartography, work by controlling exactly the kind of noise that entropy describes. The career of a theorist in a neighboring field is in the note on Mark Wise.
What this note leaves out
Several topics lie outside the sources cited, and it therefore says nothing about them. These include the debates on the physical cost of erasing information, the thought experiments about demons that sort molecules, and the recurrence of rare states in long times. Each has a literature of its own and would need separate sources.
About the image
The image shows Ludwig Boltzmann’s grave at the Zentralfriedhof in Vienna, with the formula S = k log W on the monument. It was chosen because the formula is the central statement of statistical entropy and the grave is the place where it is most visibly commemorated.
Frequently asked questions
What is entropy in simple terms?
Entropy measures how many microscopic arrangements of a system are consistent with what is observed at large scale. In thermodynamics it is also defined through heat: for a reversible process the change in entropy is the heat absorbed divided by the temperature.
What does S = k log W mean?
It is Boltzmann's entropy formula. W is the number of microstates compatible with a macrostate, k is the Boltzmann constant, and S is the entropy. The constant is exactly 1.380649 times 10 to the power minus 23 joules per kelvin in the SI.
Is entropy the same as disorder?
Not exactly. Disorder is a loose picture that fails in many cases, for example a gas at equilibrium has uniform temperature, which looks orderly, yet it is the state of highest entropy. Counting accessible microstates is the more accurate description.
How is thermodynamic entropy related to information entropy?
Shannon's 1948 entropy has the same mathematical form as the Gibbs entropy of statistical mechanics, with a different constant and unit. Shannon entropy sets the limit for lossless data compression, and it is measured in bits, nats or hartleys.
Why does time have a direction?
The microscopic laws are symmetric under time reversal. The arrow of time comes from the fact that the early universe had very low entropy, so entropy increases toward the future. Boltzmann first proposed this link in 1877.
Sources and open data
- Entropy, English Wikipedia
- Second law of thermodynamics, English Wikipedia
- Boltzmann's entropy formula, English Wikipedia
- Entropy (information theory), English Wikipedia
- Von Neumann entropy, English Wikipedia
- Entropy as an arrow of time, English Wikipedia
- Ludwig Boltzmann, English Wikipedia