What Does Entropy Mean and Where Does Entropy Come From?

There’s no such thing as a stupid question. This is perhaps the most profound question in all of science. Nevertheless, it does have a fairly simple answer ever since a genius by the name of Josiah Willard Gibbs (1839–1903) figured it all out in the late nineteenth century.

The answer is that everywhere in Nature there is a balance between two fundamental qualities: energy, which you probably know something about, and entropy, which you probably don’t (but soon will). It is this balance alone that determines whether or not something can happen.

Certain physical and chemical things can happen all by themselves, but they can’t happen in the opposite direction unless they get some outside help. For example, we could make water go uphill by hauling it or pumping it up. And if we really wanted to, we could get that sugar back out of the coffee by evaporating the water and then chemically separating the sugar from the coffee solids. Unburning a match is quite a bit tougher, but given enough time and equipment, a small army of chemists could probably reconstruct the match out of all the ash, smoke and gases.

The point is that in each of these cases a good deal of meddling, energy input from outside, is required. Left entirely to herself, Mother Nature allows many things to happen spontaneously, all by themselves. But other things will never happen spontaneously, even if we wait, hands-off, until doomsday. Nature’s grand bottom line is that if the balance between energy and entropy is proper, it will happen; if it isn’t, it won’t.

Let’s take energy first. Then we’ll explain entropy.

In general, everything will try to decrease its energy if it can. At a waterfall, the water gets rid of its pent-up gravitational energy by falling down into a pool. (We can make that cast-off energy turn a waterwheel for us on the way down, if we like.) But once the water gets down to the pool, it is “energy-dead,” at least gravitationally speaking; it can’t get back up to the top.

A lot of chemical reactions will happen for a similar reason: The chemicals are getting rid of their stored-up chemical energy by spontaneously transforming themselves into different chemicals that have less chemical energy. (The burning match is one example.) But they can’t get back up to their original energy conditions by themselves.

Thus, other things being equal, Nature’s inclination is that everything will lower its energy if it can. That’s rule number one.

But decreasing energy is only half the story of what makes things happen. The other half is increasing entropy. Entropy is just a fancy word for disorder, or randomness, the chaotic, irregular arrangement of things. At the scrimmage, football players are lined up in an orderly arrangement,  they are not disorderly, and they therefore have low entropy.

After the play, however, they may be scattered all over the field in a very disorderly, higher-entropy arrangement. It’s the same for the individual particles that make up all substances: the atoms and molecules. At any given time, they can be either in an orderly arrangement, or in a highly disordered jumble, or in any kind of arrangement in between. That is, they can have various amounts of entropy, from low to high.

Other things (namely, energy) being equal, Nature’s inclination is that everything tends to become more and more disorderly, that is, everything will increase its entropy if it can. That’s rule number two. There can be an “unnatural” increase in energy as long as there is a more-than-compensating increase in entropy. Or, there can be an “unnatural” decrease in entropy as long as there is a more-than-compensating decrease in energy. Got it?

So the question of whether or not a happening can occur in Nature spontaneously, without any interference from outside, is purely a question of balance between the energy and entropy rules.

The waterfall? That happens because there is a big drop in (gravitational) energy; there is virtually no entropy difference between the water molecules at the top and those at the bottom. It’s an energy-driven process.

The sugar in the coffee? It dissolves primarily because there’s a big entropy increase; sugar molecules swimming around in coffee are much more disorderly than when they were tied neatly together in the sugar crystals. Meanwhile, there is virtually no energy difference between the solid sugar and the dissolved sugar. (The coffee doesn’t get hotter or colder when the sugar dissolves, does it?) It’s an entropy-driven process.

The burning match? Obviously, there’s a big energy decrease, a sudden exodus of energy. The stored-up chemical energy in the match head is released as a burst of heat and light. But there is also a huge entropy increase; the billowing flame, smoke and gases are much more disorderly than the compact match head was. So this reaction is doubly blessed by Nature’s rules, being driven by both energy and entropy. That’s why it proceeds with such gusto the instant you supply the initiating scratch.

What if we have a process in which one of the quantities, energy or entropy, goes the “wrong way”? Well, the process can still occur if the other quantity is going the “right way” strongly enough to overcome it. That is, energy can increase as long as there is a big enough entropy increase to counterbalance it; and entropy can decrease as long as there is a big enough energy decrease to counterbalance it.

What J. Willard Gibbs did was to devise an equation for this energy-entropy balance. If the Gibbs equation shows that after counteracting any “wrong-way” entropy changes there is still some energy left over, that energy (Techspeak: the free energy) can be used to make things happen and the process in question will take place automatically. If, on the other hand, the amount of available (“free”) energy is inadequate to counteract any “wrong-way” entropy changes, the process will not and cannot take place unless some additional energy is obtained from outside.

By adding enough energy, then, we can always overpower nature’s entropy rule that everything tends toward disorderliness.

Here’s an example. There are about 10 million tons, 60 trillion dollars’ worth, of dissolved gold distributed throughout Earth’s oceans, just sitting there for the taking. With enough effort we could collect it all, atom by atom.

But the atoms are dispersed through 336 million cubic miles (1.4 billion cubic kilometers) of ocean in a completely chaotic arrangement that has extraordinarily high entropy. The energy that we would have to expend in order to reduce its entropy by collecting it all in one place would cost enormously more than the value of the gold.

In a fit of fervor over the laws of mechanics, Archimedes (287–212 B.C.) is reputed to have said, “Give me a lever long enough and a place to stand, and I will move the world.” If he had known about entropy and apple pie, he might have added, “Give me enough energy and I’ll put this chaotic world into apple-pie order.”