Entropy...

Did you really understand it?

Imagine making yourself a cup of tea.

The water is hot enough that you have to wait a little before taking the first sip.

You place the cup on your desk, get distracted by something else, and forget about it.

An hour later, you return.

The tea is cold.

There is nothing strange about this.

Hot drinks cool down.

You have seen it happen so many times that you would probably be more surprised if the tea were still hot.

But suppose something slightly different happened.

You leave a cold cup of tea on your desk.

An hour later, you return and discover that the tea has become steaming hot, while the room around it has become slightly colder.

That would immediately feel wrong.

Not impossible in the sense that energy had appeared from nowhere.

The tea could have gained exactly as much energy as the surrounding air lost.

Energy would still be conserved.

The strange part would be the direction in which the energy moved.

Heat naturally flows from hotter things to colder ones.

We do not ordinarily see a cold drink pulling heat out of an entire room until the drink becomes hot.

And tea is not the only example.

Open a bottle of perfume in one corner of a room.

A few minutes later, someone on the other side may smell it.

The perfume spreads.

But you never watch perfume molecules scattered throughout a room suddenly gather themselves back into the bottle.

Drop some milk into coffee.

The milk spreads through it.

Wait long enough and the two become thoroughly mixed.

But nobody has ever left a cup of milky coffee on a table and returned to find the milk neatly separating itself from the coffee again.

Break an egg.

The yolk and white spill out.

You never see the pieces of shell jump together while the yolk gathers itself back inside.

These events are so familiar that we rarely think about them.

But they are pointing towards something profound.

The world around us seems to have a preferred direction.

Some things happen naturally one way.

They almost never happen naturally the other way.

And hidden inside this simple observation is one of the most misunderstood ideas in physics.

Physicists call it entropy.

The strange difference between the past and the future

Imagine recording two billiard balls colliding on a smooth table.

One ball rolls towards another.

They collide.

Both balls move apart.

Now play the video backwards.

The balls approach one another, collide, and move apart again.

Unless you knew which version had been recorded first, it might be difficult to tell which video was running forward.

The backwards version does not obviously violate the laws of motion.

Now record a glass falling from a table.

It hits the floor and shatters.

Play that video backwards.

The pieces slide across the floor, leap into the air, reconnect perfectly, and return to the table as an intact glass.

You know immediately that something is wrong.

Why?

At the level of simple motion, many of the laws of physics do not strongly distinguish between moving forward and moving backward in time.

If we knew the positions and velocities of ideal particles and reversed all their motions perfectly, many familiar equations would still describe what happened.

But the large-scale world does distinguish between the two directions.

Glasses break.

They do not rebuild themselves.

Coffee cools.

It does not spontaneously reheat itself by cooling the room.

Perfume spreads.

It does not collect itself.

The microscopic rules seem far more reversible than the world made from them.

So where does this direction come from?

To answer that, we need to stop thinking about what particles can do.

We need to think about how many ways they can do it.

One situation can hide many different arrangements

Imagine twenty people entering a cinema.

For some reason, all twenty decide to sit on the left side of the room.

That is perfectly possible.

There are many seats on the left, so the people could arrange themselves in several ways while still keeping everyone on that side.

Now imagine another situation.

Roughly ten people sit on the left and ten sit on the right.

There are far more ways for this to happen.

Different people can sit on different sides.

They can choose different seats.

You could rearrange the entire audience repeatedly and still end up with the same broad description:

About half the people are on each side.

To someone looking at the cinema from far away, all these arrangements appear to be the same general situation.

But underneath that simple description are many different detailed arrangements.

Physics makes an important distinction between these two levels.

The complete microscopic arrangement is called a microstate.

For a gas, that would include the detailed positions and motions of its enormous number of particles.

The large-scale description is called a macrostate.

That might tell us the gas’s temperature, pressure, volume, and density without telling us where every individual molecule happens to be.

One macrostate can correspond to an enormous number of microstates.

And this is where entropy begins to make sense.

Imagine a box filled with gas.

At one moment, every molecule happens to be crowded into the left half.

That arrangement is physically possible.

But there are relatively few microscopic arrangements that produce the description:

“All the gas is on the left.”

There are vastly more arrangements in which the molecules are spread throughout the entire box.

The particles do not need to know where they are supposed to go.

They do not need an invisible force telling them to spread out.

They simply move.

And there are overwhelmingly more ways for their ordinary movement to leave them spread across the box than gathered together in one small region.

That difference in possibilities is at the heart of entropy.

Entropy is not really about mess

You may have heard entropy described as a measure of disorder.

Low entropy means order.

High entropy means disorder.

This explanation is popular because it is easy to remember.

It is also one of the reasons entropy becomes so confusing.

Consider a gas spread evenly throughout a container.

From a distance, it looks perfectly uniform.

The density is similar everywhere.

Nothing about it looks particularly messy.

Now imagine forcing every molecule into one tiny corner.

That state looks far less uniform.

Yet the evenly distributed gas has the higher entropy.

Or consider milk mixing into coffee.

At first, you might see white streaks curling through the dark liquid.

The cup contains visible patterns and irregularities.

Eventually, everything becomes one smooth colour.

The final mixture may actually look more orderly to your eyes.

Yet its entropy has increased.

So entropy cannot simply mean visual messiness.

The better idea is multiplicity.

How many microscopic arrangements can produce the large-scale condition we see?

A state that can be realised in relatively few microscopic ways has lower entropy.

A state that can be realised in vastly more microscopic ways has higher entropy.

This is the statistical understanding associated with Ludwig Boltzmann.

And once we understand that idea, his famous equation becomes much less mysterious:

S = k_B lnΩ

Here, (S) represents entropy.

(k_B) is Boltzmann’s constant, a number that connects the microscopic world of particles to the macroscopic units we use in thermodynamics.

And (Ω) represents the number of microscopic arrangements (microstates) compatible with the large-scale state.

The logarithm (ln) simply turns what can be unimaginably enormous numbers of arrangements into something more manageable and gives entropy the right mathematical behaviour when systems are combined.

But the important idea is already contained in the picture.

More possible microscopic arrangements.

Higher entropy.

That is far more useful than simply saying “more disorder.”

Why perfume spreads across a room

Now return to the perfume bottle.

When you first open it, most of the perfume molecules are concentrated near one small region.

The molecules are moving constantly.

They collide with air molecules.

They change direction.

They continue moving.

Nothing is guiding them towards the rest of the room.

But imagine how many possible arrangements exist in which nearly every perfume molecule remains close to the bottle.

Now compare that with the number of arrangements in which the molecules are scattered throughout the entire room.

The second number is incomparably larger.

So as the molecules move randomly, the system almost inevitably enters one of the vastly more numerous spread-out arrangements.

This is why perfume diffuses.

And it helps us understand the Second Law of Thermodynamics.

The law is often stated like this:

The entropy of an isolated system tends to increase.

It can sound as though entropy were a force.

As though nature had written a commandment into matter:

Thou shalt become more disordered.

But nothing is pushing the perfume towards higher entropy.

The molecules are simply moving according to ordinary physical laws.

The reason the spread-out state dominates is numerical.

There are simply far more ways to be spread out.

Could every perfume molecule, purely by chance, happen to move back towards the bottle at the same time?

For a very small number of particles, fluctuations like this are possible.

But a visible amount of perfume contains an enormous number of molecules.

Once numbers become that large, the probability of all of them coordinating by chance becomes absurdly small.

Not merely unlikely in the everyday sense.

So unlikely that you could wait far longer than the age of the universe and still have no reasonable expectation of seeing it.

The Second Law is therefore unusual among the laws of physics.

It has a deeply statistical character.

It describes what becomes overwhelmingly likely when enormous numbers of particles are involved.

With a handful of particles, probability looks like probability.

With roughly (10²³) particles, probability begins to look like certainty.

Why hot things cool

This also explains our abandoned cup of tea.

The tea begins hotter than the room.

At the microscopic level, its molecules contain more thermal energy on average than the surrounding air and objects.

Through collisions and radiation, energy moves between the tea and its environment.

There is nothing in conservation of energy alone that says all that energy could not gather more strongly inside the tea.

The total would still balance.

But there are vastly more microscopic arrangements in which the thermal energy is shared more evenly between the tea and the room than arrangements in which an unusually large amount remains concentrated inside one small cup.

So energy spreads.

The tea cools.

The room warms by an almost imperceptible amount.

Eventually, they approach thermal equilibrium.

This is why heat naturally moves from hot to cold.

It is not because coldness somehow attracts heat.

It is because the combined system has enormously more ways of distributing its energy when that energy is spread out.

Once again, entropy is counting possibilities.

Can entropy ever decrease?

At this point, another problem appears.

If entropy tends to increase, how does anything organised ever exist?

Water freezes into beautiful crystals.

Plants grow from seeds.

A fertilised egg develops into an extraordinarily organised human body.

Human beings build cities, write books, manufacture computers, and arrange matter into structures that would never form by random collisions alone.

Does all of this violate the Second Law?

No.

Because the Second Law does not say that entropy must increase everywhere, at every moment.

It matters what system we are talking about.

Consider a refrigerator.

Inside the refrigerator, heat is removed from the food.

That region becomes colder.

Its entropy can decrease.

But place your hand behind a working refrigerator.

The coils are warm.

The machine uses electrical energy to move heat from the colder interior into the warmer room, and in doing so it releases additional heat.

The entropy decrease inside the refrigerator is more than compensated for by an increase elsewhere.

Nothing has been violated.

The same principle applies to life.

You are not an isolated system.

You constantly take in energy and matter.

You eat food.

You breathe oxygen.

Your cells use energy to maintain structures, repair damage, build proteins, pump ions, copy DNA, and keep you far from equilibrium.

At the same time, you release heat and waste into your surroundings.

A living organism can maintain extraordinary local organisation while increasing the entropy of the larger environment.

Life does not defeat entropy.

It lives by exchanging entropy with the world around it.

This is an important distinction.

Whenever someone claims that a local increase in organisation violates the Second Law, one of the first questions should be:

Where did you draw the boundary?

Sometimes the apparent contradiction disappears as soon as we include the rest of the system.

The law is statistical, not absolute in the way we usually imagine laws

Suppose we flip a fair coin four times.

Getting four heads is unusual.

But nobody would call it miraculous.

Now imagine flipping the coin one thousand times and getting heads every single time.

Nothing in the rules of probability says this sequence is impossible.

But you would immediately suspect that something else was happening.

The larger the number of trials becomes, the more extreme certain outcomes become.

Entropy works with numbers far beyond a thousand.

A glass of water contains on the order of (10^{25}) molecules.

A room contains an unimaginable number of air molecules.

For all of those particles to spontaneously coordinate so that heat flows entirely from cold to hot, or gas gathers into one corner, is not forbidden in the simple sense that a conservation law forbids creating energy from nothing.

It is simply buried beneath astronomical improbability.

This is one of the most beautiful things about statistical mechanics.

It explains how behaviour that looks like a strict law at our scale can emerge from probability underneath.

The molecules are not obeying a command to increase entropy.

They are exploring possibilities.

Almost all the possibilities look like equilibrium.

Why broken things do not rebuild themselves

Now think again about the shattered glass.

It is tempting to say:

The intact glass is ordered.

The shattered glass is disordered.

Entropy increased.

There is some truth hidden inside that picture, but the real story is richer.

When the glass hits the ground, energy that was once concentrated in its motion becomes distributed into many places.

Fragments move in different directions.

The floor vibrates.

Sound waves travel through the air.

The glass and floor warm slightly.

The surrounding air is disturbed.

The original state involved very specific correlations.

The glass had to be in one piece.

Its molecules had to occupy particular positions.

Its centre of mass had to follow a particular motion.

Reversing the process would require far more than simply lifting the pieces.

Every microscopic motion produced during the crash would need to coordinate with extraordinary precision.

Sound waves would have to converge.

Thermal motion would have to organise itself.

Fragments would have to arrive at exactly the right angles and speeds.

Atoms along every fracture would have to reconnect.

The laws of mechanics do not necessarily contain a simple sign saying this reverse motion is illegal.

The obstacle is statistical.

There are vastly more ways for the energy and matter to remain dispersed than for every microscopic detail to cooperate in rebuilding the original glass.

The broken glass is not special because nature prefers ugliness.

It is special because there are enormously more microscopic roads leading away from the precise original state than back towards it.

Entropy gives the world an arrow

And now we can return to the strange difference between the billiard balls and the shattered glass.

At the microscopic level, much of physics does not obviously care which direction we label “future.”

But the large-scale world does.

A hot drink becomes cold.

Perfume spreads.

Eggs break.

People age.

Stars burn through their fuel.

You can often look at two photographs and tell which one came first.

This direction associated with increasing entropy is called the thermodynamic arrow of time.

Entropy does not necessarily explain every philosophical question about why time exists or why we experience it the way we do.

But it gives physical processes a direction.

It separates the kind of state we call “before” from the kind of state we usually call “after.”

An intact glass can become a shattered one in countless ordinary ways.

The shattered glass almost never reconstructs itself.

A concentrated gas spreads.

A spread-out gas almost never gathers itself spontaneously.

Heat flows towards equilibrium.

The reverse is overwhelmingly improbable.

The laws underneath may allow far more symmetry between past and future than our everyday experience suggests.

But statistics breaks that apparent equality at the scale we inhabit.

Why did the universe begin with low entropy?

There is one final mystery.

If systems naturally move from lower entropy towards higher entropy, then the universe we see today must have come from an earlier state with lower entropy.

This raises an enormous question.

Why?

Why did the early universe begin in a condition special enough for entropy to have so much room to increase?

Cosmology makes this question surprisingly complicated, especially once gravity enters the picture.

A smooth early universe may look visually simple, but gravitationally it represents a very unusual condition. As matter clumps into stars, galaxies, and eventually black holes, gravitational entropy can increase dramatically.

So the early universe appears to have started in a remarkably low-entropy state.

Physics can describe much of what happened afterward.

Entropy increased.

Stars formed.

Temperature differences developed and disappeared.

Complex structures arose while energy continued to spread.

But why the universe began in the kind of state that allowed this enormous thermodynamic history is still a deep foundational question.

The Second Law tells us how the arrow points.

It does not, by itself, fully explain why there was an arrow available to begin with.

Entropy is not the enemy of complexity

There is another misconception worth removing.

If entropy increases, it can sound as though the universe must constantly move from complexity towards simplicity.

But look around.

The history of the universe includes atoms, stars, planets, oceans, cells, brains, ecosystems, and technological civilisation.

Complexity clearly appears while the Second Law continues to operate.

This is possible because increasing entropy does not mean that every location becomes increasingly featureless at every moment.

Energy flowing from concentrated sources to diffuse surroundings can actually create opportunities for structure.

The Sun is hotter than the Earth.

The Earth is warmer than deep space.

Energy flows through this difference.

Plants capture some of that energy.

Atmospheres circulate.

Oceans move.

Weather forms.

Living organisms maintain themselves.

Complex structures can exist precisely because the universe has not yet reached complete equilibrium.

A universe already at perfect equilibrium would have far less ability to do interesting work.

Differences matter.

Hot and cold.

Dense and sparse.

High concentration and low concentration.

These gradients allow processes to happen.

Entropy tends to erase them over time.

But while they exist, astonishing structures can emerge in between.

The strange power of enormous numbers

Entropy is often presented as one of the darker ideas in science.

Everything decays.

Everything spreads out.

Everything moves towards equilibrium.

But I think that description misses what makes the idea remarkable.

Entropy shows us how the behaviour of the world can change when numbers become enormous.

One molecule does not have a temperature in quite the same sense as a cup of water.

One particle does not produce pressure.

A handful of particles can fluctuate unpredictably.

But put unimaginably many particles together and new regularities appear.

Temperature.

Pressure.

Diffusion.

Irreversibility.

Entropy.

The individual particles do not know these concepts exist.

None of them carries a tiny instruction telling it which way time should move.

Each particle simply follows the physical rules governing its motion.

Yet together, the statistics become so overwhelming that an arrow appears.

The microscopic world contains possibilities.

The macroscopic world contains history.

So, what is entropy?

Entropy is not simply chaos.

It is not messiness.

It is not a mysterious substance that fills the universe.

And it is not a force pushing everything towards destruction.

At its deepest statistical level, entropy tells us something about the number of microscopic ways a physical system can realise the large-scale condition we observe.

States with more compatible microscopic possibilities have greater entropy.

And because some large-scale states can be realised in incomparably more ways than others, physical systems overwhelmingly tend to move towards them.

That tendency explains why gases spread.

Why heat flows from hot to cold.

Why perfume fills a room.

Why shattered glasses do not rebuild themselves.

Why living things must continually use energy to maintain themselves.

And why the world around us contains a physical distinction between what we call the past and what we call the future.

The Second Law does not say that nature loves disorder.

It says something stranger.

There are simply more ways for some things to happen than others.

A few molecules may occasionally wander against the trend.

A small system may fluctuate.

But once the number of particles becomes enormous, the imbalance between possibilities becomes overwhelming.

Nature does not need to be pushed towards the future.

There are simply more roads leading there.

The cup cools.

The perfume spreads.

The star burns.

The universe changes.

And somewhere between the possible and the probable, probability becomes so powerful that it begins to look like destiny.