Einstein Entropy Quotes

Another way to describe the Second Law is in terms of entropy, the degree of disorder and randomness in a system. Any spontaneous process tends to increase the entropy of a system.

In 1877 he published his paper “Probabilistic foundations of heat theory”, in which he formulated what Einstein later called the Boltzmann principle; the interpretation of the concept of entropy as a mathematically well-defined measure of what one can call the “disorder” of atoms, which had already appeared in his work of 1872, is here extended and becomes a general statement.

Planck understood that Clausius was not simply stating the obvious, but something of deep significance. Heat, the transfer of energy from A to B due to a temperature difference, explained such everyday occurrences as a hot cup of coffee getting cold and an ice cube in a glass of water melting. But left undisturbed, the reverse never happened. Why not? The law of conservation of energy did not forbid a cup of coffee from getting hotter and the surrounding air colder, or the glass of water becoming warmer and the ice cooler. It did not outlaw heat flowing from a cold to a hot body spontaneously. Yet something was preventing this from happening. Clausius discovered that something and called it entropy. It lay at the heart of why some processes occur in nature and others do not.

Many of the important principles in twentieth century physics are expressed as limitations on what we can know. Einstein's principle of relativity (which was an extension of a principle of Galileo's) says that we cannot do any experiment that would distinguish being at rest from moving at a constant velocity. Heisenberg's uncertainty principle tells us that we cannot know both the position and momentum of a particle to arbitrary accuracy. This new limitation tells us there is an absolute bound to the information available to us about what is contained on the other side of a horizon. It is known as Bekenstein's bound, as it was discussed in papers Jacob Bekenstein wrote in the 1970s shortly after he discovered the entropy of black holes.

Even in the equations that had been formulated to describe electromagnetism, there is no natural directionality to the interactions of particles; the equations look the same going both directions. If you looked at a video of atoms interacting, you could play it backward and you wouldn’t be able to tell which was correct. It is only in the macroworld of objects, people, planets, and so on, the world governed by entropy, that causation appears to unfold in a single direction. The second law of thermodynamics describes the increasing disorder in the universe at macroscales and is often seen as equivalent to the one-way arrow of time. More and more physicists over the past few decades, sensitive to the nondirectionality that seems to rule at the micro or quantum level, have begun to question the no-teleology rule. Recall that the tiny particles making up the matter and energy of the physical universe are really like worms or strings snaking through the block universe of Minkowski spacetime. Their interactions, which look to us a bit like tiny balls colliding on a billiard table, are from a four-dimensional perspective more like threads intertwining; the twists and turns where they wrap around each other are what we see as collisions, interactions, and “measurements” (in the physicists’ preferred idiom). Each interaction changes information associated with those threads—their trajectory through the block universe (position and momentum) as well as qualities like “spin” that influence that trajectory. According to some recent theories, a portion of the information particles carry with them actually might propagate backward rather than forward across their world lines. For instance, an experiment at the University of Rochester in 2009 found that photons in a laser beam could be amplified in their past when interacted with a certain way during a subsequent measurement—true backward causation, in other words.8 The Israeli-American physicist Yakir Aharonov and some of his students are now arguing that the famous uncertainty principle—the extent to which the outcome of an interaction is random and unpredictable—may actually be a measure of the portion of future influence on a particle’s behavior.9 In other words, the notorious randomness of quantum mechanics—those statistical laws that captured Jung’s imagination—may be where retrocausation was hiding all along. And it would mean Einstein was right: God doesn’t play dice.*23 If the new physics of retrocausation is correct, past and future cocreate the pattern of reality built up from the threads of the material world. The world is really woven like a tapestry on a four-dimensional loom. It makes little sense to think of a tapestry as caused by one side only;

The idea that the same amount of heat causes a greater change of entropy in a cold place than a hot one can seem strange. But consider this as an analogy: A noisy, crowded pub is next to a quiet library. Five rowdy people leave the pub. The din drops but by an indiscernible amount. The five stumble into the library. The noisiness there increases noticeably. When a group of raucous people enter a quiet place, the increase in disruption there is much greater than its fall in the boisterous place from which they came.

But where did the watch come from? This watch is a jinni—elderly Miss McKenna gives it to the young playwright, who takes it back in time to deliver it to her as a young woman. She keeps it all her life until it is time to return it to him. So who made the watch? No one. The watch never went anywhere near a watch factory. Its world line is circular. Novikov has noted that in the case of a macroscopic jinni like this the outside world must always expend energy to repair any wear-and-tear (entropy) it has accumulated so it can be returned exactly to its original condition as it completes its loop. Permissible in theory, macroscopic jinn are improbable. The whole story in Somewhere in Time could have taken place without the watch. The watch seems particularly unlikely since it appears to keep good time. One could have imagined finding a nonworking watch or perhaps a paper clip that passes back and forth between the couple. How lucky to encounter a watch that works! According to quantum mechanics, if one has enough energy, one can always make a macroscopic object spontaneously appear (along with associated antiparticles, which have equal mass but opposite electric charge)—it’s just extremely unlikely. Similarly with jinn, it would be more improbable to find a watch than a paper clip and more improbable to find a paper clip than an electron. The more massive and more complex the macroscopic jinni, the rarer it will be.

Clausius conceived of entropy to capture mathematically this way in which heat strives to redistribute itself.

The less the variation in temperature and the more evenly distributed the heat, the greater the value of the entropy.

Clausius defined changes in entropy as follows: When a quantity of heat flows out of a hot room, it will always cause that room’s entropy to fall by a smaller amount than the increase in entropy caused when that same amount of heat flows into a colder room. In the two-room example, therefore, as the heat flows, the increase in Entropy(cold room) will be greater than the decrease in Entropy(hot room). And that means, the entropy of the whole two-room house has gone up.

In summary, to say the entropy of a system increases is to say the heat within it is becoming more widely dispersed.

Increasing entropy is thus a measure of the decreasing usefulness of heat.

Gibbs’s insight was to find a way showing how the two laws of thermodynamics drive all chemical reactions. He chose to start his argument with a restatement of those laws, so let’s follow his lead: First law: The energy in the universe is constant. Second law: The entropy of the universe tends to increase. Gibbs then showed how all processes of change can be judged by these two laws. He did this, essentially, by turning the two laws into one new law we can call Gibbs’s law: The flow of energy is the means by which the entropy of the universe is increased.

To start with, we have solid carbon and gaseous oxygen. For an intuitive sense of the entropy of this situation, think of the energy as more densely packed in the solid carbon and more dispersed in the gas. After burning, there’s only the gas, the carbon dioxide. The energy that was concentrated in the solid carbon has become more dispersed. What had started off as a mixture of a low-entropy solid and a high-entropy gas has turned completely into a high-entropy gas. Overall, the entropy of the materials has gone up. And, important, as carbon and oxygen combine, heat is released and flows into its surroundings, into the air around the grate, thus warming it up. This heat then disperses through the air, causing its entropy to go up. The reason carbon burns but carbon dioxide never spontaneously “unburns” is that the burning causes a twofold increase in entropy. First it creates carbon dioxide gas, and second it disperses heat through the air around the grate. All told, it’s an effective way of increasing the entropy of the universe.

Water never unburns by itself. Gibbs’s law stipulates that although the entropy of the universe must go up, the entropy of its component parts can go down. This can happen as long as the entropy of other parts of the universe go up by enough to ensure that the sum total of the entropy in the universe has increased.

it is possible for carbon dioxide and water to “unburn”—plants do it all the time—but not “by itself.” Gibbs’s equation allows us to tot up all the entropy changes in different parts of the universe to reveal a marketplace—one in which one bit of the universe pays other bits of the universe for a highly desirable commodity—a local and temporary reduction in entropy. And it does so with a specific and well-defined currency—energy.

This is the famous cycle of life. Carbon dioxide breathed out by animals is absorbed by plants, which make food and oxygen and so on. The cycle needs a constant supply of Gibbs free energy to turn. And crucially, at each step of the cycle, a small amount of free energy is lost as heat. This means at each step the entropy of the universe goes up.

Is Szilard, with his thought experiment, suggesting that information can do the opposite, overcoming the second law of thermodynamics and turning warm air at a constant temperature into useful work? Such a system would reduce the entropy of the universe because this “free” work could be used to force heat to flow the “wrong” way from cold to hot.

Szilard emphatically states that this can’t happen for the following reason: The act of measurement by which the demon determines the molecule’s position must cause an increase in entropy that compensates for any decrease in entropy caused as the piston does work.

In principle could we build a machine that could think without increasing the entropy of the universe? No, albeit with one caveat. There is an intriguing possibility that if a computer could be built that didn’t need to erase data, it wouldn’t dissipate energy.

All objects, from stars and planets to passing spaceships, can fall into a black hole, adding to its mass. As this happens, its pull on the flow of the space around it increases. Therefore, the speed of the “space flow” reaches light speed at a greater distance from the center of a black hole as its mass goes up. The radius of the event horizon grows. But nothing can fall out of a black hole and reduce its mass. Therefore the radius of its event horizon cannot shrink. Hawking spotted an uncanny similarity between this behavior and the behavior of entropy. Both event horizons and entropy never decrease.

The only way for the gas to have zero entropy would be if its molecules were fixed and motionless. But by definition that would also mean their temperature was absolute zero. The point is, if molecules have entropy, they are moving, and they therefore have a temperature. By this reasoning, for a black hole to have entropy, it must, like a gas, have a temperature. And that in turn means that it must radiate heat. But this appears impossible because nothing, including heat, can escape the event horizon.

Hawking and Bekenstein had shown that the three great ideas of modern physics—general relativity, quantum mechanics, and thermodynamics—work in harmony. For these reasons, black hole entropy and radiation have come to dominate contemporary physics as scientists search for a so-called grand unified theory, the Holy Grail of a single principle that explains nature—the world, the universe, everything—at its most fundamental level.

Because the formula he derived for measuring the average number of bits needed to encode a piece of information looked almost exactly like Ludwig Boltzmann and Josiah Willard Gibbs’s formula for calculating entropy in thermodynamics. Here’s Shannon’s equation for calculating the size of any given piece of information: H = –Σi pi logb pi And here’s one way of stating Boltzmann’s equation for calculating the entropy of any given system: S = –kB Σi pi ln pi These two equations don’t just look similar; they’re effectively the same. Shortly after deriving his equation, Shannon pointed the similarity out to John von Neumann, then widely considered the world’s best mathematician. Von Neumann shrugged, suggesting that Shannon call his measure of the number of bits needed to carry a piece of information information entropy on the grounds that no one really understood thermodynamic entropy either.

This understanding of information entropy and redundancy is why we can build the data networks. Take services such as YouTube or Netflix that hold and distribute huge files of video information. These companies reduce the number of bits that make up these files to be as close to their Shannon entropy as possible. This is called compression, and if it weren’t done, the files’ sizes would be too large for our networks. The companies that maintain these networks then add digital redundancy to the compressed files to protect them from noise. These extra bits are a sophisticated electronic version of spelling out a word for clarity over a distorted phone call.

imagine a box full of hot gas, which, of course, has entropy. Now, let’s drop the box past the event horizon of a black hole. Because nothing can come back from across the event horizon, the box has crossed a point of no return and is thus no longer part of our universe. Both the box of gas and the entropy associated with it have disappeared from our universe. But that means that the entropy of our universe has gone down, which directly contradicts the second law of thermodynamics.

Entropy increases the energy content of the black hole, increasing both its mass and the size of its event horizon. Bekenstein’s argument? Whenever the entropy of a black hole increases, so does the area of its event horizon. In other words, the area of the event horizon of a black hole was not an analogy for entropy, it was a direct measure of its entropy. In Bekenstein’s view, this saved the universal applicability of the second law of thermodynamics. The entropy of the universe always increases, even when things fall into black holes, because the entropy lost from the space outside the event horizon is made up for by an increase in the surface area of the event horizon. Bekenstein called this the generalized second law of thermodynamics, or GSL.

I was afraid that if Bekenstein found out about it, he would use it as a further argument to support his ideas about the entropy of black holes, which I still did not like.” But the more Hawking worked, the more he seemed to be proving Bekenstein right. Not only did black holes radiate heat, but they did so by exactly the amount required if the area of their event horizons was indeed a measure of their entropy. By early 1974, Hawking had developed this work into a fully fledged theory. It led to his now-famous discovery that “Hawking radiation” leaks out of all black holes.

At its heart are three concepts—energy, entropy, and temperature. Without an understanding of these and the laws they obey, all science—physics, chemistry, and biology—would be incoherent.

put another way, heat will always dissipate away from a hot region because after a period of random collisions the odds are stacked overwhelmingly in favor of that result. Entropy, by Boltzmann’s reasoning, is simply the number of indistinguishable ways the constituent parts of a system can be arranged. To say entropy increases in any given system is another way of saying that any given system evolves into ever-more-likely distributions or configurations. The second law of thermodynamics is true for the same reason that when a pack of cards arranged in suits is shuffled, it will end up jumbled. There are many more indistinguishable ways for the pack to be disordered than there are for it to end up ordered, and so shuffling takes it in that direction.

This “number of distributions” method of defining entropy extends its explanatory power far beyond the dissipation of heat. It readily explains many irreversible processes in nature. For example, air rushes out of an opened balloon, and never back in, because there are many more ways for the air particles to be spread out throughout the room than for them to be crammed inside the balloon. Similarly, there’s no way to stir milk in a cup of tea so it separates instead of mixing because there are many more ways for the milk particles to spread throughout the tea than to be concentrated in one spot.

Entropy increases with time, therefore, because the chances of its decreasing are tiny. In fact, and this is the mind-blowing part of Boltzmann’s logic, only by observing entropy increase can we tell the direction of time. We differentiate the future from the past because in the future the overall entropy is greater.

entropy increases can manifest in two ways. Either as a rise in temperature or as a change from water to steam. In the latter case, the increasing entropy shows itself as the increasing steaminess of the water.

A thermodynamic chart reveals these to be transitions at a constant temperature at which the entropy of a material changes dramatically. In other words, paradoxically, during phase transitions materials can absorb heat without getting hotter and can reject heat without getting colder.

Archaeology suggests our ancestors figured out how to make fire around a million years ago. Making ice was much harder, and the power to cool is the unglamorous but indispensable technology of the modern age. Refrigeration is the most obviously thermodynamic of all human inventions, and the most defiant of the universal tendency for entropy to increase. These devices force heat to pass from a cold interior to a warm exterior, which is in the opposite direction to the one in which heat flows spontaneously. The purpose of this is to create a space where the relentless increase of entropy is slowed down. Although ostensibly a refrigerator is a cool box, that’s a means to an end. Its ultimate purpose is to slow down decay and putrefaction, which are both examples of entropy increasing. Think of a refrigerator as a device inside which time slows down.

In the condenser, as it releases heat, another phase change occurs as the coolant turns back into a liquid. But this is quite warm, it’s the same temperature as the room. For the refrigeration process to continue, the coolant’s temperature must fall back to 4°C before it can reenter the evaporator. To achieve this, the coolant liquid passes through a tiny nozzle called an expansion valve. As the coolant is forced through it, its pressure drops, and it cools and it’s ready to enter the evaporator once again. The compressor ensures the refrigerator complies with the second law of thermodynamics. Heat flows out of the refrigerator interior lowering its entropy. But the total heat that flows out of the condenser raises the room’s entropy to compensate.

Einstein’s paper, however, goes much further than Planck’s, who by his own account had reached for Boltzmann’s ideas as “an act of desperation.” In contrast, Einstein embraces them. He turns them into what he calls a “heuristic” argument that light always exists as a stream of discrete particles. Heuristic means discovered. Einstein’s saying that though no data definitively proves it to be so, it’s very useful to imagine “that the energy of light is discontinuously distributed in space.” Einstein arrives at this conclusion by taking Boltzmann’s statistical definition of entropy further than Planck dared.

By assuming that a gas consists of tiny lumps, molecules, that are in constant motion, Boltzmann had showed that its entropy will increase by chance alone. Einstein uses these arguments to show that the way the entropy of light changes can also be explained if it is considered, like a gas, to consist of discrete particles. Just as the air in the room you’re sitting in is made up of tiny particles, so, too, is the light that illuminates it.

By assuming that a gas consists of tiny lumps, molecules, that are in constant motion, Boltzmann had showed that its entropy will increase by chance alone. Einstein uses these arguments to show that the way the entropy of light changes can also be explained if it is considered, like a gas, to consist of discrete particles. Just as the air in the room you’re sitting in is made up of tiny particles, so, too, is the light that illuminates it. Having established the particulate nature of light with arguments that mirror Boltzmann’s statistical analysis of a gas, Einstein concludes his paper by showing how this idea can render hitherto unexplained optical behavior “readily understood.

The positive energy of the outgoing radiation would be balanced by a flow of negative energy particles into the black hole. By Einstein’s equation E = mc2 (where E is energy, m is mass, and c is the speed of light), energy is proportional to mass. A flow of negative energy into the black hole therefore reduces its mass. As the black hole loses mass, the area of its event horizon gets smaller, but this decrease in the entropy of the black hole is more than compensated for by the entropy of the emitted radiation, so the second law is never violated.