The Science of ${\Theta \Delta }^{\text{cs}}$

Abstract

There is a long tradition of thinking of thermodynamics, not as a theory of fundamental physics (or even a candidate theory of fundamental physics), but as a theory of how manipulations of a physical system may be used to obtain desired effects, such as mechanical work. On this view, the basic concepts of thermodynamics, heat and work, and with them, the concept of entropy, are relative to a class of envisaged manipulations. This article is a sketch and defense of a science of manipulations and their effects on physical systems. I call this science thermo-dynamics (with hyphen), or ${\Theta \Delta }^{\text{cs}}$, for short, to highlight that it may be different from the science of thermodynamics, as the reader conceives it. An upshot of the discussion is a clarification of the roles of the Gibbs and von Neumann entropies. Light is also shed on the use of coarse-grained entropies.

Appendix

1.1 Proof of the Fundamental Theorem of Statistical Thermo-dynamics

In this appendix we prove Proposition 2.

As before, $\rho$ is used ambiguously for either a density function with respect to Liouville measure on classical phase space, or a quantum density operator. Hamiltonian evolution is, in the classical context, evolution according to Hamilton’s equations of motion, and, in the quantum context implemented by a family of unitary operators U(t). The letter S, without subscript, denotes either the Gibbs entropy or the von Neumann entropy.

In the classical context, the salient fact about Hamiltonian evolution—and, indeed, the only fact that we will use—is that Liouville measure is invariant under evolution of that type. As a consequence, the expectation value, with respect to Liouville measure $\Lambda$, of any measurable function on phase space is invariant under Hamiltonian evolution; this includes in particular the Gibbs entropy

$$\begin{aligned} S_G[\rho ] = -k \langle \rho \log \rho \rangle _\Lambda . \end{aligned}$$

(32)

In the quantum context, the salient fact about Hamiltonian evolution is that it conserves the inner product of two vectors in Hilbert space. As a consequence, the trace of any operator is invariant; this includes in particular the von Neumann entropy

$$\begin{aligned} S_{vN}[{\hat{\rho }}] = -k \text{ Tr }[{\hat{\rho }} \log {\hat{\rho }}]. \end{aligned}$$

(33)

As conservation of phase space volume (classical) and absolute magnitude of inner product (quantum) are the only features of Hamiltonian evolution used, we could expand our repertoire of operations to include fictitious operations, such as an instantaneous velocity reversal, that retain these features, and the theorem would still go through.

The relevant facts about the Gibbs and von Neumann entropies are:

• 1. Subadditivity For a composite system AB,

  $$\begin{aligned} S[\rho _{AB}] \le S[\rho _A] + S[\rho _B], \end{aligned}$$

  with equality if and only if the subsystems are probabilistically independent.

• 2. For any $T > 0$, let $\beta = 1/kT$. The canonical distribution $\tau _\beta$ minimizes

  $$\begin{aligned} \langle H \rangle _\rho - T S[\rho ]. \end{aligned}$$

With these facts in hand, the proof of the theorem is easy. For brevity, we will write $S_{AB}(t_0)$ for $S[\rho _{AB}(t_0)]$, etc.. We will consider only interactions with a single heat reservoir, as the extension to successive interactions with multiple heat reservoirs is merely a matter of repeated application of the theorem.

The evolution from $t_0$ to $t_1$ does not change the joint entropy $S_{AB}$. At $t_0$, since the systems are uncorrelated, $S_A + S_B$ is at a minimum for the value of $S_{AB}$ that obtains at both $t_0$ and $t_1$. Therefore,

$$\begin{aligned} S_A(t_0) + S_B(t_0) \le S_A(t_1) + S_B(t_1), \end{aligned}$$

(34)

or,

$$\begin{aligned} \varDelta S_A + \varDelta S_B \ge 0. \end{aligned}$$

(35)

Since B has canonical distribution $\tau _\beta$ at time $t_0$,

$$\begin{aligned} \langle H_B \rangle _{t_0} - T S_B(t_0) \le \langle H_B \rangle _{t_1} - T S_B(t_1), \end{aligned}$$

(36)

or,

$$\begin{aligned} \varDelta \langle H_B \rangle - T \varDelta S_B \ge 0. \end{aligned}$$

(37)

This gives us,

$$\begin{aligned} \langle Q \rangle = - \varDelta \langle H_B \rangle \le -T \varDelta S_B. \end{aligned}$$

(38)

From (35),

$$\begin{aligned} - \varDelta S_B \le \varDelta S_A, \end{aligned}$$

(39)

which, combined with (38), yields,

$$\begin{aligned} \langle Q \rangle \le T \varDelta S_A, \end{aligned}$$

(40)

or,

$$\begin{aligned} \frac{\langle Q \rangle }{T} \le \varDelta S_A, \end{aligned}$$

(41)

which is the desired result.

1.2 Some Quotations from the History of $\Theta \Delta ^{\text{cs}}$

The science that I am calling $\Theta \Delta^{\text{cs}}$ is not a new idea. This understanding of the basic concepts of thermodynamics has been present from the very beginning of the subject. In this appendix I provide some relevant quotations, with no pretense to exhaustiveness.

Josiah Willard Gibbs (1875). Part of this has already been quoted above; here is a fuller quotation.

When we say that when two different gases mix by diffusion as we have supposed, the energy of the whole remains constant, and the entropy receives a certain increase, we mean that the gases could be separated and brought to the same volume and temperature which they had at first by means of a certain change in external bodies, for example, by the passages of a certain amount of heat from a warmer to a colder body. But when we say that when two gas-masses of the same kind are mixed under similar circumstances there is no change of energy or entropy, we do not mean that the gases which have been mixed can be separated without change to external bodies. On the contrary, the separation of the gases is entirely impossible. We call the energy and entropy of the gas-masses when mixed the same as when they were unmixed, because we do not recognize any difference in the substance of the two masses. So when gases of different kinds are mixed, if we ask what changes in external bodies are necessary to bring the system to its original state, we do not mean a state in which each particle shall occupy more or less exactly the same position as at some previous epoch, but only a state which shall be undistinguishable from the previous one in its sensible properties. It is to states of systems thus incompletely defined that the problems of thermodynamics relate.

But if such considerations explain why the mixture of gas-masses of the same kind stands on a different footing from the mixture of gas-masses of different kinds, the fact is not less significant that the increase of entropy due to the mixture of gases of different kinds, in such a case as we have supposed, is independent of the nature of the gases.

Now we may without violence to the general laws of gases which are embodied in our equations suppose other gases to exist than such as actually do exist, and there does not appear to be any limit to the resemblance which there might be between two such kinds of gas. But the increase of entropy due to the mixing of given volumes of the gases at a given temperature and pressure would be independent of the degree of similarity or dissimilarity between them. We might also imagine the case of two gases which should be absolutely identical in all the properties (sensible and molecular) which come into play while they exist as gases either pure or mixed with each other, but which should differ in respect to the attractions between their atoms and the atoms of some other substances, and therefore in their tendency to combine with other substances. In the mixture of such gases by diffusion an increase of entropy would take place, although the process of mixture, dynamically considered, might be absolutely identical in its minutest details (even with respect to the precise path of each atom) with processes which might take place without any increase of entropy. In such respects, entropy stands strongly contrasted with energy ([19], pp. 228–229; in [20], pp. 166–167).

Rudolf Clausius (1877). Responding to Tait’s (unfair) charge that the possibility of a demon that could, without expenditure of work, cool a body below the temperature of its surroundings “is absolutely fatal to Clausius’ reasoning,” ([46], pp. 118–120; see also [47], p. 37), Clausius wrote,

Dieses kann ich in keiner Weise zugeben. Wenn die Wärme als eine Molecularbewegung betrachtet wird, so ist dabei zu bedenken, dass die Molecüle so kleine Körpertheilchen sind, dass es für uns unmöglich ist, sie einzeln wahrzunehmen. Wir können daher nicht auf einzelne Molecüle für sich allein wirken, oder die Wirkungen einzelner Molecüle für sich allein erhalten, sondern haben es bei jeder Wirkung, welche wir auf einen Körper ausüben oder von ihm erhalten, gleichzeitig mit einer ungeheuer grossen Menge von Molecülen zu thun, welche sich nach allen möglichen Richtungen und mit allein überhaupt bei den Molecülen vorkommenden Geschwindigkeiten bewegen, und sich an der Wirkung in der Weise gleichmässig betheiligen, dass nur zufällige Verschiedenheiten vorkommen, die den allgemeinen Gesetzen der Wahrscheinlichkeit unterworfen sind. Dieser Umstand bildet gerade die charakteristische Eigenthümlichkeit derjenigen Bewegung, welche wir Wärme nennen, und auf ihm beruhen die Gesetze, welche das Verhalten der Wärme von dem anderer Bewegungen unterscheiden.

Wenn nun Dämonen eingreifen, und diese charakteristische Eigenthümlichkeit zerstören, indem sie unter den Molecülen einen Unterschied machen, und Molecülen von gewissen Geschwindigkeiten den Durchgang durch eine Scheidewand gestatten, Molecülen von anderen Geschwindigkeiten dagegen den Durchgang verwehren, so darf man das, was unter diesen Umständen geschieht, nicht mehr als eine Wirkung der Wärme ansehen und erwarten, dass es mit den für die Wirkungen der Wärme geltenden Gesetzen übereinstimmt ([48], p. 32).

This I can in no way concede. If heat is regarded as a molecular motion, it should be remembered that the molecules are parts of bodies that are so small that it is impossible for us to perceive them individually. We can therefore not act on single molecules by themselves, or obtain effect from individual molecules by themselves, but rather, in every action that we exert on a body or receive from it, we have simultaneously to do with an immensely large collection of molecules, which move in all possible directions and with all the speeds occurring among the molecules, and participate in the action uniformly, in such a way that there occur only random differences, which are subject to the general laws of probability. This circumstance forms precisely the characteristic property of that motion which we call heat, and on it depends the laws that distinguish the behavior of heat from that of other motions.

If now demons intervene, and disturb this characteristic property by distinguishing between the molecules, and molecules of certain speeds are permitted passage through a partition, molecules of other speeds refused passage, then one may no longer regard what happens under these conditions as an action of heat and expect it to agree with the laws valid for the action of heat.

James Clerk Maxwell (1877, 1878).

Available energy is energy which we can direct into any desired channel. Dissipated energy is energy we cannot lay hold of and direct at pleasure, such as the energy of the confused agitation of molecules which we call heat. Now, confusion, like the correlative term order, is not a property of material things in themselves, but only in relation to the mind which perceives them. A memorandum-book does not, provided it is neatly written, appear confused to an illiterate person, or to the owner who understands thoroughly, but to any other person able to read it appears to be inextricably confused. Similarly the notion of dissipated energy could not occur to a being who could not turn any of the energies of nature to his own account, or to one who could trace the motion of every molecule and seize it at the right moment. It is only to a being in the intermediate stage, who can lay hold of some forms of energy while others elude his grasp, that energy appears to be passing inevitably from the available to the dissipated state ([49], p. 221, in [50], p. 646).

The second law relates to that kind of communication of energy which we call the transfer of heat as distinguished from another kind of communication of energy which we call work. According to the molecular theory the only difference between these two kinds of communication of energy is that the motions and displacements which are concerned in the communication of heat are those of molecules, and are so numerous, so small individually, and so irregular in their distribution, that they quite escape all our methods of observation; whereas when the motions and displacements are those of visible bodies consisting of great numbers of molecules moving altogether, the communication of energy is called work.

Hence we have only to suppose our senses sharpened to such a degree that we could trace the motions of molecules as easily as we now trace those of large bodies, and the distinction between work and heat would vanish, for the communication of heat would be seen to be a communication of energy of the same kind as that which we call work. ([51], p. 279, in [50], p. 669).

John von Neumann (1929).

If we take into account that the observer can measure only macroscopically then we find different entropy values (in fact, greater ones, as the observer is now less skilful and possibly can therefore extract less mechanical work from the system) .... ([52], p. 214, from [53], p. 47).

Harold Grad (1961).

Whether or not a diffusion occurs when a barrier is removed depends not on a difference in physical properties of the two substances but on a decision that we are or are not interested in such a difference (which is what governs the choice of an entropy function) ...A very illuminating example is given by the “spin-echo” effect. In this experiment, it is found that it is possible to produce a highly ordered microscopic state and, at a later time, effectively reverse all velocities. To a person who has access to such equipment, a very high level “reversible” entropy will be appropriate; to one who has not, a lower order entropy will properly describe all phenomena ([54], pp. 326–327)

Nicolaas Godfried van Kampen (1984). In regards to the difference in expression of entropies for a uniform sample of gas and a system composed of two different gases, van Kampen wrote,

The origin of the difference is that two different processes had to be chosen for extending the definition of entropy. They are mutually exclusive; the first one cannot be used for two different gases and the second one does not apply to a single gas. But suppose that A and B are so similar that the experimenter has no physical way of distinguishing between them. Then he does not have the semi-permeable walls needed for the second process, but on the other hand the first will look reversible to him. ...The point is, that this is perfectly justified and that he will not be led to any wrong results. If you tell him that ‘actually’ the entropy increased when he opened the channel he will answer that this is a useless statement since he cannot utilize the entropy increase for running a machine. The entropy increase is no more physical to him than the one that could be manufactured by taking a single gas and mentally tagging the molecules by A or B.

In fact, this still holds when the experimenter would be able to distinguish between A and B, by means of a mass spectrograph for instance, but is not interested in the difference because it is not relevant for his purpose. This is precisely what engineers do when they make tables of the entropy of steam, ignoring the fact that it is actually a mixture of normal and heavy water. Thus, whether such a process is reversible or not depends on how discriminating the observer is. The expression for the entropy (which one constructs by one or the other processes mentioned above) depends on whether he is able and willing to distinguish between the molecules A and B. This is a paradox only for those who attach more physical reality to the entropy than is implied by its definition ([55], pp. 306–307).

Edward T. Jaynes (1992).

In the first place, it is necessary to decide at the outset of a problem which macroscopic variables or degrees of freedom we shall measure and/or control; and within the context of the thermodynamic system thus defined, entropy will be some function $S(X_1,\ldots , X_n)$ of whatever variables we have chosen. We expect this to obey the second law $T dS \ge dQ$ only as long as all experimental manipulations are confined to that chosen set. If someone, unknown to us, were to vary a macrovariable $X_{n+1}$ outside that set, he could produce what would appear to us as a violation of the second law, since our entropy function $S(X_1,\ldots , X_n)$ might decrease spontaneously, while his $S(X_1,\ldots , X_n, X_{n+1})$ increases ([56], p. 5).

John Goold, Marcus Huber, Arnau Riera, Lídia del Rio, and Paul Skrzypczyk (2016).

If physical theories were people, thermodynamics would be the village witch. Over the course of three centuries, she smiled quietly as other theories rose and withered, surviving major revolutions in physics, like the advent of general relativity and quantum mechanics. The other theories find her somewhat odd, somehow different in nature from the rest, yet everyone comes to her for advice, and no-one dares to contradict her. Einstein, for instance, called her ‘the only physical theory of universal content, which I am convinced, that within the framework of applicability of its basic concepts will never be overthrown.’

Her power and resilience lay mostly on her frank intentions: thermodynamics has never claimed to be a means to understand the mysteries of the natural world, but rather a path towards efficient exploitation of said world. She tells us how to make the most of some resources, like a hot gas or a magnetized metal, to achieve specific goals, be them moving a train or formatting a hard drive. Her universality comes from the fact that she does not try to understand the microscopic details of particular systems. Instead, she only cares to identify which operations are easy and hard to implement in those systems, and which resources are freely available to an experimenter, in order to quantify the cost of state transformations ([7], pp. 1–2).