An explicit derivation of the time–information uncertainty relation in thermodynamics

The evolution of entropy and related uncertainty relations are of great importance in nonequilibrium thermodynamics and statistical mechanics [1–3]. Nicholson et al have recently discovered a time–information uncertainty relation in thermodynamics [4]:

$$\begin{eqnarray}&&\left|\dot{S}\right|/{k}_{B}\leqslant {\rm{\Delta }}\dot{I}{\rm{\Delta }}I.\end{eqnarray} \tag{ 1 }$$

S is the Shannon entropy:

$$\begin{eqnarray}&&S/{k}_{B}=-\displaystyle \sum {p}_{x}\,\mathrm{ln}\,{p}_{x},\end{eqnarray} \tag{ 2 }$$

where k_B is the Boltzmann constant, p_x is the probability of the state x (= 1, 2, ... , N), and the simplified symbol of summation in this note denotes:

$$\begin{eqnarray}&&\displaystyle \sum {a}_{x}\equiv \displaystyle \sum _{x=1}^{N}{a}_{x},\end{eqnarray} \tag{ 3 }$$

where a_x is a general variable. $\dot{S}$ is the evolution rate or time derivative of the entropy:

$$\begin{eqnarray}&&\dot{S}\equiv \displaystyle \frac{dS}{dt}.\end{eqnarray} \tag{ 4 }$$

ΔI is the standard deviation of the surprisal or information content I_x :

$$\begin{eqnarray}&&{I}_{x}=-\,\mathrm{ln}\,{p}_{x},\end{eqnarray} \tag{ 5 }$$

and ${\rm{\Delta }}\dot{I}$ is the standard deviation of the evolution rate of the surprisal [5–7]. Because p_x is a stochastic variable, other statistical variables such as S, I_x , and their derivatives are also stochastic. The relation of equation (1) provides an upper bound of the entropy evolution rate in a system, and thus is positioned as a milestone in multiple fields including informatics, nonequilibrium thermodynamics, and energy science and engineering. The time–information uncertainty relation and the associated speed limit for flows of heat and entropy were validated with a number of examples [4]. This uncertainty applies to various systems ranging from energy transducers [8–10] to consciousness neuroinformatics [11–13]. In this short note, we present a more explicit derivation of the time–information uncertainty relation of equation (1), solely using the most primitive variable, p_x , as a supplement of the original article by Nicholson et al [4], for the convenience of the community.

Let us start with the formulation of the standard deviation of the surprisal and of the surprisal evolution rate. From equation (5),

$$\begin{eqnarray}&&\begin{array}{l}{\rm{\Delta }}I=\sqrt{\displaystyle \sum {p}_{x}{\left({I}_{x}-\left\langle I\right\rangle \right)}^{2}}\\ \,\,=\,\sqrt{\displaystyle \sum {p}_{x}{\left(-\mathrm{ln}{p}_{x}+\displaystyle \sum {p}_{x}\mathrm{ln}{p}_{x}\right)}^{2}},\end{array}\end{eqnarray} \tag{ 6 }$$

where the symbol of expectation denotes:

$$\begin{eqnarray}&&\left\langle a\right\rangle \equiv \displaystyle \sum {p}_{x}{a}_{x}.\end{eqnarray} \tag{ 7 }$$

$$\begin{eqnarray}&&\begin{array}{l}{\rm{\Delta }}\dot{I}=\sqrt{\displaystyle \sum {p}_{x}{\left({\dot{I}}_{x}-\left\langle \dot{I}\right\rangle \right)}^{2}}\\ \,\,=\sqrt{\displaystyle \sum {p}_{x}{\left(-{\dot{p}}_{x}/{p}_{x}+\displaystyle \sum {\dot{p}}_{x}\right)}^{2}},\end{array}\end{eqnarray} \tag{ 8 }$$

since

$$\begin{eqnarray}&&{\dot{I}}_{x}=\displaystyle \frac{d}{dt}\left(-\,\mathrm{ln}\,{p}_{x}\right)=-{\dot{p}}_{x}/{p}_{x}.\end{eqnarray} \tag{ 9 }$$

Next, let us move onto the entropy evolution rate. From equation (2),

$$\begin{eqnarray}&&\begin{array}{l}\dot{S}/{k}_{B}=\displaystyle \frac{d}{dt}\left(-\displaystyle \sum {p}_{x}\,\mathrm{ln}\,{p}_{x}\right)\\ \,\,\,=-\displaystyle \sum {\dot{p}}_{x}\,\mathrm{ln}\,{p}_{x}-\displaystyle \sum {p}_{x}\displaystyle \frac{d}{dt}\left(\mathrm{ln}\,{p}_{x}\right)\\ \,\,\,=-\displaystyle \sum {\dot{p}}_{x}\,\mathrm{ln}\,{p}_{x}-\displaystyle \sum {\dot{p}}_{x}.\end{array}\end{eqnarray} \tag{ 10 }$$

Because $\displaystyle \sum {p}_{x}=1,$ $\displaystyle \sum {\dot{p}}_{x}=\displaystyle \frac{d}{dt}\displaystyle \sum {p}_{x}=0.$ This fact allows for a trick to multiply the second term of equation (10) by $-\displaystyle \sum {p}_{x}\,\mathrm{ln}\,{p}_{x}$ (=$S/{k}_{B}$):

$$\begin{eqnarray}&&\dot{S}/{k}_{B}=-\displaystyle \sum {\dot{p}}_{x}\,\mathrm{ln}\,{p}_{x}+\left(\displaystyle \sum {\dot{p}}_{x}\right)\left(\displaystyle \sum {p}_{x}\,\mathrm{ln}\,{p}_{x}\right).\end{eqnarray} \tag{ 11 }$$

From equation (11), by further tricky transformations,

$$\begin{eqnarray}&&\begin{array}{l}\dot{S}/{k}_{B}=-\displaystyle \sum {\dot{p}}_{x}\,\mathrm{ln}\,{p}_{x}+\left(\displaystyle \sum {\dot{p}}_{x}\right)\left(\displaystyle \sum {p}_{x}\,\mathrm{ln}\,{p}_{x}\right)\\ \,\,+\left(\displaystyle \sum {\dot{p}}_{x}\right)\left(\displaystyle \sum {p}_{x}\,\mathrm{ln}\,{p}_{x}\right)-\left(\displaystyle \sum {\dot{p}}_{x}\right)\left(\displaystyle \sum {p}_{x}\,\mathrm{ln}\,{p}_{x}\right)\\ \,\,=-\displaystyle \sum {\dot{p}}_{x}\,\mathrm{ln}\,{p}_{x}+\left(\displaystyle \sum {\dot{p}}_{x}\right)\left(\displaystyle \sum {p}_{x}\,\mathrm{ln}\,{p}_{x}\right)\\ \,\,+\left(\displaystyle \sum {p}_{x}\,\mathrm{ln}\,{p}_{x}\right)\left(\displaystyle \sum {\dot{p}}_{x}\right)-\displaystyle \sum {p}_{x}\left\{\left(\displaystyle \sum {\dot{p}}_{x}\right)\left(\displaystyle \sum {p}_{x}\,\mathrm{ln}\,{p}_{x}\right)\right\}\\ \,\,=-\displaystyle \sum \left\{{\dot{p}}_{x}\,\mathrm{ln}\,{p}_{x}-{\dot{p}}_{x}\displaystyle \sum {p}_{x}\,\mathrm{ln}\,{p}_{x}-{p}_{x}\,\mathrm{ln}\,{p}_{x}\displaystyle \sum {\dot{p}}_{x}+{p}_{x}\left(\displaystyle \sum {\dot{p}}_{x}\right)\left(\displaystyle \sum {p}_{x}\,\mathrm{ln}\,{p}_{x}\right)\right\}\\ \,\,=-\displaystyle \sum {p}_{x}\left(-{\dot{p}}_{x}/{p}_{x}+\displaystyle \sum {\dot{p}}_{x}\right)\left(-\,\mathrm{ln}\,{p}_{x}+\displaystyle \sum {p}_{x}\,\mathrm{ln}\,{p}_{x}\right).\end{array}\end{eqnarray} \tag{ 12 }$$

For the transformation across the second equality sign in equation (12), a general property $A=\displaystyle \sum {p}_{x}A,$ where A is a constant, was used. Using the Cauchy–Schwarz inequality ${\left(\displaystyle \sum {a}_{x}{b}_{x}\right)}^{2}\leqslant \left(\displaystyle \sum {{a}_{x}}^{2}\right)\left(\displaystyle \sum {{b}_{x}}^{2}\right),$ where b_x is a general variable, from equation (12),

$$\begin{eqnarray}&&\begin{array}{l}{\left(\dot{S}/{k}_{B}\right)}^{2}={\left\{\displaystyle \sum {p}_{x}\left(-{\dot{p}}_{x}/{p}_{x}+\displaystyle \sum {\dot{p}}_{x}\right)\left(-\mathrm{ln}{p}_{x}+\displaystyle \sum {p}_{x}\mathrm{ln}{p}_{x}\right)\right\}}^{2}\\ \,\,\,\leqslant \left\{{\displaystyle \sum {p}_{x}\left(-{\dot{p}}_{x}/{p}_{x}+\displaystyle \sum {\dot{p}}_{x}\right)}^{2}\right\}\left\{\displaystyle \sum {p}_{x}{\left(-\mathrm{ln}{p}_{x}+\displaystyle \sum {p}_{x}\mathrm{ln}{p}_{x}\right)}^{2}\right\}\\ \,\,\,={\left({\rm{\Delta }}\dot{I}\right)}^{2}{\left({\rm{\Delta }}I\right)}^{2},\end{array}\end{eqnarray} \tag{ 13 }$$

via equations (6) and (8). Hence, $\left|\dot{S}\right|/{k}_{B}\leqslant {\rm{\Delta }}\dot{I}{\rm{\Delta }}I$ (equation (1)).

Data availability statement