Cronograma de Actividades Diagrama de Gantt

The Gibbs entropy was already discussed in disguise through the Gibbs vari- ational principle, which was postulated in the axiomatic approach to thermo- dynamics, see section 2.2.2. The statistical mechanics of Gibbs allows us to understand and motivate this postulate.

4_{For finite phase spaces, it is actually the uniform distribution on the entire phase space.}

Figure 2.3: Gibbs (1839-1903)

The approach of Gibbs towards statistical mechanics is probably best described as the experimentalist’s approach. Suppose that the measurements in the laboratory give us information about a set of macroscopic parameters, e.g., the position and velocity profile. The question that Gibbs answers is how we can make an unbiased estimate for the entropy of the system, using only the information from the experiment. To cite Jaynes [65]:

We have to recognize the distinction between two different kinds of reasoning; deduction from the laws of physics, and human inference from whatever information you or I happen to have. Instead of asking, “What do the laws of physics require the system to do?”, which cannot be answered without knowledge of the exact microstate, Gibbs asked a more modest question, which can be answered: “What is the best guess we can make, from the partial information that we have?”

– Edwin T. Jaynes

This question resembles much the question that Shannon answered, see p.52, so it will not be surprising that there is an intimate relation between the Shannon and Gibbs entropy.

Definition

The Gibbs entropy is an extension of the Shannon entropy, corrected for the remarks which made the application of the Shannon entropy unsuitable for thermodynamics:

• The constant k in definition (2.9) is chosen equal to kB, Boltzmann’s

• For a given set of macroscopic constraints and corresponding macro- statistics ˆν on the reduced phase space ˆΓ, only distributions ρ on the phase space Γ which are compatible with these constraints are consid- ered. We write this compatibility as p(ρ) = ˆν, i.e., the projection ofρ on ˆΓ should be equal to ˆν.

Suppose we only specify a distribution of macrovalues. This means that we do not know the exact macrostate, perhaps not even initially. The statistics of macroscopic values is then given by a probability distribution ˆν(M), M ∈Γ,ˆ e.g., our best guess about the position and velocity profile. In the absence of any further information, there is a natural probability density on Γ, writtenρνˆ,

which gives equal probability to each microstate compatible with a macrostate M:

ρνˆ(x) =

ˆ ν(M(x))

|M(x)| . (2.12)

The Gibbs entropy is then defined as the Shannon entropy of this inferred phase space measure ρνˆ(x), corrected for the modifications mentioned above.

We prefer to give the Gibbs entropy in the form of the Gibbs variational principle: SG(ˆν) =−kB sup p(µ)=ˆν Z Γ µ(x) lnµ(x), (2.13) where the supremum is taken over all phase space densitiesµ with the same macrostatistics as ˆν, i.e., the projectionp(µ) ofρνˆ on ˆΓ coincides with ˆν:

µ(M) = Z

Γ

dx µ(x)δ(M(x)−M) = ˆν(M).

Solving this variational principle gives that the supremum is reached forµ=ρνˆ

(2.12). This is the most unbiased probability measure on the phase space, which is compatible with the macroscopic constraints ˆν of the experiment.

Equilibrium

The traditional ensembles from equilibrium statistical mechanics are recov- ered for suitable choices of the macroscopic variables and constraints: the microcanonical ensemble takes the total energyE and particle number N as macroscopic variables, the contraint that the energy should be fixed to the valueU gives

ˆ

ν(E) =δ(E₋U)

as the distribution on the N-particle phase space Γ. Then, the microscopic statistics (2.12) is precisely the microcanonical ensemble and the Gibbs en- tropy (2.17) reduces to the Boltzmann entropy.

The canonical ensemble is recovered by maximizing (2.13) under the constraint that the average energy takes the valueU,

Z

which gives the Gibbs distribution ˆ ν(E) =e −βE C , C= Z dE e−βE, (2.14)

where β is a Lagrange multiplier (the inverse temperature) which is fixed by the equation R dEν(E)Eˆ =U. This distribution can be extrapolated to the full microscopic phase space Γ as

ρeq(x) =

e−βH(x)

Z , Z=

Z

dx e−βH(x)_=e−βF_{, (2.15)}

whereF is the Helmholtz free energy.

In equilibrium, the Gibbs entropy coincides with the Shannon entropy SG(ρeq)≡SI(ρeq), (2.16)

fork=kB in formula (2.9), since by definition, the equilibrium distributions

(2.15) satisfy the supremum in the variational principle (2.13). As above, we will sometimes write SG(ρeq), even though the Gibbs functional (2.13) is in

principle only defined for distributions on ˆΓ.

Link with the Boltzmann entropy

By inserting equation (2.12) into the definition (2.13), the Gibbs entropy equals SG(ˆν)≡ X M ˆ ν(M) ln|M| −X M ˆ ν(M) ln ˆν(M). (2.17) Obviously, if ˆν concentrates on just one macrostateM, ˆν(M′_{) =δ}

M,M′, then SG(ˆν) = ln|M|, which is the Boltzmann entropy for that macrostate.

More generally, the first term in equation (2.17) is the expected or estimated entropy, being unsure as we are about the exact macrostate. The second term in equation (2.17) is the Shannon entropy (2.9) of the macrostatistics ˆν, and thus nonnegative. It is most often (hopefully) negligible, certainly upon dividing by the number of particles.5

5_{The Shannon entropyS}

Ifor a discrete probability distributionp1, . . . , pnis always less

thankln(n) by concavity: SI(p) =−k X j pjlnpj=k X j pjln(1/pj)6kln 0 @ X j pj pj 1 A=klnn,

see also equation (2.31) in the context of relative entropy. The Shannon entropy on the macrovariables is hence bounded bySI(ˆν)6kln(#ˆΓ), where #ˆΓ is the number of different

macroscopic states. For a macroscopic, reduced description, the latter usually does not grow exponentially withN(which is typical only for microscopic descriptions). Thus,SI(ˆν)

Link with thermodynamical entropy

In equilibrium, the Gibbs entropy coincides with the thermodynamic entropy. Inserting the canonical distribution (2.14) into the Gibbs definition (2.13) yields SG(ρeq) =−kB Z dx ρeq(x)[−βH(x)−lnZ] = 1 T(U−F).

In thermodynamics, we learn thatF =U −T S, with S the thermodynamic entropy, which yields the equivalence between both, at least in the thermody- namic limit.

Since the Gibbs entropy is maximal in equilibrium, we immediately obtain that for general distributions ˆν, constrained to the average energyU and fixed6 V andN,

SG(ˆν)6S(U, V, N), (2.18)

whereS(U, V, N) is the thermodynamic equilibrium entropy, see section 2.2.2. As explained in the previous paragraph, the equality is reached when taking ˆ

ν the equilibrium distribution (2.14).

We can go one step further: when the average energy U and the volume V are varied over a reversible path, composed of equilibrium distributions (2.15) for constant N, then the difference of Gibbs entropies between the initial distributionρi

eq and the final distributionρfeq can be written as

SG(ρfeq)−SG(ρieq) =

Z f i

δQ T ,

which is precisely the Clausius entropy (2.2). We refer to reference [64] for a detailed proof and more details.

Gibbs entropy and the second law of thermodynamics

The problem of deriving the second law of thermodynamics from a microscopic point of view and its interpretation outside the realm of equilibrium thermo- dynamics, has existed since Boltzmann’s work as early as 1872. Gibbs entropy allows us to go to the core of this problem: we will show that for the Gibbs entropy

SG(ˆν0)6SG(ˆνt), (2.19)

when starting from macrostatistics ˆν0. This is not on itself the second law of

thermodynamics, but an entropy inequality, closely related to the second law. We postpone this discussion to section 2.4.2.

6_{Of course, it is perfectly possible to constrain to an average volumehViand average}

particle numberhNi, e.g., as for the grandcanonical ensemble. The thermodynamic entropy