Insight into entropy Daniel F.Styer) Department of Physics.Oberlin College.Oberlin.Ohio 44074 (Received 20 September 1999;accepted 1 May 2000) What is the qualitative character of entropy?Several examples from statistical mechanics(including liquid crystal reentrant phases,two different lattice gas models,and the game of poker)demonstrate facets of this difficult question and point toward an answer.The common answer of "entropy as disorder'is regarded here as inadequate.An alternative but equally problematic analogy is "entropy as freedom.Neither simile is perfect,but if both are used cautiously and not too literally, then the combination provides considerable insight.2000 American Association of Physics Teachers. Of all the difficult concepts of classical physics-concepts found simply by counting:One counts the number W of mi- like acceleration,energy,electric field,and time-the most crostates that correspond to the given macrostate,and com- difficult is entropy.Even von Neumann'claimed that "no- putes the entropy of the macrostate through body really knows what entropy is anyway."It is concerning entropy that students are most likely to invert their familiar S=kg In W, (1) lament and say "'I can do the problems,but I just can't where k is Boltzmann's constant.Clearly,S is high for a understand the material." macrostate when many microstates correspond to that mac- The itative character of entropy has been discussed rostate,whereas it is low when few microstates correspond. widely, although often only in the restricted context of The entropy of a macrostate measures the number of ways in gases or even in the highly restricted context of noninteract- which a system can be different microscopically and yet still ing gases.The metaphoric images invoked for entropy in- be a member of the same macroscopic state.The corre- clude“disorder,”s“randomness,.”s“smoothness,'s“disper- sponding microstates are often called "accessible'for rea- sion,'and "homogeneity."In a posthumous fragment, sons relating to ergodic theory.Because ergodic theory is Gibbs4 mentionedentropy as mixed-up-ness.Images rarely discussed at the undergraduate level,I feel it best to such as these can be useful and important,but if taken too avoid that term.Other synonyms for "a microstate corre- literally they can confuse as well as enlighten,and when sponding to a macrostate''are "a microstate consistent with misused!5.16 they can result in simple error.Analogies and (or compatible with)a macrostate''or "a permissible mi- visualizations should be employed,but their limitations as crostate.''] well as their strengths must be kept firmly in mind. Note that it requires some skill and interpretation to trans- Section I of this paper serves to set the stage and fix the late this formal definition into an expression applicable to terminology by presenting the formal,mathematical defini- specific situations.For example,suppose the macrostate of tion of microcanonical entropy in statistical mechanics.(The the system is specified by an energy from E to E+AE.If the definition is extended to other ensembles in the Appendix. system in question is quantum mechanical with discrete en- Section II(Cautionary Tales")gives three examples of the ergy levels,then one must count not all quantal states with surprises nature provides when this definition is applied to mean energies in this range,nor all energy eigenstates with physical systems,and hence illustrates the difficulties in- energies in this range,but instead the number of elements of volved in seeking qualitative insight into entropy.This sur- an energy eigenbasis with eigenvalues in this range.(In other vey serves to frame the terms of debate and show why some words,one must properly count degenerate energy states.)If visualizations of entropy are not acceptable.Section IlI goes the system is a collection of N identical classical particles, to the heart of the matter by examining two versions of a whether interacting or not,then the macroscopic state may simple model system (the "lattice gas')in which the rela- be a gas,liquid,or solid,but in all cases W is the volume of tionship between microscopic configurations and macro- phase space corresponding to this energy range,divided by scopic thermodynamic states is particularly clear.Section IV (N!h).(In classical statistical mechanics,ho is an arbi- reinforces the ideas of Sec.III by applying them to the game trary constant with the dimensions of action.In quantal sta- of poker,and Sec.V draws conclusions.The ideal conclu- tistical mechanics,it takes on the value of Planck's constant. sion for this paper would be to resolve the difficulties raised The so-called delabeling factor,N!,reflects the fact that N! by producing a concise yet accurate visualization for entropy that appeals to both the gut and the intellect.I am not able to different phase space points correspond to the same physical system.These N!points all represent N particles at N given do this.But I am able to use a combination of mathematics and analogy to illuminate the character of entropy locations and with corresponding given velocities,and differ only in the labels affixed to the various particles.On a more pragmatic vein,if the factor of N!were absent,then the I.MATHEMATICAL DEFINITION OF ENTROPY resulting entropy would not be extensive.7) (MICROCANONCIAL ENSEMBLE) In statistical mechanics,many microstates may correspond IL.CAUTIONARY TALES to a single macrostate.(A macrostate is also called a "ther- modynamic state":for example,T,V,and N for the canoni- Before seeking qualitative insight into this formal defini- cal ensemble or E.I.and N for the microcanonical en- tion,we examine three situations that demonstrate just how semble.)In the microcanonical ensemble the entropy is hazardous our search for insight can be 1090 Am.J.Phys.68 (12),December 2000 http://ojps.aip.org/ajp/ 2000 American Association of Physics Teachers 1090Insight into entropy Daniel F. Styera) Department of Physics, Oberlin College, Oberlin, Ohio 44074 ~Received 20 September 1999; accepted 1 May 2000! What is the qualitative character of entropy? Several examples from statistical mechanics ~including liquid crystal reentrant phases, two different lattice gas models, and the game of poker! demonstrate facets of this difficult question and point toward an answer. The common answer of ‘‘entropy as disorder’’ is regarded here as inadequate. An alternative but equally problematic analogy is ‘‘entropy as freedom.’’ Neither simile is perfect, but if both are used cautiously and not too literally, then the combination provides considerable insight. © 2000 American Association of Physics Teachers. Of all the difficult concepts of classical physics—concepts like acceleration, energy, electric field, and time—the most difficult is entropy. Even von Neumann1 claimed that ‘‘no￾body really knows what entropy is anyway.’’ It is concerning entropy that students are most likely to invert their familiar lament and say ‘‘I can do the problems, but I just can’t understand the material.’’ The qualitative character of entropy has been discussed widely,2–13 although often only in the restricted context of gases or even in the highly restricted context of noninteract￾ing gases. The metaphoric images invoked for entropy in￾clude ‘‘disorder,’’ ‘‘randomness,’’ ‘‘smoothness,’’ ‘‘disper￾sion,’’ and ‘‘homogeneity.’’ In a posthumous fragment, Gibbs14 mentioned ‘‘entropy as mixed-up-ness.’’ Images such as these can be useful and important, but if taken too literally they can confuse as well as enlighten, and when misused15,16 they can result in simple error. Analogies and visualizations should be employed, but their limitations as well as their strengths must be kept firmly in mind. Section I of this paper serves to set the stage and fix the terminology by presenting the formal, mathematical defini￾tion of microcanonical entropy in statistical mechanics. ~The definition is extended to other ensembles in the Appendix.! Section II ~‘‘Cautionary Tales’’! gives three examples of the surprises nature provides when this definition is applied to physical systems, and hence illustrates the difficulties in￾volved in seeking qualitative insight into entropy. This sur￾vey serves to frame the terms of debate and show why some visualizations of entropy are not acceptable. Section III goes to the heart of the matter by examining two versions of a simple model system ~the ‘‘lattice gas’’! in which the rela￾tionship between microscopic configurations and macro￾scopic thermodynamic states is particularly clear. Section IV reinforces the ideas of Sec. III by applying them to the game of poker, and Sec. V draws conclusions. The ideal conclu￾sion for this paper would be to resolve the difficulties raised by producing a concise yet accurate visualization for entropy that appeals to both the gut and the intellect. I am not able to do this. But I am able to use a combination of mathematics and analogy to illuminate the character of entropy. I. MATHEMATICAL DEFINITION OF ENTROPY „MICROCANONCIAL ENSEMBLE… In statistical mechanics, many microstates may correspond to a single macrostate. ~A macrostate is also called a ‘‘ther￾modynamic state’’: for example, T, V, and N for the canoni￾cal ensemble or E, V, and N for the microcanonical en￾semble.! In the microcanonical ensemble the entropy is found simply by counting: One counts the number W of mi￾crostates that correspond to the given macrostate, and com￾putes the entropy of the macrostate through S5kB ln W, ~1! where kB is Boltzmann’s constant. Clearly, S is high for a macrostate when many microstates correspond to that mac￾rostate, whereas it is low when few microstates correspond. The entropy of a macrostate measures the number of ways in which a system can be different microscopically and yet still be a member of the same macroscopic state. @The corre￾sponding microstates are often called ‘‘accessible’’ for rea￾sons relating to ergodic theory. Because ergodic theory is rarely discussed at the undergraduate level, I feel it best to avoid that term. Other synonyms for ‘‘a microstate corre￾sponding to a macrostate’’ are ‘‘a microstate consistent with ~or compatible with! a macrostate’’ or ‘‘a permissible mi￾crostate.’’# Note that it requires some skill and interpretation to trans￾late this formal definition into an expression applicable to specific situations. For example, suppose the macrostate of the system is specified by an energy from E to E1DE. If the system in question is quantum mechanical with discrete en￾ergy levels, then one must count not all quantal states with mean energies in this range, nor all energy eigenstates with energies in this range, but instead the number of elements of an energy eigenbasis with eigenvalues in this range. ~In other words, one must properly count degenerate energy states.! If the system is a collection of N identical classical particles, whether interacting or not, then the macroscopic state may be a gas, liquid, or solid, but in all cases W is the volume of phase space corresponding to this energy range, divided by (N!h0 3N). ~In classical statistical mechanics, h0 is an arbi￾trary constant with the dimensions of action. In quantal sta￾tistical mechanics, it takes on the value of Planck’s constant. The so-called delabeling factor, N!, reflects the fact that N! different phase space points correspond to the same physical system. These N! points all represent N particles at N given locations and with corresponding given velocities, and differ only in the labels affixed to the various particles. On a more pragmatic vein, if the factor of N! were absent, then the resulting entropy would not be extensive.17! II. CAUTIONARY TALES Before seeking qualitative insight into this formal defini￾tion, we examine three situations that demonstrate just how hazardous our search for insight can be. 1090 Am. J. Phys. 68 ~12!, December 2000 http://ojps.aip.org/ajp/ © 2000 American Association of Physics Teachers 1090