their properties,in particular,convergence of ergodic averages and central limit theorems.In one of the discussions of that paper,Chan and Geyer(1994)were able to relax a condition on Tierney's Central Limit Theorem,and this new condition plays an important role in research today (see Section 5.4).A pair of very influential,and innovative papers is the work of Liu et al.(1994,1995),who very carefully analyzed the covariance structure of Gibbs sampling, and were able to formally establish the validity of Rao-Blackwellization in Gibbs sampling. Gelfand and Smith(1990)had used Rao-Blackwellization,but it was not justified at that time,as the original theorem was only applicable to iid sampling,which is not the case in MCMC.Other early theoretical developments include the Duality Theorem of Diebolt and Robert (1994),who showed that in the two-stage Gibbs sampler (which is equivalent to the Data Augmentation algorithm of Tanner and Wong 1987),convergence properties of one chain can be transferred to other chains,a fact also found in Liu et al.(1994,1995). This turns out to be particularly important in mixture models,where it is typical that one part of the Gibbs chain is discrete and finite,and the other is continuous.The convergence properties of the finite chain carry over to the continuous chain. Another paper must be singled out,namely Mengersen and Tweedie(1996),for setting the tone for the study of the speed of convergence of MCMC algorithms to the target distribution. Subsequent works in this area by Richard Tweedie,Gareth Roberts,Jeff Rosenthal and co- authors are too numerous to be mentioned here,even though the paper Roberts et al. (1997)must be quoted for setting explicit targets on the acceptance rate of the random walk Metropolis-Hastings algorithm,as well as Roberts and Rosenthal (1999)for getting an upper bound on the number of iterations(523)needed to approximate the target up to 1% by a slice sampler.The unfortunate demise of Richard Tweedie in 2001 alas had a major impact on the book about MCMC convergence he was contemplating with Gareth Robert. One pitfall arising from the widespread use of Gibbs sampling was the tendency to spec- ify models only through their conditional distributions,almost always without referring to the positivity conditions in Section 3.Unfortunately,it is possible to specify a perfectly legitimate-looking set of conditionals that do not correspond to any joint distribution,and the resulting Gibbs chain cannot converge.Hobert and Casella(1996)were able to document the conditions needed for a convergent Gibbs chain,and alerted the Gibbs community to this problem(which only arises if improper priors are used,but this is a frequent occurrence). Much other work followed,and continues to grow today.Geyer and Thompson (1995) describe how to put a "ladder"of chains together to have both"hot"and "cold"exploration, followed by Neal's 1996 introduction of tempering;Athreya et al.(1996)gave more easily verifiable conditions for convergence;Meng and van Dyk (1999)and Liu and Wu (1999) developed the theory of parameter expansion in the Data Augmentation algorithm,leading 11their properties, in particular, convergence of ergodic averages and central limit theorems. In one of the discussions of that paper, Chan and Geyer (1994) were able to relax a condition on Tierney’s Central Limit Theorem, and this new condition plays an important role in research today (see Section 5.4). A pair of very influential, and innovative papers is the work of Liu et al. (1994, 1995), who very carefully analyzed the covariance structure of Gibbs sampling, and were able to formally establish the validity of Rao-Blackwellization in Gibbs sampling. Gelfand and Smith (1990) had used Rao-Blackwellization, but it was not justified at that time, as the original theorem was only applicable to iid sampling, which is not the case in MCMC. Other early theoretical developments include the Duality Theorem of Diebolt and Robert (1994), who showed that in the two-stage Gibbs sampler (which is equivalent to the Data Augmentation algorithm of Tanner and Wong 1987), convergence properties of one chain can be transferred to other chains, a fact also found in Liu et al. (1994, 1995). This turns out to be particularly important in mixture models, where it is typical that one part of the Gibbs chain is discrete and finite, and the other is continuous. The convergence properties of the finite chain carry over to the continuous chain. Another paper must be singled out, namely Mengersen and Tweedie (1996), for setting the tone for the study of the speed of convergence of MCMC algorithms to the target distribution. Subsequent works in this area by Richard Tweedie, Gareth Roberts, Jeff Rosenthal and co￾authors are too numerous to be mentioned here, even though the paper Roberts et al. (1997) must be quoted for setting explicit targets on the acceptance rate of the random walk Metropolis–Hastings algorithm, as well as Roberts and Rosenthal (1999) for getting an upper bound on the number of iterations (523) needed to approximate the target up to 1% by a slice sampler. The unfortunate demise of Richard Tweedie in 2001 alas had a major impact on the book about MCMC convergence he was contemplating with Gareth Robert. One pitfall arising from the widespread use of Gibbs sampling was the tendency to spec￾ify models only through their conditional distributions, almost always without referring to the positivity conditions in Section 3. Unfortunately, it is possible to specify a perfectly legitimate-looking set of conditionals that do not correspond to any joint distribution, and the resulting Gibbs chain cannot converge. Hobert and Casella (1996) were able to document the conditions needed for a convergent Gibbs chain, and alerted the Gibbs community to this problem (which only arises if improper priors are used, but this is a frequent occurrence). Much other work followed, and continues to grow today. Geyer and Thompson (1995) describe how to put a “ladder” of chains together to have both “hot” and “cold” exploration, followed by Neal’s 1996 introduction of tempering; Athreya et al. (1996) gave more easily verifiable conditions for convergence; Meng and van Dyk (1999) and Liu and Wu (1999) developed the theory of parameter expansion in the Data Augmentation algorithm, leading 11