Statistical Seience 1996.VoL.11.N0.3.189-228 Bootstrap Confidence Intervals Thomas J.DiCiccio and Bradley Efron Abstract.This article surveys bootstrap methods for producing good approximate confidence intervals.The goal is to improve by an order of magnitude upon the accuracy of the standard intervals6+z(),in a way that allows routine application even to very complicated problems. Both theory and examples are used to show how this is done.The first seven sections provide a heuristic overview of four bootstrap confidence interval procedures:BCa,bootstrap-t,ABC and calibration.Sections 8 and 9 describe the theory behind these methods,and their close connec- tion with the likelihood-based confidence interval theory developed by Barndorff-Nielsen,Cox and Reid and others. Key words and phrases:Bootstrap-t,BCa and ABC methods,calibra- tion,second-order accuracy 1.INTRODUCTION ate,2(0.95)=1.645 and so on.Often,and always in Confidence this paper,6 and a are obtained by maximum like- intervals have become familiar lihood theory. friends in the applied statistician's collection of data-analytic tools.They combine point estima- The standard intervals,as implemented by maxi- tion and hypothesis testing into a single inferen- mum likelihood theory,are a remarkably useful tool tial statement of great intuitive appeal.Recent The method is completely automatic:the statisti- advances in statistical methodology allow the con- cian inputs the data,the class of possible probabil- struction of highly accurate approximate confidence ity models and the parameter of interest;a com- intervals,even for very complicated probability puter algorithm outputs the intervals(1.1),with no models and elaborate data structures.This article further intervention required.This is in notable con- trast to the construction of an exact interval,which discusses bootstrap methods for constructing such intervals in a routine,automatic way. requires clever thought on a problem-by-problem Two distinct approaches have guided confidence basis when it is possible at all. interval construction since the 1930's.A small cata- The trouble with standard intervals is that they logue of exact intervals has been built up for special are based on an asymptotic approximation that can situations.like the ratio of normal means or a sin- be quite inaccurate in practice.The example below gle binomial parameter.However,most confidence illustrates what every applied statistician knows, that (1.1)can considerably differ from exact inter- intervals are approximate,with by far the favorite vals in those cases where exact intervals exist.Over approximation being the standard interval the years statisticians have developed tricks for im- (1.1) 0±za)6. proving(1.1),involving bias-corrections and param- eter transformations.The bootstrap confidence Here 6 is a point estimate of the parameter of in- intervals that we will discuss here can be thought terest 0,is an estimate of 0's standard deviation, of as automatic algorithms for carrying out these and z()is the 100ath percentile of a normal devi- improvements without human intervention.Of course they apply as well to situations so compli- Thomas J.DiCiccio is Assocciate Professor,De- cated that they lie beyond the power of traditional partment of Social Statistics,358 Ives Hall,Cor- analysis. nell University,Ithaca,New York 14853-3901 We begin with a simple example,where we can (email:tidg@cornell.edu).Bradley Efron is Pro- compute the bootstrap methods with an exact inter- fessor,Department of Statistics and Department val.Figure 1 shows the cd4 data:20 HIV-positive of Health Research and Policy,Stanford Uni- subjects received an experimental antiviral drug; versity,Stanford,California 94305-4065 (e-mail: cd4 counts in hundreds were recorded for each sub- brad@playfair.stanford.edu). ject at baseline and after one year of treatment,giv- 189Statistical Science 1996, Vol. 11, No. 3, 189–228 Bootstrap Confidence Intervals Thomas J. DiCiccio and Bradley Efron Abstract. This article surveys bootstrap methods for producing good approximate confidence intervals. The goal is to improve by an order of magnitude upon the accuracy of the standard intervals θˆ ± z α‘σˆ , in a way that allows routine application even to very complicated problems. Both theory and examples are used to show how this is done. The first seven sections provide a heuristic overview of four bootstrap confidence interval procedures: BCa , bootstrap-t, ABC and calibration. Sections 8 and 9 describe the theory behind these methods, and their close connec￾tion with the likelihood-based confidence interval theory developed by Barndorff-Nielsen, Cox and Reid and others. Key words and phrases: Bootstrap-t, BCa and ABC methods, calibra￾tion, second-order accuracy 1. INTRODUCTION Confidence intervals have become familiar friends in the applied statistician’s collection of data-analytic tools. They combine point estima￾tion and hypothesis testing into a single inferen￾tial statement of great intuitive appeal. Recent advances in statistical methodology allow the con￾struction of highly accurate approximate confidence intervals, even for very complicated probability models and elaborate data structures. This article discusses bootstrap methods for constructing such intervals in a routine, automatic way. Two distinct approaches have guided confidence interval construction since the 1930’s. A small cata￾logue of exact intervals has been built up for special situations, like the ratio of normal means or a sin￾gle binomial parameter. However, most confidence intervals are approximate, with by far the favorite approximation being the standard interval 1:1‘ θˆ ± z α‘σˆ : Here θˆ is a point estimate of the parameter of in￾terest θ, σˆ is an estimate of θˆ’s standard deviation, and z α‘ is the 100αth percentile of a normal devi￾Thomas J. DiCiccio is Assocciate Professor, De￾partment of Social Statistics, 358 Ives Hall, Cor￾nell University, Ithaca, New York 14853-3901 (email: tjd9@cornell.edu). Bradley Efron is Pro￾fessor, Department of Statistics and Department of Health Research and Policy, Stanford Uni￾versity, Stanford, California 94305-4065 (e-mail: brad@playfair.stanford.edu). ate, z 0:95‘ = 1:645 and so on. Often, and always in this paper, θˆ and σˆ are obtained by maximum like￾lihood theory. The standard intervals, as implemented by maxi￾mum likelihood theory, are a remarkably useful tool. The method is completely automatic: the statisti￾cian inputs the data, the class of possible probabil￾ity models and the parameter of interest; a com￾puter algorithm outputs the intervals (1.1), with no further intervention required. This is in notable con￾trast to the construction of an exact interval, which requires clever thought on a problem-by-problem basis when it is possible at all. The trouble with standard intervals is that they are based on an asymptotic approximation that can be quite inaccurate in practice. The example below illustrates what every applied statistician knows, that (1.1) can considerably differ from exact inter￾vals in those cases where exact intervals exist. Over the years statisticians have developed tricks for im￾proving (1.1), involving bias-corrections and param￾eter transformations. The bootstrap confidence intervals that we will discuss here can be thought of as automatic algorithms for carrying out these improvements without human intervention. Of course they apply as well to situations so compli￾cated that they lie beyond the power of traditional analysis. We begin with a simple example, where we can compute the bootstrap methods with an exact inter￾val. Figure 1 shows the cd4 data: 20 HIV-positive subjects received an experimental antiviral drug; cd4 counts in hundreds were recorded for each sub￾ject at baseline and after one year of treatment, giv- 189