Probability. A(very) brief histo Ionut flo What is Probability? In essence Mathematical modeling of random events and phenomena. It is fundamentally different from modeling deterministic events and functions, which constitutes the traditional study of Mathemat- However, the study of probability uses concepts and notions taken straight from Mathematics; in fact Measure Theory and Potential theory are expres- sions of abstract mathematics generalizing the theory of Probability The XVII-th century records the first documented evidence of the use of Probability Theory. More precisely in 1654 Antoine Gombaud, Chevalier de Mere, a French nobleman with an interest in gaming and gambling questions was puzzled by an apparent contradiction concerning a popular dice game The game consisted in throwing a pair of dice 24 times; the problem was to decide whether or not to bet even money on the occurrence of at least one double six"during the 24 throws. A seemingly well-established gambling rule led de mere to believe that betting on a double six in 24 throws would be profitable, but his own calculations based on many repetitions of the 24 throws indicated just the opposite. Using modern probability language, de Mere was trying to establish if such an event has probability greater than 0.5 Puzzled by this and other similar gambling problems he called the attention of the famous mathematician blaise pascal. In turn this led to an exchange of letters between Pascal and another famous French mathematician Pierre e fermat, this exchange becoming the first documented evidence of the fundamental principles of the theory of probability
Probability. A (very) brief history. Ionut Florescu What is Probability? In essence: Mathematical modeling of random events and phenomena. It is fundamentally different from modeling deterministic events and functions, which constitutes the traditional study of Mathematics. However, the study of probability uses concepts and notions taken straight from Mathematics; in fact Measure Theory and Potential theory are expressions of abstract mathematics generalizing the theory of Probability. The XVII-th century records the first documented evidence of the use of Probability Theory. More precisely in 1654 Antoine Gombaud, Chevalier de M´er´e, a French nobleman with an interest in gaming and gambling questions, was puzzled by an apparent contradiction concerning a popular dice game. The game consisted in throwing a pair of dice 24 times; the problem was to decide whether or not to bet even money on the occurrence of at least one ”double six” during the 24 throws. A seemingly well-established gambling rule led de M´er´e to believe that betting on a double six in 24 throws would be profitable, but his own calculations based on many repetitions of the 24 throws indicated just the opposite. Using modern probability language, de M´er´e was trying to establish if such an event has probability greater than 0.5. Puzzled by this and other similar gambling problems he called the attention of the famous mathematician Blaise Pascal. In turn this led to an exchange of letters between Pascal and another famous French mathematician Pierre de Fermat, this exchange becoming the first documented evidence of the fundamental principles of the theory of probability. 1
A few other particular problems on games of chance had been solved fore in the XV-th and XVI-th centuries by Italian mathematicians; however o general theory had been formulated before this famous correspondence In 1655, during his first visit to Paris, the Dutch scientist Christian Huy gens, learns of the work on probability carried out in this correspondence On his return to Holland in 1657, Huygens wrote a small work De ratiociniis in Ludo Aleae the first printed work on the calculus of probabilities. It was a treatise on problems associated with gambling. Because of the inherent appeal of games of chance, probability theory soon became popular, and the subject developed rapidly during the XVIIl-th century The major contributors during this period were Jacob Bernoulli(1654- 1705)and Abraham de Moivre(1667-1754) Jacob(Jacques)Bernoulli was a Swiss mathematician who was the first to use the term integral. He was the first mathematician in the Bernoulli family, a family of famous scientists of the XVIlI-th century. Jacob Bernoulli's most original work was Ars Conjectandi published in Basel in 1713, eight years after his death. The work was incomplete at the time of his death but it was still a work of the greatest significance in the development of the theory of probability. De moivre was a french mathematician who lived most of his life in eng land. A protestant, he was pushed to leave France after Louis XIV revoked the Edict of Nantes in 1685, leading to the expulsion of the Huguenots. De Moivre pioneered the modern approach to the theory of probability, when he published The Doctrine of Chance: A method of calculating the probabilities of events in play in 1718. The definition of statistical independence appears in this book for the first time. The Doctrine of Chance appeared in new ex- panded editions in 1718, 1738 and 1756. The birthday problem(in a slightly different form) appeared in the 1738 edition, the gambler's ruin problem the 1756 edition. The 1756 edition of The Doctrine of Chance contained what is probably de Moivre's most significant contribution to probability, namely the approximation of the binomial distribution by the normal distribution in the case of a large number of trials-which is honored by most probability textbooks as"The First Central Limit Theorem"(we will discuss this the- orem in the course of this semester). He perceives the notion of standard deviation and is the first scientist to write the normal integral A Latin version of the book had been presented earlier to the royal Society and published in the Philosophical Transactions in 1711
A few other particular problems on games of chance had been solved before in the XV-th and XVI-th centuries by Italian mathematicians; however, no general theory had been formulated before this famous correspondence. In 1655, during his first visit to Paris, the Dutch scientist Christian Huygens, learns of the work on probability carried out in this correspondence. On his return to Holland in 1657, Huygens wrote a small work De Ratiociniis in Ludo Aleae the first printed work on the calculus of probabilities. It was a treatise on problems associated with gambling. Because of the inherent appeal of games of chance, probability theory soon became popular, and the subject developed rapidly during the XVIII-th century. The major contributors during this period were Jacob Bernoulli (1654- 1705) and Abraham de Moivre (1667-1754). Jacob (Jacques) Bernoulli was a Swiss mathematician who was the first to use the term integral. He was the first mathematician in the Bernoulli family, a family of famous scientists of the XVIII-th century. Jacob Bernoulli’s most original work was Ars Conjectandi published in Basel in 1713, eight years after his death. The work was incomplete at the time of his death but it was still a work of the greatest significance in the development of the theory of probability. De Moivre was a French mathematician who lived most of his life in England. A protestant, he was pushed to leave France after Louis XIV revoked the Edict of Nantes in 1685, leading to the expulsion of the Huguenots. De Moivre pioneered the modern approach to the theory of probability, when he published The Doctrine of Chance: A method of calculating the probabilities of events in play in 17181 . The definition of statistical independence appears in this book for the first time. The Doctrine of Chance appeared in new expanded editions in 1718, 1738 and 1756. The birthday problem (in a slightly different form) appeared in the 1738 edition, the gambler’s ruin problem in the 1756 edition. The 1756 edition of The Doctrine of Chance contained what is probably de Moivre’s most significant contribution to probability, namely the approximation of the binomial distribution by the normal distribution in the case of a large number of trials - which is honored by most probability textbooks as ”The First Central Limit Theorem” (we will discuss this theorem in the course of this semester). He perceives the notion of standard deviation and is the first scientist to write the normal integral. 1A Latin version of the book had been presented earlier to the Royal Society and published in the Philosophical Transactions in 1711. 2
In Miscellanea Analytica(1730) he derives Stirlings formula(wrongly attributed to Stirling) which he uses in his proof of the central limit theorem n the second edition of the book in 1738 de moivre gives credit to Stirling for an improvement to the formula. De Moivre wrote I desisted in proceeding farther till my worthy and learned friend Mr James Stirling, who had applied after me to that inquiry discovered that c=√2 De moivre also investigated mortality statistics and the foundation of the theory of annuities. In 1724 he publishes based on population data for the city of Breslau Annuities on lives one of the first statistical applications into finance. In fact in A history of the mathematical theory of probability (London, 1865), Todhunter says that probability owes more to de Moivre than any other mathematician, with the le exception of lapa Despite de moivre's extraordinary scientific eminence his main income was as a private tutor of mathematics and he died in poverty. None of his influential friends: Leibnitz, Newton, Halley could help him find a university position. De moivre, like Cardan, is famed for predicting the day of his own death He found that he was sle 15 minutes longer each night and the arithmetic progression, calculated that he would die on the day that he slept for 24 hours. He was right The XIX-th century saw the development and generalization of the early theory. Pierre-Simon marquis de Laplace(1749-1827) publishes in 1812 Theorie analytique des Probabilites. This is the first fundamental book in probability ever published (the second being Kolmogorov's monograph from 1933). Before Laplace, probability theory was solely concerned with devel- oping a mathematical analysis of games of chance. The first edition was dedicated to Napoleon-lIe-Grand but, for obvious reasons, the dedication was removed in later editions Laplace the scientist was a genius but Laplace n was human e was put into office by Napoleon as a minister of the interior although demitted 6 weeks afterwards. and later raised to senate. He owes his olitical career to Napoleon n 1814 when it was clear that the empire was failing nt to the bourbons and as a reward he gained the title of marquis. The gesture was held in contempt by the majority of his colleagues(see Paul Luis Courier) 3
In Miscellanea Analytica (1730) he derives Stirling’s formula (wrongly attributed to Stirling) which he uses in his proof of the central limit theorem. In the second edition of the book in 1738 de Moivre gives credit to Stirling for an improvement to the formula. De Moivre wrote: ”I desisted in proceeding farther till my worthy and learned friend Mr James Stirling, who had applied after me to that inquiry, [discovered that c = √ 2].” De Moivre also investigated mortality statistics and the foundation of the theory of annuities. In 1724 he publishes based on population data for the city of Breslau Annuities on lives one of the first statistical applications into finance. In fact in A history of the mathematical theory of probability (London, 1865), Todhunter says that probability: ... owes more to [de Moivre] than any other mathematician, with the single exception of Laplace. Despite de Moivre’s extraordinary scientific eminence his main income was as a private tutor of mathematics and he died in poverty. None of his influential friends: Leibnitz, Newton, Halley could help him find a university position. De Moivre, like Cardan, is famed for predicting the day of his own death. He found that he was sleeping 15 minutes longer each night and summing the arithmetic progression, calculated that he would die on the day that he slept for 24 hours. He was right! The XIX-th century saw the development and generalization of the early theory. Pierre-Simon marquis de Laplace (1749-1827) publishes in 1812 Th´eorie Analytique des Probabilit´es. This is the first fundamental book in probability ever published (the second being Kolmogorov’s monograph from 1933). Before Laplace, probability theory was solely concerned with developing a mathematical analysis of games of chance. The first edition was dedicated to Napoleon-le-Grand but, for obvious reasons2 , the dedication was removed in later editions! 2Laplace the scientist was a genius but Laplace the man was human. He was put into office by Napoleon as a minister of the interior affairs, although demitted 6 weeks afterwards, and later raised to senate. He owes his entire political career to Napoleon. In 1814 when it was clear that the empire was failing he went to the Bourbons and as a reward he gained the title of marquis. The gesture was held in contempt by the majority of his colleagues (see Paul Luis Courier) 3
The work consisted of two books and a second edition two years later saw an increase in the material by about an extra 30 per cent. The work tudies generating functions, Laplace's definition of probability, Bayes rule (so named by Poincare many years later), the notion of mathematical ex pectation, probability approximations, a discussion of the method of least squares, Buffon's needle problem, and inverse Laplace transform. Later edi- tions of the"Theorie Analytique des Probabilites" also contains supplement which consider applications of probability to determine errors in observations arising in astronomy, the other passion of Laplace Laplace had always changed his views with the changing political events of the time, modifying his opinions to fit in with the frequent political changes which were typical of this period. Laplace became Count of the Empire in 1806 and he was named a marquis in 1817 after the restoration of the Bourbons On the morning of monday 5 March 1827 Laplace died. Few events would cause the Academy to cancel a meeting but they did so on that day as a mark of respect for one of the greatest scientists of all time. Many workers have contributed to the theory since Laplace's time; among the most important are Chebyshev, Markov, von Mises, and Kolmogorov One of the difficulties in developing a mathematical theory of probability has been to arrive at a definition of probability that is precise enough for use in mathematics, yet comprehensive enough to be applicable to a wide range of phenomena. The search for a widely acceptable definition took nearly three centuries and was marked by much controversy. The matter was finally resolved in the 20th century by treating probability theory on an axiomatic basis In 1933 a monograph by the Russian giant mathematician Andrey Niko- laevich Kolmogorov(1903-1987)outlined an axiomatic approach that form the basis for the modern theory. In 1925 the year he started his doctoral stud ies, Kolmogorov published his first paper with Khinchin on the probability theory. The paper contains among other inequalities about partial series of random variables the three series theorem which provides important tools for stochastic calculus. In 1929 when he finished his doctorate he already had published 18 papers among them versions of the strong law of large numbers and the iterated logarithm In 1933, two years after his appointment as a professor at moscow Univer- sity, Kolmogorov publishes Grundbegriffe der Wahrscheinlichkeitsrechnung his most fundamental book. In it he builds up probability theory in a rigor
The work consisted of two books and a second edition two years later saw an increase in the material by about an extra 30 per cent. The work studies generating functions, Laplace’s definition of probability, Bayes rule (so named by Poincar´e many years later), the notion of mathematical expectation, probability approximations, a discussion of the method of least squares, Buffon’s needle problem, and inverse Laplace transform. Later editions of the ”Th´eorie Analytique des Probabilit´es” also contains supplements which consider applications of probability to determine errors in observations arising in astronomy, the other passion of Laplace. Laplace had always changed his views with the changing political events of the time, modifying his opinions to fit in with the frequent political changes which were typical of this period. Laplace became Count of the Empire in 1806 and he was named a marquis in 1817 after the restoration of the Bourbons. On the morning of Monday 5 March 1827 Laplace died. Few events would cause the Academy to cancel a meeting but they did so on that day as a mark of respect for one of the greatest scientists of all time. Many workers have contributed to the theory since Laplace’s time; among the most important are Chebyshev, Markov, von Mises, and Kolmogorov. One of the difficulties in developing a mathematical theory of probability has been to arrive at a definition of probability that is precise enough for use in mathematics, yet comprehensive enough to be applicable to a wide range of phenomena. The search for a widely acceptable definition took nearly three centuries and was marked by much controversy. The matter was finally resolved in the 20th century by treating probability theory on an axiomatic basis. In 1933 a monograph by the Russian giant mathematician Andrey Nikolaevich Kolmogorov (1903-1987) outlined an axiomatic approach that forms the basis for the modern theory. In 1925 the year he started his doctoral studies, Kolmogorov published his first paper with Khinchin on the probability theory. The paper contains among other inequalities about partial series of random variables the three series theorem which provides important tools for stochastic calculus. In 1929 when he finished his doctorate he already had published 18 papers among them versions of the strong law of large numbers and the iterated logarithm. In 1933, two years after his appointment as a professor at Moscow University, Kolmogorov publishes Grundbegriffe der Wahrscheinlichkeitsrechnung his most fundamental book. In it he builds up probability theory in a rigor- 4
ous way from fundamental axioms in a way comparable with Euclids treat- ment of geometry. He gives a rigorous definition of the conditional expecta- tion which later becomes fundamental for the definition of brownian motion stochastic integration, and Mathematics of Finance.(Kolmogorov's mono- graph is available in English translation as Foundations of Probability Theory, Chelsea, New York, 1950). And he was not finished. In 1938 he publishes the paper Analytic methods in probability theory which lay the foundation work for the Markov processes, and toward a more rigurous approach to the Markov chair Kolmogorov later extended his work to study the motion of the planets and the turbulent How of air from a jet engine. In 1941 he published two pa- pers on turbulence which are of fundamental importance. In 1953 and 1954 two papers by Kolmogorov, each of four pages in length, appeared. These are on the theory of dynamical systems with applications to Hamiltonian dynam- ics. These papers mark the beginning of KAM-theory, which is named after Kolmogorov arnold and moser. Kolmogorov addressed the international Congress of Mathematicians in Amsterdam in 1954 on this topic with his important talk General theory of dynamical systems and classical mechanics He thus demonstrated the vital role of probability theory in physics. His contribution in the topology theory is of outmost importance Kolmogorov had many interests outside mathematics, in particular he as interested in the form and structure of the poetry of Pushkin Like so many other branches of mathematics, the development of proba- bility theory has been stimulated by the variety of its applications. In its turn, each advance in the theory has enlarged the scope of its influence. Mathe- matical statistics is one important branch of applied probability; other appli cations occur in such widely different fields as genetics, biology, psychology economics, finance, engineering, mechanics, optics, thermodynamics, quan- tum mechanics, computer vision, etcetc. etc.. In fact I compel the reader to find one area in today's science where no applications of the probability theory can be found For its immense success and wide variety of applications the Theory of Probability can be arguably viewed as the most important area of Mathe matics 5
ous way from fundamental axioms in a way comparable with Euclid’s treatment of geometry. He gives a rigorous definition of the conditional expectation which later becomes fundamental for the definition of Brownian motion, stochastic integration, and Mathematics of Finance. (Kolmogorov’s monograph is available in English translation as Foundations of Probability Theory, Chelsea, New York, 1950). And he was not finished. In 1938 he publishes the paper Analytic methods in probability theory which lay the foundation work for the Markov processes, and toward a more rigurous approach to the Markov Chains. Kolmogorov later extended his work to study the motion of the planets and the turbulent flow of air from a jet engine. In 1941 he published two papers on turbulence which are of fundamental importance. In 1953 and 1954 two papers by Kolmogorov, each of four pages in length, appeared. These are on the theory of dynamical systems with applications to Hamiltonian dynamics. These papers mark the beginning of KAM-theory, which is named after Kolmogorov, Arnold and Moser. Kolmogorov addressed the International Congress of Mathematicians in Amsterdam in 1954 on this topic with his important talk General theory of dynamical systems and classical mechanics. He thus demonstrated the vital role of probability theory in physics. His contribution in the topology theory is of outmost importance. Kolmogorov had many interests outside mathematics, in particular he was interested in the form and structure of the poetry of Pushkin. Like so many other branches of mathematics, the development of probability theory has been stimulated by the variety of its applications. In its turn, each advance in the theory has enlarged the scope of its influence. Mathematical statistics is one important branch of applied probability; other applications occur in such widely different fields as genetics, biology, psychology, economics, finance, engineering, mechanics, optics, thermodynamics, quantum mechanics, computer vision, etc.etc.etc.. In fact I compel the reader to find one area in today’s science where no applications of the probability theory can be found. For its immense success and wide variety of applications the Theory of Probability can be arguably viewed as the most important area of Mathematics. 5
