ERROR ANALYSIS (UNCERTAINTY ANALYSIS) 16.621 Experimental Projects Lab I
1 ERROR ANALYSIS (UNCERTAINTY ANALYSIS) 16.621 Experimental Projects Lab I
TOPICS TO BE COVERED Why do error analysis? If we don't ever know the true value, how do we estimate the error in the true value? Error propagation in the measurement chain How do errors combine?(How do they behave in general?) How do we do an end-to-end uncertainty analysis? What are ways to mitigate errors? a hypothetical dilemma (probably nothing to do with anyone in the class) When should i throw out some data that i dont like? Answer: never, but there are reasons to throw out data Backup slides: an example of an immense amount of money and effort directed at error analysis and mitigation -jet engine testing
2 TOPICS TO BE COVERED • Why do error analysis? • If we don’t ever know the true value, how do we estimate the error in the true value? • Error propagation in the measurement chain – How do errors combine? (How do they behave in general?) – How do we do an end-to-end uncertainty analysis ? – What are ways to mitigate errors? • A hypothetical dilemma (probably nothing to do with anyone in the class) – When should I throw out some data that I don’t like? – Answer: NEVER, but there are reasons to throw out data • Backup slides: an example of an immense amount of money and effort directed at error analysis and mitigation - jet engine testing
ERROR AND UNCERTAINTY In engineering the word "error, when used to describe an aspect of measurement does not necessarily carry the connotation of mistake or blunder(although it can!) Error in a measurement means the inevitable uncertainty that attends all measurements We cannot avoid errors in this sense We can ensure that they are as small as reasonably possible and that we have a reliable estimate of how small they are [Adapted from Taylor, J. R, An Introduction to Error Analysis; The study of Uncertainties in Physical Measurements
3 ERROR AND UNCERTAINTY • In engineering the word “error”, when used to describe an aspect of measurement does not necessarily carry the connotation of mistake or blunder (although it can!) • Error in a measurement means the inevitable uncertainty that attends all measurements • We cannot avoid errors in this sense • We can ensure that they are as small as reasonably possible and that we have a reliable estimate of how small they are [Adapted from Taylor, J. R, An Introduction to Error Analysis; The Study of Uncertainties in Physical Measurements]
USES OF UNCERTAINTY ANALYSIS (O Assess experimental procedure including identification of potential difficulties Definition of necessary steps Gaps Advise what procedures need to be put in place for measurement Identify instruments and procedures that control accuracy and precision Usually one, or at most a small number, out of the large set of possibilities Inform us when experiment cannot meet desired accuracy
4 USES OF UNCERTAINTY ANALYSIS (I) • Assess experimental procedure including identification of potential difficulties – Definition of necessary steps – Gaps • Advise what procedures need to be put in place for measurement • Identify instruments and procedures that control accuracy and precision – Usually one, or at most a small number, out of the large set of possibilities • Inform us when experiment cannot meet desired accuracy
USES OF UNCERTAINTY ANALYSIS () Provide the only known basis for deciding whether Data agrees with theory Tests from different facilities get engine performance)agree Hypothesis has been appropriately assessed (resolved) Phenomena measured are real Provide basis for defining whether a closure check has been achieved s continuity satisfied (does the same amount of mass go in as goes out?) Is energy conserved? Provide an integrated grasp of how to conduct the experiment [Adapted from Kline, S.J., 1985, "The Purposes of Uncertainty Analysis", ASME J Fluids Engineering, pp 153-160
5 USES OF UNCERTAINTY ANALYSIS (II) • Provide the only known basis for deciding whether: – Data agrees with theory – Tests from different facilities (jet engine performance) agree – Hypothesis has been appropriately assessed (resolved) – Phenomena measured are real • Provide basis for defining whether a closure check has been achieved – Is continuity satisfied (does the same amount of mass go in as goes out?) – Is energy conserved? • Provide an integrated grasp of how to conduct the experiment [Adapted from Kline, S. J., 1985, “The Purposes of Uncertainty Analysis”, ASME J. Fluids Engineering, pp. 153-160]
UNCERTAINTY ESTIMATES AND HYPOTHESIS ASSESSMENT g 8300 200 ss [gl
6 UNCERTAINTY ESTIMATES AND HYPOTHESIS ASSESSMENT 0 100 200 300 400 500 600 0 20 40 60 80 100 120 Mass [g] Distance [cm] 0 100 200 300 400 500 600 0 20 40 60 80 100 120 Mass [g] Distance [cm] 0 100 200 300 400 500 600 0 20 40 60 80 100 120 Mass [g] Distance [cm]
HOW DO WE DEAL WITH NOT KNOWING THE TRUE VALUE? In" alpreal situations we don 't know the true value we are looking for We need to decide how to determine the best representation of this from our measurements We need to decide what the uncertainty is in our best representation
7 HOW DO WE DEAL WITH NOT KNOWING THE TRUE VALUE? • In “all” real situations we don’t know the true value we are looking for • We need to decide how to determine the best representation of this from our measurements • We need to decide what the uncertainty is in our best representation
AN IMPLICATION OF NOT KNOWING THE TRUE VALUE We easily divided errors into precision(bias)errors and random errors when we knew what the value was The target practice picture in the next slide is an example How about if we don't know the true value? Can we, by looking at the data in the slide after this, say that there are bias errors? How do we know if bias errors exist or not?
8 AN IMPLICATION OF NOT KNOWING THE TRUE VALUE • We easily divided errors into precision (bias) errors and random errors when we knew what the value was • The target practice picture in the next slide is an example • How about if we don’t know the true value? Can we, by looking at the data in the slide after this, say that there are bias errors? • How do we know if bias errors exist or not?
A TEAM EXERCISE List the variables you need to determine in order to carry out your hypothesis assessment What uncertainties do you foresee? (Qualitative description) Are you more concerned about bias errors or random errors? What level of uncertainty in the final result do you need to assess your hy pothesis in a rigorous manner? Can you make an estimate of the level of the uncertainty in the final result? If so, what is it? If not, what additional information do you need to do this?
11 A TEAM EXERCISE • List the variables you need to determine in order to carry out your hypothesis assessment • What uncertainties do you foresee? (Qualitative description) • Are you more concerned about bias errors or random errors? • What level of uncertainty in the final result do you need to assess your hypothesis in a rigorous manner? • Can you make an estimate of the level of the uncertainty in the final result? – If so, what is it? – If not, what additional information do you need to do this?
lOW DO WE COMBINE ERRORS? Suppose we measure quantity x with an error of dx and quantity Y with an error of dy What is the error in quantity z if: Z= AX where a is a numerical constant such as T? ·Z=X+Y? ·Z=X-Y? ·Z=XY? ·z=XY? Z is a general function of many quantities?
12 HOW DO WE COMBINE ERRORS? • Suppose we measure quantity X with an error of dx and quantity Y with an error of dy • What is the error in quantity Z if: • Z = AX where A is a numerical constant such as π ? • Z = X + Y? • Z = X - Y? • Z = XY? • Z = X/Y? • Z is a general function of many quantities?