EXperimental Design 16.621
Experimental Design 16.621
Elements of Designing an Experiment Goal is to assess your hypothesis The main thing is to keep the main thing the main thing-Tom Curran, AFRL] The design goal for your experiment is to achieve your objective and satisfy your success criteria Your hos are the touchstones for your experimental design
Elements of Designing an Experiment • Goal is to assess your hypothesis – The main thing is to keep the main thing the main thing - [Tom Curran, AFRL] • The design goal for your experiment is to achieve your objective and satisfy your success criteria. • Your HOS are the touchstones for your experimental design
SOME FRAMING QUESTIONS Once the Hos is set we need to define in depth the path to get there Need to think through the whole process on an end-to-end basis Need to identify the hard parts Need to identify the key milestones(" mid term exams) This will include(at some level)questions such as What will we do? Why will we do it? Where will we do it? Who will help us? How will we do it? How well do we have to do it? When will we do it?
SOME FRAMING QUESTIONS • Once the HOS is set we need to define in depth the path to get there • Need to think through the whole process on an end-to-end basis • Need to identify the hard parts • Need to identify the key milestones (“mid term exams”) • This will include (at some level) questions such as: - What will we do? - Why will we do it? - Where will we do it? - Who will help us? - How will we do i t ? - How well do we have to do it? - When will we do it?
Project Specific applications of these Questions What is the best overall approach to achieve my objective? e.g., should I test children's car seats in the lab or in actual airplane environment? What are my choices of facilities? e.g., should I use a water tunnel or a wind tunnel to determine the performance of a split-tip keel? How will I measure my independent variables? e.g., Should I measure rotational speed of my airspeed indicator with a stroboscope or a shaft encoder?
Project Specific Applications of these Questions • What is the best overall approach to achieve my objective? – e.g., should I test children’s car seats in the lab or in actual airplane environment? • What are my choices of facilities? – e.g., should I use a water tunnel or a wind tunnel to determine the performance of a split-tip keel? • How will I measure my independent variables? – e.g., Should I measure rotational speed of my airspeed indicator with a stroboscope or a shaft encoder?
Observations about design Design is an inductive process Ways to learn design Study past designs that have been successful and extract patterns Look at the online models Work with a master designer Involve your advisor Do it yourself and receive feedback Basically this is the 16.621 process
Observations About Design • Design is an inductive process. • Ways to learn design – Study past designs that have been successful and extract patterns • Look at the online models – Work with a “master” designer • Involve your advisor – Do it yourself and receive feedback • Basically this is the 16.621 process
The 621 Design Process Ideal HoS COnceptual Design Detailed Design Version Version‖ VersionⅢl Overall experimental Detailed drawings and approach;“ scope dimensions; pseudo and scale” tyr pe of code; specITIc model(s), materials, equipment; make/buy software: data to be decisions: construction taken sources of methods: calibration error methods: detailled error analysis, budget and schedule
The 621 Design Process Idea! HOS Conceptual Design Detailed Design Version II Overall experimental approach; “scope and scale”; type of model(s), materials, software; data to be taken; sources of error,…. Version I Version III Detailed drawings and dimensions; pseudo code; specific equipment; make/buy decisions; construction methods; calibration methods; detailed error analysis, budget and schedule,…
Concpetual Design Process Start with an open mind. Avoid preconceived approach Brainstorm with your partner Think of multiple ways to accomplish your objective(s) Ask other people, keep your "eyes open Avoid rejecting ideas, particularly wild ones, at this stage Keep on the cusp of creative thinking Write these ideas in your notebooks Develop a set of desired attributes to use to compare your different designs, keeping your HOS in mind Sort through the different options to arrive at a best design to achieve your objecⅳve Verify your conceptual design supports your HOS Revise Hos if original set is not feasible Then proceed with detailed design
Concpetual Design Process • Start with an open mind. Avoid preconceived approach. • Brainstorm with your partner – Think of multiple ways to accomplish your objective(s) – Ask other people, keep your “eyes open” – Avoid rejecting ideas, particularly wild ones, at this stage. – Keep on the cusp of creative thinking. – Write these ideas in your notebooks. • Develop a set of desired attributes to use to compare your different designs, keeping your HOS in mind. • Sort through the different options to arrive at a best design to achieve your objective. • Verify your conceptual design supports your HOS – Revise HOS if original set is not feasible • Then proceed with “detailed design
A 621 EXample The potential strategies for creating the aerated ice are as follows Drill air holes into the ice 2 To use a mixture of crushed ice and water close to freezing point 3 Suspend small pockets of air in the ice mix (using tethered balloons 4 Construction of a specialized rig that constantly blows air through the ice mⅸ. 5 the use of soda water or carbonated water 6 A combination of these strategies The most effective method for creating aerating ice will be determined during the preliminary testing stage of the project. The method will be chosen based on the best combination of speed ease of production, and quality of ice produced Source: Bennett, W. "Simulating and Testing Ice Screw Performance in the Laboratory, 16.621 Design Proposal, Fall 2002
A 621 Example The potential strategies for creating the aerated ice are as follows: 1 Drill air holes into the ice. 2 To use a mixture of crushed ice and water close to freezing point. 3 Suspend small pockets of air in the ice mix (using tethered balloons) 4 Construction of a specialized rig that constantly blows air through the ice mix. 5 The use of soda water or carbonated water. 6 A combination of these strategies. The most effective method for creating aerating ice will be determined during the preliminary testing stage of the project. The method will be chosen based on the best combination of speed, ease of production, and quality of ice produced. Source: B ennett, W. “Simulating an d Testing Ice Screw Performance in t he Laboratory ”, 16.621 Design Proposal, Fall 2002
Fidelity of your approach An experiment is usually a model"of the"real thing It may be too complex or expensive to test the real thing It usually will be difficult to have control over the variables in a real operational environment e.g. a part task simulator will be used to test a cockpit display instead of modifying a real cockpit The fidelity"*of your(model)experiment needs to be assessed as part of the design process Lack of fidelity is one source of errors a good way to consider these errors in the design stage is to write them down assess their importance and consider mitigation strategⅰes * Fidelity: Exact correspondence with fact or with a given quality conditionoreventaccuracyhttp://dictionary.reference.com/
Fidelity of Your Approach • An experiment is usually a “model” of the “real thing”. – It may be too complex or expensive to test the real thing – It usually will be difficult to have control over the variables in a real operational environment – e.g. a part task simulator will be used to test a cockpit display instead of modifying a real cockpit. • The “fidelity”* of your (model) experiment needs to be assessed as part of the design process. • Lack of fidelity is one source of errors. • A good way to consider these errors in the design stage is to write them down, assess their importance, and consider mitigation strategies. *Fidelity: Exact correspondence with fact or with a given quality, condition, or event; accuracy. http://dictionary.reference.com/
A 621 Example a number of potential error sources in the experiment have been identified. These are summarized in table l along with a brief description of the problem and how to decrease or eliminate the error Table 1: Sources of error Source of error Description Method of mitigation steresis Shape memory alloys have a In order to decrease the effects of hysteresis effect that could ysteresis, ample cooling time will be potentially cause the actuators to given between consecutive test runs "stuck" in a particular state Cyclic Loading Depending upon the frequency and A low frequency of perturbations, and duration of the perturbations, the iodic checks of a basell actuators could experience several condition will be used to ensure that thousand cycles of heating and the actuators have not deformed over cooling throughout the run of the time Sensors Commercially available These errors will be corrected during accelerometers will have a certain data analysis by shifting and rescaling offset, and will drift with time raw data Inconsistent If the actuators start functioning In order to ensure that the parachute Release before the parachute is released or has completely inflated before a test fully inflated, errors resulting from function is sent to the actuators, a initial conditions could occur time delay will be built into the function generator based on the initial trial run of the experiment Source: Wong, B. Assessment of Shape Memory Alloys for Steered Parachutes",16.621 Design Proposal, Fall 2002
A 621 Example A nu mb e r o f potential erro r so u r c es in the experi ment have b e e n identi fied. T h ese a r e summari z e d in T able 1 along with a brief des c ription o f th e problem and ho w to d e c r e ase or eliminate the er ror. T a ble 1: Sources of Error Sou r c e o f E r r o r D esc ription M ethod of Mitigation Hyste r esis Sh ape memor y allo ys ha v e a h ysteresis effect t hat co uld p ote ntially ca use the act u ators to g et “stu c k” in a partic ular st ate In order to decrease t he effects o f h ysteresis, a mple co olin g tim e will b e giv en betwee n co nsec utive test runs C ycli c Loadin g Depe n din g up o n the freq uenc y an d d uratio n of the perturbatio ns, th e actu ators c o uld experience several th o usan d c ycles of heatin g a n d co oling t hro u g h o ut the ru n of t h e experiment A l o w freq uenc y o f p ert urbatio ns, a n d p eri o dic checks of a baseline co n dition will be used to ensure th at the act u ators have n ot deformed o ver time. Sensors Co mm ercially a vaila ble accelerometers will ha v e a cert ain o ffset, a nd will drift with tim e T h ese errors will be c orrecte d d uri n g d ata analysis b y shiftin g an d rescalin g raw data Inconsiste nt Rele ase If t he act u ators start functio nin g b efore t he parac hute is release d or fully i nflate d, errors res ultin g fro m initial co n ditio ns co uld occ ur. In order to ensure th at the parach ute h as co mpletely inflated before a test functio n is se nt t o the actu ators, a tim e d ela y will b e built into th e functio n ge nerat or based o n the initial trial r un o f the e xperiment. Source: Wo n g, B. “Assessment of Shape Memory Alloy s for Steered Parachutes ”, 16.6 21 Design Proposal, Fall 2002
