MIest 16888 Today's Topics Multidisciplinary System Design variable linking Design Optimization(MSDO) Reduced-Basis Methods Approximation Methods Response Surface Approximations Lecture 19 Kriging 16Apml2004 Variable-Fidelity Models Karen willcox e Massachusetts Institute of Technology -Prof de Weck and Prof Willcox G Massachusetts Insttute of Technology .Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems Division and Dept of Aeronautics and Astronautics MIlesd Why Approximation Methods? Approximation Methods Ve have seen throughout the course the constant trade-off Recall that the analysis or simcode must be invoked between computational cost and fidelity. each time the optimizer selects a new design vector to how to (e.g CFD, FEM) Typically, hundreds( thousands)of design vectors will e analyzed throughout an optimization run from Giesing. 1998 Can use Approximation Models(Surrogate Models) for objective functions and constraints Level of MsDo If approximate models are inexpensive to evaluate, can imed MDO analyze many more design vector options without studies optimizationiteration Approximation methods provide a way to get high-fidelity Concept first introduced in structural optimization by model information throughout the optimization without the Barthelemy and Haftka, 1993 computational e Massachusetts Institute of Technology -Prof de Weck and Prof Willcox e Massachusetts Insttute of Technology. Prof de Weck and Prof Willcox Engineening Systems Division and Dept of Aeronautics and Astronautics
1 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Approximation Methods Lecture 19 16 April 2004 Karen Willcox Multidisciplinary System Multidisciplinary System Design Optimization (MSDO) Design Optimization (MSDO) 2 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Today’s Topics Today’s Topics • Design variable linking • Reduced-Basis Methods • Response Surface Approximations • Kriging • Variable-Fidelity Models 3 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Why Approximation Methods? Why Approximation Methods? We have seen throughout the course the constant trade-off between computational cost and fidelity. Level of MSDO Fidelity Level trade studies limited optimization/iteration full MDO empirical models intermediate fidelity (e.g. vortex lattice, beam theory) high fidelity (e.g. CFD,FEM) increasing difficulty from Giesing, 1998 can we do better? can the results be believed? how to implement? Approximation methods provide a way to get high-fidelity model information throughout the optimization without the computational expense. 4 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Approximation Methods Approximation Methods • Recall that the analysis or simcode must be invoked each time the optimizer selects a new design vector to try • Typically, hundreds (thousands) of design vectors will be analyzed throughout an optimization run • Can use Approximation Models (Surrogate Models) for objective functions and constraints • If approximate models are inexpensive to evaluate, can analyze many more design vector options without worrying about computational resources • Concept first introduced in structural optimization by Barthelemy and Haftka, 1993
Mlesd Approximation Methods Overview E0 Mles Design Variable Linking reduce the number of Design variable linking design variables in Not all design variables may be independent Reduced-basis methods the optimization code For example, symmetry may exist in the problem simcode analysis full optimizer uses y, same number of y=TX+y Response surface methods design variables angIng simcode analysis Define simplified X Variable-fidelity methods combine high-fidelity and approximation models rxn x1 @ Massachusetts Institute of Technology -Prof de Weck and Prof Willcox G Massachusetts Insttute of Technology .Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems Division and Dept of Aeronautics and Astronautics Mesd Reduced-Basis Methods Reduced-Basis Methods Consider r feasible design vectors: x1, x2,.,x' We can now optimize J(x)by finding the optimal values for the coefficients ap Ve could consider the desired design to be a linear dimension n dimension r combination of these basis vectors do one full-order evaluation of resulting answe approach is efficient if r<< n will give the true optimum only if x* lies in the span of x) added fo basis vectors could be scalar vector generality previous designs coefficient lutions over a particular range(DoE) derived in some other way( POD) e Massachusetts Institute of Technology- Prof de Weck and Prof Willcox etts Institute of Technology .Prof de Weck and Prof Willcox Engineening Systems Division and Dept of Aeronautics and Astronautics ngineering Systems Divsion and Dept of Aeronautics and Astronautics
5 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Approximation Methods Overview Design variable linking Reduced-basis methods • reduce the number of design variables in the optimization code • simcode analysis full order Response surface methods Kriging • same number of design variables • simcode analysis simplified Variable-fidelity methods • combine high-fidelity and approximation models 6 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Design Variable Linking Design Variable Linking • Not all design variables may be independent • For example, symmetry may exist in the problem Define: C = + ⎡ ⎤ ⎢ ⎥ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ = ⎢ ⎥ + ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎢ ⎥ ⎣ ⎦ y Tx y r ×1 n ×1 r ×n r ×1 optimizer uses y, but provides x to simcode for analysis y T x yC 7 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Reduced Reduced-Basis Methods Basis Methods Consider r feasible design vectors: x1, x2, ..., xr We could consider the desired design to be a linear combination of these basis vectors: 1 * αι = = ∑ + r i C i x x x scalar coefficient basis vector added for generality 8 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Reduced Reduced-Basis Methods Basis Methods We can now optimize J(x) by finding the optimal values for the coefficients αi. dimension n dimension r • do one full-order evaluation of resulting answer • approach is efficient if r << n • will give the true optimum only if x* lies in the span of {xi} • basis vectors could be – previous designs – solutions over a particular range (DoE) – derived in some other way (POD)
Mlesd Reduced-Basis Example 16888 Reduced-Basis Example Example using a reduced-basis approach(van der Plaats Fig 7-2 airfoil design for a unique application many airfoil shapes with known performance are Vanderplaats, G. N Numerical Optimization Techniques for Engineering Des available design variables are(x, y) coordinates at chordwise locations(n-100) use four basis airfoil shapes(low-speed airfoils)which contain the n geometry points plus two basis shapes which allow trailing edge thickness to vary rplaats, G N. Numerical Optimization Techniques for Engineering Design rplaats R&D, 1999. Figure 7-3 =6(K<n) optimize for high speed, maximum lift with a constraint on drag @ Massachusetts Institute of Technology -Prof de Weck and Prof Willcox e Massachusetts Insttute of Technology. Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems Division and Dept of Aeronautics and Astronautics Mlesd Proper Orthogonal Decomposition E0t Mlesd Proper Orthogonal Decomposition E89 Loeve decomposition, singular value decomposition en- Also known as principal components analysis, Kahunen- These optimal basis functions can be calculated by Evaluating the correlation matrix R 2. Solving the MxM eigenvalue probler p are orthog They are computed from M empirical solutions x', x2,.1 They are optimal in the sense that they minimize the error 3. Constructing the basis vectors between the original and the projected data Use components of the x eigenvector to calculate the /th max(, g=∑ POD basis vector. The th ortant is the / th e Massachusetts Institute of Technology -Prof de Weck and Prof. Willcox e Massachusetts Insttute of Technology. Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems DiMsion and Dept of Aeronautics and Astronautics
9 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Reduced Reduced-Basis Example Basis Example Example using a reduced-basis approach (van der Plaats Fig 7-2): airfoil design for a unique application. • many airfoil shapes with known performance are available • design variables are (x,y) coordinates at chordwise locations (n~100) • use four basis airfoil shapes (low-speed airfoils) which contain the n geometry points • plus two basis shapes which allow trailing edge thickness to vary • r=6 (r<<n) • optimize for high speed, maximum lift with a constraint on drag 10 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Reduced Reduced-Basis Example Basis Example 11 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Proper Orthogonal Decomposition Proper Orthogonal Decomposition 1 * αιϕ = = ∑ r i i x • The r basis vectors ϕi are orthogonal • They are computed from M empirical solutions {x1, x2,... xM } • They are optimal in the sense that they minimize the error between the original and the projected data 2 2 max ( , ) ϕ ϕ ϕ x Also known as principal components analysis, KahunenLoève decomposition, singular value decomposition 12 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Proper Orthogonal Decomposition Proper Orthogonal Decomposition These optimal basis functions can be calculated by: 1. Evaluating the correlation matrix: 2. Solving the M×M eigenvalue problem: 3. Constructing the basis vectors: i j T R ij = xx R v v i i i = λ 1 M i i j j i ϕ v = = ∑ x Use components of the jth eigenvector to calculate the jth POD basis vector. The jth eigenvalue tells us how important is the jth basis vector. Vanderplaats, G. N. Numerical Optimization Techniques for Engineering Design. Vanderplaats R&D, 1999. Figure 7-2. Vanderplaats, G. N. Numerical Optimization Techniques for Engineering Design. Vanderplaats R&D, 1999. Figure 7-3
Mlesd Approximation Methods Overview E0 Mles Response Surface Methodology 505 reduce the number of Keep the same number of design variables, but simplify the Design variable linking design variables in simcode analysis Reduced-basis methods the optimization code Create approximating functions to objective and constraints simcode analysis full Optimize using the approximations order Update approximations using current optimal solution same number of Response surface methods design variables Response surfaces are smooth even if design space is noisy Kriging simcode analysis simplified Polynomial-based modeling technique Provide compact, explicit functional relationship between combine high-fidelity and response and design variables Variable-fidelity methods approximation models Least squares is computationally inexpensive and easy to @Massachusetts Institute of Technology -Prof de Weck and Prof Willcox G Massachusetts Insttute of Technology .Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems Division and Dept of Aeronautics and Astronautics MIlesd Local Approximations Local Approximations Most common are taylor series expansions ()4×)+J(x)]x+2xH(x)x+ It is expensive to update the gradient vector and Hessian matrⅸ Sx=x-x ne appre Could use the first two terms linear approximation perform several approximation cycles updating only the constant term Solve, reanalyze and repeat= sequential linear programming then update the linear term and repeat Could also include quadratic term finally, update the Hessian only when no other (-x)+(x3).x+20xH(x)x+ progress can be made If the design space is highly nonlinear, there is no 1 function guarantee that this approach will work n func function evaluat setts Institute of Technology -Prof de Weck and Prof Willcox etts Institute of Technology .Prof de Weck and Prof Willcox Engineening Systems Division and Dept of Aeronautics and Astronautics ngineering Systems Divsion and Dept of Aeronautics and Astronautics
13 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Approximation Methods Overview Design variable linking Reduced-basis methods • reduce the number of design variables in the optimization code • simcode analysis full order Response surface methods Kriging • same number of design variables • simcode analysis simplified Variable-fidelity methods • combine high-fidelity and approximation models 14 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Response Surface Methodology Response Surface Methodology • Keep the same number of design variables, but simplify the simcode analysis • Create approximating functions to objective and constraints • Optimize using the approximations • Update approximations using current optimal solution guess and repeat • Response surfaces are smooth even if design space is noisy • Polynomial-based modeling technique • Provide compact, explicit functional relationship between response and design variables • Least squares is computationally inexpensive and easy to implement 15 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Local Approximations Local Approximations Most common are Taylor series expansions: = + ⎡∇ ⎤ δ + δ δ + 0 ⎣ 0 ⎦ " 1 ( ) ( ) ( ) ( ) 2 T T J J J 0 x x x x x H x x 0 δ x xx = − • Could use the first two terms ⇒ linear approximation • Solve, reanalyze and repeat = sequential linear programming • Could also include quadratic term: = + ⎡∇ ⎤ δ + δ δ + 0 ⎣ 0 ⎦ " 1 ( ) ( ) ( ) ( ) 2 T T J J J 0 x x x x x H x x update requires: 1 function evaluation n function evaluations n(n+1)/2 function evaluations 16 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Local Approximations Local Approximations • It is expensive to update the gradient vector and Hessian matrix • One approach: – perform several approximation cycles updating only the constant term – then update the linear term and repeat – finally, update the Hessian only when no other progress can be made • If the design space is highly nonlinear, there is no guarantee that this approach will work
M sd Response Surface Methodology 5. 33 RSM Another approach: use what information is available to e.g. define AJ=J(x)-J(x) create the approximate Use this approximation to make a small move in the quadratic approximation A=VSx+OxHey Analyze the result precisely = new function evaluation Use the new function evaluation to improve the 6x1+6x2++dx approximation to the design spa ax → Fit a response surface Can use a quadratic or higher order surface (H1bx+H25x2+…+Hmbx) Might choose to use only some of the function evaluations +Hnox x,+hnox, dx, ++H,Ox,dx (e.g. those in a local neighborhood +H25xbx3+…+ Hr-l,dx-5 (all derivatives and entries of H are evaluated at x o) @Massachusetts Institute of Technology -Prof de Weck and Prof Willcox e Massachusetts Insttute of Technology. Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems Division and Dept of Aeronautics and Astronautics MIest RSM RSM Assume we have evaluated the baseline plus g designs q equations, N unknowns If g<N, only some coefficients can be calculated x",x,,x→J0J,J If N, use weighted least squares could write q equations of the form(*)using Weight designs closer to current xq more heavily In general, use g2n+ 1 initial designs so an initial linear approximation can be provided If have baseline plus n designs, can fit a linear There is a total of N=ntn(n+1)/2 unknowns approximation in each direction (i.e. a hyperplane) If have an fit a quadratic or higher order surface a Massachusetts Institute of Technology -Prof de Weck and Prof Willcox etts Institute of Technology .Prof de Weck and Prof Willcox Engineening Systems Division and Dept of Aeronautics and Astronautics ngineering Systems Divsion and Dept of Aeronautics and Astronautics
17 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Response Surface Methodology Response Surface Methodology • Another approach: use what information is available to create the approximation • Use this approximation to make a small move in the design variables • Analyze the result precisely ⇒ new function evaluation • Use the new function evaluation to improve the approximation to the design space ⇒ Fit a response surface • Can use a quadratic or higher order surface • Might choose to use only some of the function evaluations (e.g. those in a local neighborhood) 18 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics RSM ∆ = − 0 eg J J J . . define ( ) ( ) x x ( ) T T 1 2 1 2 2 2 2 11 1 22 2 12 1 2 13 1 3 1 1 23 2 3 1, 1 ... 1 ... 2 ... ... n n nn n n n n n n n J J J J J x x x x x x H x H x H x H x x H x x H x x H x x H x x δ δ δ δ δ δ δ δ δ δ δ δ δ δ δ δ δ δ δ − − ∆ =∇ + ∂ ∂ ∂ = + + + ∂ ∂ ∂ + + + + + + + + + + + x x H x quadratic approximation: (all derivatives and entries of H are evaluated at x0) (∗) 19 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics RSM Assume we have evaluated the baseline plus q designs: 0 1 → 0 1 , ,..., , ,... q q xx x J J J We could write q equations of the form (∗) using There is a total of N=n+n(n+1)/2 unknowns: ∂ ∂ ∂ ∂ ∂ ∂ 11 12 1 2 , ,..., , , ,..., nn n J J J HH H x x x ( ) ( ) ( ) 1 0 1 0 2 0 2 0 0 0 , , , , , , q q x x x x − J J J J − − − " x x − J J − 20 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics RSM • q equations, N unknowns • If qN, use weighted least squares • Weight designs closer to current xq more heavily • In general, use q≥n+1 initial designs so an initial linear approximation can be provided • If have baseline plus n designs, can fit a linear approximation in each direction (i.e. a hyperplane) • If have more solutions, can fit a quadratic or higher order surface
MIest RSM 16888 RSM In other words, fit objective function with a polynomial Estimation problem J≈Xc e.g. quadratic approximation J(x)=a+∑6x+∑cx+∑cx J=vector containing c=vector containing Update model by including a new function evaluation p coefficients then doing least squares fit to compute the new oefficients containing M design Columns depend on solution:c=(XX)X e Massachusetts Institute of Technology - Prof de Weck and Prof Willcox e Massachusetts Insttute of Technology. Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems Division and Dept of Aeronautics and Astronautics MIest iging RSM tends to create a global model of the design space Unknown function to be modeled expressed as (especially if all points weighted equally in the LS J(x)=f(x)+Z(x) May be of limited accuracy when multiple extrema exist (especially quadratic polynomial models · known function Gaussian random function Kriging: combine global model of design space with a model for local deviations which interpolates sample points global" model for design Developed in fields of spatial statistics and geostatistics variance a e.g. use RSM to fit f(x) using ."localized"deviation from Original statistical techniques developed by mining M observations engineer D.G. Krige global model Interpolation-based modeling technique Computationally more expensive and not as simple to cy2×2X]=R[R(xx) implement as rsm covariance matrix of z(x) correlation matrix correlation function e Massachusetts Institute of Technology- Prof de Weck and Prof Willcox e Massachusetts Insttute of Technology. Prof de Weck and Prof Willcox Engineening Systems Division and Dept of Aeronautics and Astronautics ngineering Systems Divsion and Dept of Aeronautics and Astronautics
21 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics RSM • In other words, fit objective function with a polynomial • e.g. quadratic approximation: • Update model by including a new function evaluation then doing least squares fit to compute the new coefficients 2 , () i i ii i ij i j i i i j i J b x c x c x x < x a = 0 + ∑ + ∑ + ∑ 22 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics RSM Estimation problem: J Xc ≈ T 1 2 M = ⎡J J J ⎤ ⎣ ⎦ J " 1 1 1 1 1 2 1 1 2 1 1 2 1 1 1 1 1 M M M M x x x x x x x x x ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ X " " # # 0 1 p 1 c c c − ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ ⎢⎣ ⎥⎦ c J=vector containing M responses c=vector containing p coefficients x=M×p matrix containing M design vector inputs as rows. Columns depend on approximation. Least squares solution: ( ) 1 T T − c X X X J = 23 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Kriging Kriging • RSM tends to create a global model of the design space (especially if all points weighted equally in the LS) • May be of limited accuracy when multiple extrema exist (especially quadratic polynomial models) • Kriging : combine global model of design space with a model for local deviations which interpolates sample points • Developed in fields of spatial statistics and geostatistics • Original statistical techniques developed by mining engineer D.G. Krige • Interpolation-based modeling technique • Computationally more expensive and not as simple to implement as RSM 24 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Kriging Kriging Unknown function to be modeled expressed as: J f Z () () () x x x = + • known function • “global” model for design space • e.g. use RSM to fit f(x) using M observations • Gaussian random function • zero mean • variance σ2 • “localized” deviation from global model cov ( ), ( ) ( , ) i j i j Z Z σ R 2 ⎡ ⎤ = ⎡ ⎤ ⎣ ⎦ ⎣ ⎦ x x R x x covariance matrix of Z(x) correlation matrix correlation function
Mlest Approximation Methods Overview s0 Mles Variable Fidelity Models 5079 reduce the number of Design variable linking design variables in Use information from high-fidelity model to check approximate designs Reduced-basis methods the optimization code simcode analysis full Initialization.Optimize using approximate model order Recalibrate using high-fidelity model same number of ecourse to High-fidelity model detailed mod Response surface methods design variables Kriging simcode analysis simplified J(x), gx Approximation model Variable-fidelity methods combine high-fidelity and approximation models optimization on a simplified model From: Fig. 1, Alexandrov et al. e Massachusetts Institute of Technology - Prof de Weck and Prof Willcox G Massachusetts Insttute of Technology .Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems Division and Dept of Aeronautics and Astronautics Mlesd variable Fidelity Models 甥 Mlesd variable Fidelity Models Questions When optimization with approximation model is What do we do when the design derived from the low- unsuccessful, there are two possible approaches fidelity optimization does not provide an improvement in the true objective? prove the fidelity of the approximate mod How can we use information about the predictive 2. Do"less" optimization capability of the approximation model to decide when to go back to the high-fidelity model? One option: use a trust region approach How do we decide when to update the approximate model? trust region current iterate xk design spa a Massachusetts Institute of Technology -Prof de Weck and Prof Willcox e Massachusetts Insttute of Technology. Prof de Weck and Prof Willcox Engineening Systems Division and Dept of Aeronautics and Astronautics Engineering Systems DiMsion and Dept of Aeronautics and Astronautics
25 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Approximation Methods Overview Design variable linking Reduced-basis methods • reduce the number of design variables in the optimization code • simcode analysis full order Response surface methods Kriging • same number of design variables • simcode analysis simplified Variable-fidelity methods • combine high-fidelity and approximation models 26 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Variable Fidelity Models Variable Fidelity Models Initialization High-fidelity model Approximation model Optimizer x J(x), g(x) optimization on a simplified model recourse to detailed model From: Fig. 1, Alexandrov et al. • Use information from high-fidelity model to check approximate designs • Optimize using approximate model • Recalibrate using high-fidelity model 27 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Variable Fidelity Models Variable Fidelity Models Questions: • What do we do when the design derived from the lowfidelity optimization does not provide an improvement in the true objective? • How can we use information about the predictive capability of the approximation model to decide when to go back to the high-fidelity model? • How do we decide when to update the approximate model? 28 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Variable Fidelity Models Variable Fidelity Models When optimization with approximation model is unsuccessful, there are two possible approaches: 1. Improve the fidelity of the approximate model 2. Do “less” optimization One option: use a trust region approach current iterate xk trust region design space
Mlesa Trust Region Approach 50 Mles Trust Region Approach 505 Classic approach Restrict the step size to a region in which we trust the Regulate the length of the steps taken by the iterative quadratic model to approximate J well optimization algorithm Done by adding a constraint on the step Regulate based on quality of current approximation model trust region subproblem min g"(x+s) e.g. Quadratic Taylor series model ls。 J(x+s)g(x+)=J(x)+VJ(x)s+3s'B's In practice, variables are scaled to improve performance (step size can vary in different directions) where s is the prospective step in the design variables, AX Then decide whether to accept the step and bk is a model of the hessian matrix at xk x=x+4x+ if model did a good job of predicting J, or if there r=x)-4(x+s) was more improvement than predicted, then J(x)-gX+s Increase冰 if model did a bad job of predicting J(J increased or Define constants r,, r2(,<r2)and apply the rules decrease was much lower than predicted), then if r< r, decrease the trust radius decrease &k if pr2, increase the trust radius if model did an acceptable job of predicting J, then Typical values are 1=0.1, [2=0.75 do not change k Note that prediction of ascent/descent is more important than prediction of actual value e Massachusetts Institute of Technology- Prof de Weck and Prof Willcox e Massachusetts insttute of Tech Prof de Weck and Prof willcox Engineening Systems Division and Dept of Aeronautics and Astronautics Aeronautics and astronautics
29 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Trust Region Approach Trust Region Approach Classic approach: • Regulate the length of the steps taken by the iterative optimization algorithm • Regulate based on quality of current approximation model e.g. Quadratic Taylor series model: 1 ( ) ( ) ( ) ( ) 2 T k k k k k T k J q J J + ≈ + = + ⎡∇ ⎤ + ⎣ ⎦ x s x s x x s s B s where s is the prospective step in the design variables, ∆x, and Bk is a model of the Hessian matrix at xk. 30 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Trust Region Approach Trust Region Approach • Restrict the step size to a region in which we trust the quadratic model to approximate J well • Done by adding a constraint on the step ⇒ trust region subproblem min ( ) s.t. k k k q δ + ≤ x s s • In practice, variables are scaled to improve performance (step size can vary in different directions) • Then decide whether to accept the step: 1 if ( ) ( ) otherwise k k k k k + ⎧ + J J + r2, increase the trust radius • Typical values are r1=0.1, r2=0.75 • Note that prediction of ascent/descent is more important than prediction of actual value
Mlesd Classic Trust Region Algorithm蠲M画 d B-Correlation Method齜 Basic idea is to take low-fidelity approximate model, Ja, and correct it by scaling For k=0, 1,. until convergence do i Define the scale factor Find an approximate solution s, to the subproblem J(E min g"(x+s) .t.ss Given the current design x, build a first order model Compare the actual and predicted decrease of B about xk J(x)-√(x2+s) B(x)=B(x)+VB(x)(x-x) J(x2)-q(x2+s) Optimize with approximate model and use local model of Update x and 8 β to scale result From Fig. 2, Alexandrov et al ()≈BJ2(x @Massachusetts Institute of Technology -Prof de Weck and Prof Willcox e Massachusetts Insttute of Technology. Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems Division and Dept of Aeronautics and Astronautics VCM i Sight uses a Variable Complexity Model (VCM) Simeone Use two simcodes for the same physical proces Optimization Simcode (1)more accurate, longer running(exact)J(x) Jr(x) (2)less accurate, shorter running(approx )Ja(x) Conventional optimization Compute a multiplicative or additive correction factor, with simcode ith approximate model, update with simcode σ=√(x)-J(x°) Optimization Approx. 1. Run both models calculate oo J(x") J(x) J(x)=0+J,(x) 2. Optimize using approximate (x)=a(x) del and oo Conventional optimization 3. Update correction factor using with approximate model simcode and repeat e Massachusetts Institute of Technology- Prof de Weck and Prof Willcox e Massachusetts Instilute of Technology .Prof de Weck and Prof. Willcox Engineening Systems Division and Dept of Aeronautics and Astronautics Engineering Systems Division and Dept of Aeronautics and Astronautics
33 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Classic Trust Region Algorithm For k=0,1,... until convergence do { Find an approximate solution sk to the subproblem: 0 0 Choose and 0 n x ∈ \ δ > min ( ) s.t. k k k q δ + ≤ x s s Compare the actual and predicted decrease: ( ) ( ) ( ) ( ) k k k k J J r J q − + = − + x x s x x s Update xk and δk } From Fig. 2, Alexandrov et al. 34 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics β-Correlation Method Correlation Method • Basic idea is to take low-fidelity approximate model, Ja, and correct it by scaling • Define the scale factor: () ( ) a J J β = x x Given the current design xk, build a first order model of β about xk : ( ) ) )( ) k k k k β x x x x x = ( β +∇ ( β − Optimize with approximate model and use local model of β to scale result : () () k a J J x x ≈ β 35 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Variable Complexity Model Variable Complexity Model • iSight uses a Variable Complexity Model (VCM) • Use two simcodes for the same physical process: (1) more accurate, longer running (exact) J(x) (2) less accurate, shorter running (approx.) Ja(x) • Compute a multiplicative or additive correction factor, σ : 0 0 ) ) () ) a a J J J J σ σ ( = ( = ( x x x x 0 0 ) ) () ) a a J J J J σ σ = ( − ( = + ( x x x x or 36 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics VCM Optimization Simcode J(x) Optimization Approx. Ja(x) Optimization Approx. Ja(x) Simcode J(x) Conventional optimization with simcode Conventional optimization with approximate model Optimize with approximate model, update with simcode: 1. Run both models, calculate σ0 2. Optimize using approximate model and σ0 3. Update correction factor using simcode and repeat
MIest ecture Summary g88 References Approximation methods are one way to capture high-fidelity responses Barthelemy, J-F. M and Haftka, R.T., "Approximation concepts for optimum without the computational cost structural design-a review, Structural Optimization, 5: 129-144, 1993 Design variable linking if physical problem allows Giunta, A.A. and Watson, L.T., A comparison of approximation modeling Reduced-basis methods use low-order representation of the design vector techniques polynomial versus interpolating models, AIAA Paper 98-4758, 1998 e previous designs, DoE or POD Le Gresley, P.A. and Alonso, J.J., "Airfoil design optimization using reduced ord models based on proper orthogonal decomposition, AIAA Paper 2000-2545 Response surface me Alexandrov, N, Dennis, J.E., Lewis, R. M. and Torczon, V, "A trust region use polynomial models framework for managing the use of approximation models in optimization, NASA weighted least squares CR-201745, ICASE Report No. 97-50, October 1997. Kriging Practical Optimization, P.E. Gill, W. Murray and M H. Wright, Academic Press interpolation models Numencal Optimization Techniques for Engineering Design, G N. Vanderplaats global local behavior Vanderplaats R&D, 1999 Variable-fidelity models Trust region approach B-correlation e Massachusetts Institute of Technology- Prof de Weck and Prof Willcox G Massachusetts Insttute of Technology Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems Division and Dept of Aeronautics and Astronautics
37 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Lecture Summary Lecture Summary • Approximation methods are one way to capture high-fidelity responses without the computational cost • Design variable linking if physical problem allows • Reduced-basis methods use low-order representation of the design vector • use previous designs, DoE or POD • Response surface methodology • use polynomial models • weighted least squares • Kriging • interpolation models • global + local behavior • Variable-fidelity models • Trust region approach • β-correlation 38 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics References References Barthelemy, J-F. M. and Haftka, R.T., “Approximation concepts for optimum structural design – a review”, Structural Optimization, 5:129-144, 1993. Giunta, A.A. and Watson, L.T.,”A comparison of approximation modeling techniques: polynomial versus interpolating models”, AIAA Paper 98-4758, 1998. LeGresley, P.A. and Alonso, J.J., “Airfoil design optimization using reduced order models based on proper orthogonal decomposition”, AIAA Paper 2000-2545. Alexandrov, N., Dennis, J.E., Lewis, R.M. and Torczon, V., “A trust region framework for managing the use of approximation models in optimization”, NASA CR-201745, ICASE Report No. 97-50, October 1997. Practical Optimization, P.E. Gill, W. Murray and M.H. Wright, Academic Press, 1986. Numerical Optimization Techniques for Engineering Design, G.N. Vanderplaats, Vanderplaats R&D, 1999