3.1.6 Markov processes 3.1.7 Observation Processes 3. 1.8 Linear Representation of a Markov Chain 3.2 Controlled State Space Models 2.1 Feedback Control Laws or Policies 3.2.2 Partial and Full State Information 3.3 Filtering 34 3.3.1 Introduction 3.3.2 The Kalman Filter 3.3.3 The Kalman Filter for Controlled Linear Systems 3.3.4 The HMM Filter(Markov Chain) 3.3.5 Filter for Controlled HMM 3.4 Dynamic Programming -Case I: Complete State Information 3.4.1 Optimal Control Problem 3.5 Dynamic Programming -Case II: Partial State Information 3.5.1 Optimal Control of HMMs 2678 3.5.2 Optimal Control of Linear Systems(LQG) 3.6 Two Continuous Time Problems 3.6.1 System and Kalman Filte 3.6.2 LQG Control 51 3.6.3 LEQG Control 51 4 Robust Control 4.1 Introduction and Background 4.2 The Standard Problem of Hoo Control 54 4. 2.1 The Plant(Physical System Being Controlled) 4.2.2 The Class of Controllers 4.2.3 Control Objectives 4.3 The Solution for Linear Systems 4.3.1 Problem formulation 56 4.3.2 Background on Riccati equations 4.3.3 Standard Assumptions 57 4.3.4 Problem Solution 4.4 Risk-Sensitive Stochastic Control and Robustness 5 Optimal Feedback Control of Quantum Systems 5.1 Preliminaries 5.2 The Feedback Control Problem 5.3 Conditional Dynamics 63 5.3.1 Controlled State Transfer 5.3.2 Feedback Control 5.4 Optimal Control 5.5 Appendix: Formulas for the Two-State System with Feedback Example.. 733.1.6 Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.1.7 Observation Processes . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.1.8 Linear Representation of a Markov Chain . . . . . . . . . . . . . . . 32 3.2 Controlled State Space Models . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.2.1 Feedback Control Laws or Policies . . . . . . . . . . . . . . . . . . . 34 3.2.2 Partial and Full State Information . . . . . . . . . . . . . . . . . . . 34 3.3 Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.3.2 The Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.3.3 The Kalman Filter for Controlled Linear Systems . . . . . . . . . . 39 3.3.4 The HMM Filter (Markov Chain) . . . . . . . . . . . . . . . . . . . 39 3.3.5 Filter for Controlled HMM . . . . . . . . . . . . . . . . . . . . . . . 42 3.4 Dynamic Programming - Case I : Complete State Information . . . . . . . 42 3.4.1 Optimal Control Problem . . . . . . . . . . . . . . . . . . . . . . . 43 3.5 Dynamic Programming - Case II : Partial State Information . . . . . . . . 46 3.5.1 Optimal Control of HMM’s . . . . . . . . . . . . . . . . . . . . . . 47 3.5.2 Optimal Control of Linear Systems (LQG) . . . . . . . . . . . . . . 48 3.6 Two Continuous Time Problems . . . . . . . . . . . . . . . . . . . . . . . . 50 3.6.1 System and Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . 50 3.6.2 LQG Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.6.3 LEQG Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 4 Robust Control 53 4.1 Introduction and Background . . . . . . . . . . . . . . . . . . . . . . . . . 53 4.2 The Standard Problem of H∞ Control . . . . . . . . . . . . . . . . . . . . 54 4.2.1 The Plant (Physical System Being Controlled) . . . . . . . . . . . . 54 4.2.2 The Class of Controllers . . . . . . . . . . . . . . . . . . . . . . . . 55 4.2.3 Control Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 4.3 The Solution for Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . 56 4.3.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 56 4.3.2 Background on Riccati Equations . . . . . . . . . . . . . . . . . . . 57 4.3.3 Standard Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.3.4 Problem Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 4.4 Risk-Sensitive Stochastic Control and Robustness . . . . . . . . . . . . . . 59 5 Optimal Feedback Control of Quantum Systems 61 5.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 5.2 The Feedback Control Problem . . . . . . . . . . . . . . . . . . . . . . . . 62 5.3 Conditional Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 5.3.1 Controlled State Transfer . . . . . . . . . . . . . . . . . . . . . . . 63 5.3.2 Feedback Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 5.4 Optimal Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 5.5 Appendix: Formulas for the Two-State System with Feedback Example . . 73 2