Filtering concerns the processing of measurement information. In optimal control filters are used to represent information about the system and control problem of interest In general, this information is incomplete, i.e. the state is typically not fully accessible, and may be corrupted by noise. To solve optimal control problems in these situations, the cost function is expressed in terms of the state of a suitably chosen filter, which is often called an information state. Dynamic programming can then be applied using the information state dynamics. The nature of the measurements and the purpose for which the data is to be used determine the architecture of the filter. In stochastic situations this is closely linked to the probabilistic concept of conditional expectation. The famous Kalman filter computes dynamically conditional expectations(of states given measure- ments in linear gaussian models), which are also optimal estimates in the mean square error sense. The quantum Belaukin filter, or stochastic master equation, also computes a quantum version of conditional expectation. In linear gaussian cases, the information states are gaussian, a fact which considerably simplifies matters due to the finite num ber of parameters. Filters such as these based on computing conditional expectations of states or system variables do not include any information about the cost or performance objective. While this is not an issue for many problems such as LQG, where the task of estimation can be completely decoupled from that of control [17, there are important problems where the filter dynamics must be modified to take into account the control objective. These problems include LEQG[48, 49] or risk-sensitive control[8, 37, and Hoo robust control [19, 54 Figure 1 shows a physical system being controlled in a feedback loop. The so-calle separation structure of the controller is shown. The control values are computed in the box marked"control", using a function of the information state determined using dynamic programming. The information state, as has been mentioned, is the state of the filter whose dynamics are built into the box marked"filter". This structure embodies the two themes of these notes output physical system control filter feed back controller Figure 1: Feedback controller showing the separation structure These notes were assembled from various lecture notes and research papers, and so we apologize for the inevitable inconsistencies that resultedFiltering concerns the processing of measurement information. In optimal control, filters are used to represent information about the system and control problem of interest. In general, this information is incomplete, i.e. the state is typically not fully accessible, and may be corrupted by noise. To solve optimal control problems in these situations, the cost function is expressed in terms of the state of a suitably chosen filter, which is often called an information state. Dynamic programming can then be applied using the information state dynamics. The nature of the measurements and the purpose for which the data is to be used determine the architecture of the filter. In stochastic situations, this is closely linked to the probabilistic concept of conditional expectation. The famous Kalman filter computes dynamically conditional expectations (of states given measure￾ments in linear gaussian models), which are also optimal estimates in the mean square error sense. The quantum Belavkin filter, or stochastic master equation, also computes a quantum version of conditional expectation. In linear gaussian cases, the information states are gaussian, a fact which considerably simplifies matters due to the finite num￾ber of parameters. Filters such as these based on computing conditional expectations of states or system variables do not include any information about the cost or performance objective. While this is not an issue for many problems such as LQG, where the task of estimation can be completely decoupled from that of control [17], there are important problems where the filter dynamics must be modified to take into account the control objective. These problems include LEQG [48, 49] or risk-sensitive control [8, 37], and H∞ robust control [19, 54]. Figure 1 shows a physical system being controlled in a feedback loop. The so-called separation structure of the controller is shown. The control values are computed in the box marked “control”, using a function of the information state determined using dynamic programming. The information state, as has been mentioned, is the state of the filter whose dynamics are built into the box marked “filter”. This structure embodies the two themes of these notes. ✛ ✲ filter ✛ physical system u y control feedback controller input output Figure 1: Feedback controller showing the separation structure. These notes were assembled from various lecture notes and research papers, and so we apologize for the inevitable inconsistencies that resulted. 4