LEARNING FROM OBSERVATIONS CHAPTER 18,SECTIONS 1-3 Chapter 18,Sections 1-3 1
Learning from Observations Chapter 18, Sections 1–3 Chapter 18, Sections 1–3 1
Outline ◇Learning agents ◇Inductive learning Decision tree learning Measuring learning performance Chapter 18,Sections 1-3 2
Outline ♦ Learning agents ♦ Inductive learning ♦ Decision tree learning ♦ Measuring learning performance Chapter 18, Sections 1–3 2
Learning Learning is essential for unknown environments, i.e.,when designer lacks omniscience Learning is useful as a system construction method, i.e.,expose the agent to reality rather than trying to write it down Learning modifies the agent's decision mechanisms to improve performance Chapter 18,Sections 1-3 3
Learning Learning is essential for unknown environments, i.e., when designer lacks omniscience Learning is useful as a system construction method, i.e., expose the agent to reality rather than trying to write it down Learning modifies the agent’s decision mechanisms to improve performance Chapter 18, Sections 1–3 3
Learning agents Performance standard Critic Sensors feedback changes Learning Performance element element knowledge learning Environment goals Problem experiments generator Agent Effectors Chapter 18,Sections 1-3 4
Learning agents Performance standard Agent Environment Sensors Effectors Performance element changes knowledge learning goals Problem generator feedback Learning element Critic experiments Chapter 18, Sections 1–3 4
Learning element Design of learning element is dictated by what type of performance element is used which functional component is to be learned how that functional compoent is represented what kind of feedback is available Example scenarios: Performance element Component Representation Feedback Alpha-beta search Eval.fn. Weighted linear function Win/loss Logical agent Transition model Successor-state axioms Outcome Utility-based agent Transition model Dynamic Bayes net Outcome Simple reflex agent Percept-action fn Neural net Correct action Supervised learning:correct answers for each instance Reinforcement learning:occasional rewards Chapter 18,Sections 1-3 5
Learning element Design of learning element is dictated by ♦ what type of performance element is used ♦ which functional component is to be learned ♦ how that functional compoent is represented ♦ what kind of feedback is available Example scenarios: Performance element Alpha−beta search Logical agent Simple reflex agent Component Eval. fn. Transition model Transition model Representation Weighted linear function Successor−state axioms Neural net Utility−based agent Dynamic Bayes net Percept−action fn Feedback Outcome Outcome Win/loss Correct action Supervised learning: correct answers for each instance Reinforcement learning: occasional rewards Chapter 18, Sections 1–3 5
Inductive learning (a.k.a.Science) Simplest form:learn a function from examples(tabula rasa) f is the target function O OX An example is a pair f(x),e.g.. +1 Problem:find a(n)hypothesis h such that h≈f given a training set of examples (This is a highly simplified model of real learning: -Ignores prior knowledge -Assumes a deterministic,observable "environment" Assumes examples are given -Assumes that the agent wants to learn f-why?) Chapter 18,Sections 1-3 6
Inductive learning (a.k.a. Science) Simplest form: learn a function from examples (tabula rasa) f is the target function An example is a pair x, f(x), e.g., O O X X X , +1 Problem: find a(n) hypothesis h such that h ≈ f given a training set of examples (This is a highly simplified model of real learning: – Ignores prior knowledge – Assumes a deterministic, observable “environment” – Assumes examples are given – Assumes that the agent wants to learn f—why?) Chapter 18, Sections 1–3 6
Inductive learning method Construct/adjust h to agree with f on training set (h is consistent if it agrees with f on all examples) E.g.,curve fitting: f() X + X 义 X X Chapter 18,Sections 1-3 7
Inductive learning method Construct/adjust h to agree with f on training set (h is consistent if it agrees with f on all examples) E.g., curve fitting: x f(x) Chapter 18, Sections 1–3 7
Inductive learning method Construct/adjust h to agree with f on training set (h is consistent if it agrees with f on all examples) E.g.,curve fitting: f(x) X X Chapter 18,Sections 1-3 8
Inductive learning method Construct/adjust h to agree with f on training set (h is consistent if it agrees with f on all examples) E.g., curve fitting: x f(x) Chapter 18, Sections 1–3 8
Inductive learning method Construct/adjust h to agree with f on training set (h is consistent if it agrees with f on all examples) E.g.,curve fitting: f() X Chapter 18,Sections 1-3 9
Inductive learning method Construct/adjust h to agree with f on training set (h is consistent if it agrees with f on all examples) E.g., curve fitting: x f(x) Chapter 18, Sections 1–3 9
Inductive learning method Construct/adjust h to agree with f on training set (h is consistent if it agrees with f on all examples) E.g.,curve fitting: f(x) Chapter 18.Sections 13 10
Inductive learning method Construct/adjust h to agree with f on training set (h is consistent if it agrees with f on all examples) E.g., curve fitting: x f(x) Chapter 18, Sections 1–3 10
