Kumar s. ray Soft Computing Approach to Pattern Classification and Object recognition A Unified Concept Springer
Soft Computing Approach to Pattern Classification and object Recognition
Soft Computing Approach to Pattern Classification and Object Recognition
Kumar S Ray Soft Computing Approach to Pattern classification and object Recognition A Unified Concept Springer
Kumar S. Ray Soft Computing Approach to Pattern Classification and Object Recognition A Unified Concept 123
Kumar S Ray Indian statistical Institute Kolkata West bengal India ISBN978-1-4614-5347-5 ISBN978-1-4614-5348-2( eBook) DOI10.1007978-1-46145348-2 Springer New York Heidelberg Dordrecht London o Springer Science+Business Media New York 2012 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, roadcasting, reproduction on microfilms or in any other physical way, and transmission or adaptation, ware, or by similar or dissimila methodology now known or hereafter developed. Exempted from this legal res briet excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publishers location. in its current obtained from Springer. Permissions for use may be obtained through Rights Link at the Copyright learance Center. Violations are liable to prosecution under the respective Copy The use eneral descriptive names, registered names, trademarks, service wright kaw.c in th blication does not imply, even in the absence of a specific statement, that such names are exempt rom the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with Printed on acid-free paper SpringerispartofSpringerScience+businessmeDia(www.springer.com)
Kumar S. Ray Indian Statistical Institute Kolkata West Bengal India ISBN 978-1-4614-5347-5 ISBN 978-1-4614-5348-2 (eBook) DOI 10.1007/978-1-4614-5348-2 Springer New York Heidelberg Dordrecht London Library of Congress Control Number: 2012944977 Springer Science+Business Media New York 2012 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)
Dedicated to Aratrika (Daughter
Dedicated to: Dhira (Wife) Aratrika (Daughter)
peace The basic aim of this research monograph is to develop a unified approach to supervised pattern classification and model-based occluded object recognition. To perform this task we essentially consider soft computing tools, viz., fuzzy relational calculus(FRC), genetic algorithm(GA), and multilayer perceptron(MLP). The supervised approach to pattern classification and model-based approach to occluded object recognition are treated in one framework which is based on either conven- tional interpretation or new interpretation of multidimensional fuzzy implication (MFD) and a novel notion of fuzzy pattern vector(FPV). A completely independent design methodology has been developed on a unified framework which has been thoroughly tested on several synthetic and real life data. In the field of soft com- puting such application-oriented design study is unique in nature. The monograph essentially mimics the cognitive process of human decision making. It carries a message of perceptual integrity in representational diversity The monograph is very much useful to the researchers in the area of pattern classification and computer vision. It is useful for the academics as well as for the professional computer scientists of different research and development centers of industry. The monograph has a combined flavor of theory and practice t The monograph is basically a collection of research contributions of Prof Kumar Ray at Electronics and Communication Sciences Unit of Indian Statistical Institute. Kolkata. Prof. Kumar S. Ray is grateful to Mandrita Mondal for her constant encour agement and support to complete the monograph. Kumar S Ray
Preface The basic aim of this research monograph is to develop a unified approach to supervised pattern classification and model-based occluded object recognition. To perform this task we essentially consider soft computing tools, viz., fuzzy relational calculus (FRC), genetic algorithm (GA), and multilayer perceptron (MLP). The supervised approach to pattern classification and model-based approach to occluded object recognition are treated in one framework which is based on either conventional interpretation or new interpretation of multidimensional fuzzy implication (MFI) and a novel notion of fuzzy pattern vector (FPV). A completely independent design methodology has been developed on a unified framework which has been thoroughly tested on several synthetic and real life data. In the field of soft computing such application-oriented design study is unique in nature. The monograph essentially mimics the cognitive process of human decision making. It carries a message of perceptual integrity in representational diversity. The monograph is very much useful to the researchers in the area of pattern classification and computer vision. It is useful for the academics as well as for the professional computer scientists of different research and development centers of industry. The monograph has a combined flavor of theory and practice. The monograph is basically a collection of research contributions of Prof. Kumar S. Ray at Electronics and Communication Sciences Unit of Indian Statistical Institute, Kolkata. Prof. Kumar S. Ray is grateful to Mandrita Mondal for her constant encouragement and support to complete the monograph. Kumar S. Ray vii
Contents 1 Soft Computing Approach to Pattern Classification and Object Recognition 1.1 Introduction 2 1. 2 Passage Between Conventional Approach to Pattern Classification(Object Recognition) and Soft Computing Approach to Pattern Classification(Object Recognition) References 2 Pattern Classification Based on Conventional Interpretation of mFI 2.1 Introduction 2.2 Statement of the problem 557 2.3 Existing Method to Solve Fuzzy Relation Equation 2.4 Modified Approach to Solve Fuzzy Relational Equation 2.4.1 Derivative of max-Function 2. 4.2 Derivative of Min-Function 2.4.3 Modified Approach to the Computation of Derivative of Fuzzy-Max and Fuzzy-Min Functions 2.4.4 Algorithm for the Estimation of R 2.4.5 Illustration of the Modified Approach to the estimation of究 2.5 Design of the Classifier Based on Fuzzy Relational Calculus 27 2.6. 1 Classification of First Synthetic Data 2.6.2 Classification of Second Synthetic Data 2.7 Applications 2.7.1 Experimental Results 9 Conclusion
Contents 1 Soft Computing Approach to Pattern Classification and Object Recognition ................................ 1 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Passage Between Conventional Approach to Pattern Classification (Object Recognition) and Soft Computing Approach to Pattern Classification (Object Recognition) . . . . . . 5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2 Pattern Classification Based on Conventional Interpretation of MFI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.2 Statement of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.3 Existing Method to Solve Fuzzy Relation Equation . . . . . . . . . . 18 2.4 Modified Approach to Solve Fuzzy Relational Equation . . . . . . 22 2.4.1 Derivative of Max-Function . . . . . . . . . . . . . . . . . . . . . 22 2.4.2 Derivative of Min-Function . . . . . . . . . . . . . . . . . . . . . 23 2.4.3 Modified Approach to the Computation of Derivative of Fuzzy-Max and Fuzzy-Min Functions . . . . . . . . . . . . 24 2.4.4 Algorithm for the Estimation of < . . . . . . . . . . . . . . . . 26 2.4.5 Illustration of the Modified Approach to the Estimation of <. . . . . . . . . . . . . . . . . . . . . . . . . 27 2.5 Design of the Classifier Based on Fuzzy Relational Calculus . . . 27 2.6 Effectiveness of the Proposed Method . . . . . . . . . . . . . . . . . . . 32 2.6.1 Classification of First Synthetic Data. . . . . . . . . . . . . . . 33 2.6.2 Classification of Second Synthetic Data. . . . . . . . . . . . . 35 2.7 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.7.1 Experimental Results. . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.8 Comparative Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 2.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 ix
3 Pattern Classification Based on New Interpretation of MFI 43 3.1 Introduction 3.2 Statement of the problem 3.3 Design of the Multidimensional Classifier Based on Fuzzy Relational Calculus(FRC) 46 3.4 Effectiveness of the Proposed Method 3.4.1 Classification of First Synthetic Data 3.4.2 Classification of Second Synthetic Data 3.5 Applications 3.5.1 Experiment on the Classification of the Telugu Vowels 3.5.2 Experiment on the Classification of Bengali Vowels 3.6 Conclusion References 4 Pattern Classification Based on New Interpretation of MFT and Floating Point Genetic algorithm 61 4.1 Introduction 4.2 Solution of Fuzzy Relational Equation by GA 63 4.2.1 Computational Details to Solve Eq(4. 4) Using Floating Point ga 4.2.2 Algorithm for the Estimation of 4.3 Designing of the Classifier Based on the Fuzzy Relational Calculus and genetic algorithms 4.4 Effectiveness of the Proposed Method 4.4.1 Classification of First Synthetic Data 68 4. 4.2 Classification of Second Synthetic Data 4.5 Application of the Proposed Method for Vowel Classification problem 4.5.1 Experiment on the Classification of the Telugu Vowels 7 .6 Comparative Study 4.7 Benchmark stud 73 5 Neuro-Genetic Approach to Pattern Classification Based on the New Interpretation of MFI 5.1 Introduction 77 5.2 Implementation of the New Interpretation of MFI on MLP Type Neural Network 5.3 Genetic-Algorithm-Based Learning Environment 5.3.1 Genetic Algorithm for Global Optimizatio 5.3.2 Backpropagation Versus GA
3 Pattern Classification Based on New Interpretation of MFI. . . . . . 43 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.2 Statement of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.3 Design of the Multidimensional Classifier Based on Fuzzy Relational Calculus (FRC) . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 3.4 Effectiveness of the Proposed Method . . . . . . . . . . . . . . . . . . . 48 3.4.1 Classification of First Synthetic Data. . . . . . . . . . . . . . . 49 3.4.2 Classification of Second Synthetic Data. . . . . . . . . . . . . 49 3.5 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.5.1 Experiment on the Classification of the Telugu Vowels . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.5.2 Experiment on the Classification of Bengali Vowels . . . . 57 3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4 Pattern Classification Based on New Interpretation of MFI and Floating Point Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . 61 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 4.2 Solution of Fuzzy Relational Equation by GA . . . . . . . . . . . . . 63 4.2.1 Computational Details to Solve Eq. (4.4) Using Floating Point GA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 4.2.2 Algorithm for the Estimation of < . . . . . . . . . . . . . . . . 66 4.3 Designing of the Classifier Based on the Fuzzy Relational Calculus and Genetic Algorithms . . . . . . . . . . . . . . . . . . . . . . 67 4.4 Effectiveness of the Proposed Method . . . . . . . . . . . . . . . . . . . 68 4.4.1 Classification of First Synthetic Data. . . . . . . . . . . . . . . 68 4.4.2 Classification of Second Synthetic Data. . . . . . . . . . . . . 68 4.5 Application of the Proposed Method for Vowel Classification Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 4.5.1 Experiment on the Classification of the Telugu Vowels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 4.6 Comparative Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 4.7 Benchmark Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 4.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 5 Neuro-Genetic Approach to Pattern Classification Based on the New Interpretation of MFI . . . . . . . . . . . . . . . . . . . . . . . . 77 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 5.2 Implementation of the New Interpretation of MFI on MLP Type Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 5.3 Genetic-Algorithm-Based Learning Environment. . . . . . . . . . . . 79 5.3.1 Genetic Algorithm for Global Optimization . . . . . . . . . . 79 5.3.2 Backpropagation Versus GA . . . . . . . . . . . . . . . . . . . . 80 x Contents
5.3.3 Four Basic Features of ga 5.4 Improvement of the Network Performance Using Regularization 5.5 Formulation of the problem 5.6 Numerical Examples 5.6.1 Classification of Synthetic Data 5.6.2 Classification of Vowels 28887%9 5.6.3 Classification with Regularization 5.7 Conclusion References 6 Knowledge-Based Occluded Object Recognition Based on New Interpretation of MFI and Floating Point Genetic 6.1 Introduction 6.2 Statement of the problem 6.3 Design of Knowledge-Based Occluded Object Recognizer 6.3.1 Local Feature Extraction 3.2 Training phase 108 6.3.3 Testing Phase l11 63 4 Condition of recognition l11 6.4 Control scheme of the vision process 6.5 Effectiveness of the Proposed Classifier 116 6.5.1 Recognition of the scene Consist of model Objects Mi and M2 116 6.5.2 Recognition of the Scene Consist of Model Objects M, M. and m 118 ro-Fuzzy Approach to Occluded Object Recognition Based on New Interpretation of MFI .2 Implementation of the New Interpretation of MFI on Back Propagation-Type Neural Network 122 7.3 Formulation of the problem 7.3.1 Local Feature Extraction 7.3.2 Process of fuzzification 7.3.3 Assignment of the Membership Function to the Consequent Part of the If-Then Rules 7. 3. 4 Process of Defuzzification 7.3.5 Generation of the Model-Based Object Recognition
5.3.3 Four Basic Features of GA. . . . . . . . . . . . . . . . . . . . . . 80 5.3.4 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.4 Improvement of the Network Performance Using Regularization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5.5 Formulation of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5.6 Numerical Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 5.6.1 Classification of Synthetic Data . . . . . . . . . . . . . . . . . . 87 5.6.2 Classification of Vowels . . . . . . . . . . . . . . . . . . . . . . . 98 5.6.3 Classification with Regularization . . . . . . . . . . . . . . . . . 99 5.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 6 Knowledge-Based Occluded Object Recognition Based on New Interpretation of MFI and Floating Point Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 6.2 Statement of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.3 Design of Knowledge-Based Occluded Object Recognizer . . . . . 105 6.3.1 Local Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . 106 6.3.2 Training Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 6.3.3 Testing Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 6.3.4 Condition of Recognition . . . . . . . . . . . . . . . . . . . . . . . 111 6.4 Control Scheme of the Vision Process . . . . . . . . . . . . . . . . . . . 113 6.5 Effectiveness of the Proposed Classifier . . . . . . . . . . . . . . . . . . 116 6.5.1 Recognition of the Scene Consist of Model Objects M1 and M2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 6.5.2 Recognition of the Scene Consist of Model Objects M1, M2, and M3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 6.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 7 Neuro-Fuzzy Approach to Occluded Object Recognition Based on New Interpretation of MFI . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 7.2 Implementation of the New Interpretation of MFI on Back Propagation-Type Neural Network. . . . . . . . . . . . . . . . . . . . . . 122 7.3 Formulation of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 123 7.3.1 Local Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . 123 7.3.2 Process of Fuzzification. . . . . . . . . . . . . . . . . . . . . . . . 125 7.3.3 Assignment of the Membership Function to the Consequent Part of the If–Then Rules . . . . . . . . . . . . . . 125 7.3.4 Process of Defuzzification . . . . . . . . . . . . . . . . . . . . . . 126 7.3.5 Generation of the Model-Based Object Recognition Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Contents xi
7.4 Experimental Results 7.4.1 Case Study 1 7.4.2 Case Study 2 7.5 Conclusion 142 References Appendix A: on and F Pattern vecto 147 ppendix B: Good Function Appendix C: Operators of Fuzzy Equation 163 Appendix D: Genetic Operators on the Floating Point Chromosomes
7.4 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 7.4.1 Case Study 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 7.4.2 Case Study 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 7.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 Appendix A: Multidimensional Fuzzy Implication and Fuzzy Pattern Vector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Appendix B: Good Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Appendix C: Operators of Fuzzy Equation . . . . . . . . . . . . . . . . . . . 163 Appendix D: Genetic Operators on the Floating Point Chromosomes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 xii Contents
