of 10 units:one per class.When a pattern belonging to class Fis presented,the desired output is +1 for the Eh output unit,and-1 for the other output units. Figure 3:Input image (left),weight vector (center),and resulting feature map (right).The feature map is obtained by scanning the input image with a single neuron that has a local receptive field,as indicated.White represents -1,black represents +1. A fully connected network with enough discriminative power for the task would have far too many parameters to be able to generalize correctly.Therefore a restricted connection-scheme must be devised,guided by our prior knowledge about shape recognition.There are well-known advantages to performing shape recognition by detecting and combining local features.We have required our network to do this by constraining the connections in the first few layers to be local.In addition,if a feature detector is useful on one part of the image,it is likely to be useful on other parts of the image as well.One reason for this is that the salient features of a distorted character might be displaced slightly from their position in a typical char- acter.One solution to this problem is to scan the input image with a single neuron that has a local receptive field,and store the states of this neuron in corresponding locations in a layer called a eea:ure map (see figure 3).This operation is equivalent to a convolution with a small size kernel,followed by a squashing function.The process can be performed in parallel by implementing the feature map as a plane of neurons whose weight vectors are constrained to be equal.That is,units in a feature map are constrained to perform the same operation on different parts of the image.An interesting side-effect of this weige EPrig technique,already described in (Rumelhart,Hinton and Williams,1986),is to reduce the number of free param- eters by a large amount,since a large number of units share the same weights.In addition,a certain level of shift invariance is present in the system:shifting the input will shift the result on the feature map,but will leave it unchanged otherwise. In practice,it will be necessary to have multiple feature maps,extracting different features from the same image.❆❲⑧✿➇❍♦☎❀✌❁✣❦⑥❥❊❈ ✢✱❆❉❁✌✼❻⑤✻✼●❋✫❜✐❝⑦❳❩❈❙❈❍✸☎❭↕❸✌✼❘❁✝❳✖⑤✻❳❲❥❙❥❊✼●❋❙❁✈❫✍✼❘❝⑥❆▼❁✌❷❉❦❣❁✌❷☎❥❙❆④❜❹❝ ❳❩❈❙❈✁￾❖❦❣❈✵⑤✣❋❙✼●❈❊✼●❁◗❥❙✼❍P✽❂❇❥❙❸✌✼ P◗✼●❈❊❦❣❋❙✼❍P✧❆▼❀✌❥❙⑤✣❀✌❥☛❦❣❈✄✂✫➇❅⑧❆❉❋✜❥❙❸✌✼✁￾ ❥❙❸✧❆▼❀✌❥❙⑤✌❀✣❥☛❀✌❁✌❦❣❥❍❂✌❳❩❁✍P✌✞➙➇❅⑧❆❉❋✜❥❙❸✌✼✿❆▼❥❙❸✌✼❘❋☛❆❉❀✌❥❊⑤✌❀✌❥☛❀✌❁✌❦❣❥❊❈❍✸ ￾✂✁☎✄✍✆✞✡✠✆☎ ✌✞✝♠❁✌⑤✌❀✌❥❵❦❣❧✲❳❲❷▼✼✖➜ ❝❣✼✐⑧➄❥❬➝❘❂➊❱⑨✼❘❦⑥❷▼❸◗❥✫➎▼✼❍❜✐❥❙❆▼❋✖➜♠❜✐✼●❁◗❥❙✼❘❋❬➝●❂☛❳❩❁✍P✢❋❊✼●❈❙❀✌❝❣❥❊❦⑥❁✣❷✰⑧➄✼✕❳❲❥❙❀✣❋❙✼✧❧✲❳❩⑤ ➜➄❋❊❦❣❷❉❸◗❥❬➝❘✸ ❡❸✣✼❻⑧➄✼❍❳❩❥❙❀✌❋❊✼✧❧✲❳❲⑤②❦❣❈✫❆▼❫✌❥❬❳❩❦❣❁✌✼❍P②❫◗❾✢❈❬❜●❳❩❁✌❁✌❦❣❁✌❷②❥❙❸✌✼✧❦❣❁✌⑤✌❀✣❥✿❦❣❧✲❳❲❷▼✼✫❱☛❦❣❥❊❸ ❳✧❈❊❦⑥❁✣❷❉❝❣✼ ❁✌✼●❀✣❋❙❆❉❁t❥❊❸✍❳❲❥❵❸✍❳❲❈✖❳ ❝❣❆✌❜●❳❲❝☛❋❊✼❍❜✐✼●⑤✌❥❊❦❣➎❉✼☎➁✍✼●❝⑦P✽❂☛❳❲❈❵❦❣❁✍P◗❦⑦❜●❳❩❥❙✼❍P✽✸✈❭↕❸✌❦❣❥❊✼✖❋❙✼❘⑤✌❋❙✼●❈❊✼●❁◗❥❙❈☎⑩●➇❉❂➊❫✌❝⑦❳❉❜❊▲ ❋❙✼❘⑤✌❋❙✼●❈❊✼●❁◗❥❙❈✁✂❵➇▼✸ ❞ ⑧❀✌❝❣❝⑥❾✩❜✐❆❉❁✣❁✌✼❍❜✐❥❊✼❍P❵❁✌✼●❥♠❱✜❆❉❋❊▲✵❱☛❦❣❥❊❸✿✼●❁✌❆▼❀✌❷❉❸✫P❼❦⑥❈✐❜✐❋❙❦❣❧☎❦⑥❁✻❳❲❥❙❦❣➎▼✼✮⑤✻❆❲❱✜✼●❋✽⑧➄❆▼❋❃❥❊❸✌✼⑨❥✐❳❲❈❊▲✵❱✜❆❉❀✣❝ P❅❸✍❳✕➎▼✼ ⑧❶❳❩❋✑❥❙❆◗❆☎❧✲❳❲❁◗❾❻⑤✍❳❩❋❬❳❩❧✖✼●❥❊✼●❋❙❈✩❥❙❆☎❫✍✼❻❳❲❫✣❝⑥✼✱❥❙❆✖❷▼✼●❁✌✼●❋✐❳❲❝❣❦❣➅●✼❻❜✐❆❉❋❊❋❙✼❍❜✐❥❊❝❣❾❛✸ ❡❸✌✼❘❋❙✼✐⑧➄❆▼❋❙✼✖❳➞❋❙✼●❈❊❥❙❋❙❦⑦❜✐❥❊✼❍P ❜✐❆▼❁✌❁✌✼❍❜✐❥❊❦❣❆❉❁◗⑩♠❈❬❜❊❸✌✼●❧☎✼✧❧❻❀✌❈❙❥✿❫✻✼✰P◗✼●➎◗❦❣❈❙✼❍P✏❂❴❷❉❀✣❦ P❼✼❍P✢❫◗❾✢❆❉❀✣❋✫⑤✌❋❊❦❣❆❉❋❵▲❛❁✣❆❲❱☛❝⑥✼✕P◗❷❉✼✧❳❲❫✻❆❉❀✌❥❵❈❙❸✍❳❩⑤✍✼ ❋❙✼✕❜✐❆❉❷▼❁✌❦❣❥❙❦❣❆❉❁❃✸ ❡❸✌✼●❋❙✼✖❳❩❋❙✼✿❱✜✼●❝❣❝➂⑩❶▲◗❁✌❆❲❱☛❁✰❳▼P◗➎❉❳❲❁◗❥❬❳❩❷❉✼●❈❅❥❙❆☎⑤✍✼●❋⑧➄❆❉❋❊❧✖❦❣❁✌❷❵❈❙❸✍❳❩⑤✍✼❵❋❙✼✕❜✐❆❉❷▼❁✌❦❣❥❙❦❣❆❉❁✰❫◗❾ P◗✼●❥❊✼❍❜✐❥❊❦⑥❁✣❷④❳❲❁✍P②❜✐❆▼❧❻❫✌❦❣❁✌❦❣❁✌❷❻❝❣❆❼❜❘❳❲❝❤⑧➄✼❍❳❩❥❙❀✌❋❊✼●❈❍✸❻❭✢✼❵❸✍❳✕➎▼✼✫❋❊✼❍❽▼❀✌❦❣❋❙✼❍P②❆▼❀✌❋✿❁✌✼●❥♠❱✜❆❉❋❊▲✰❥❙❆✰P◗❆✧❥❊❸✌❦❣❈ ❫◗❾④❜✐❆▼❁✌❈❙❥❊❋❬❳❲❦❣❁✌❦❣❁✌❷✧❥❊❸✌✼✖❜✐❆❉❁✣❁✌✼❍❜✐❥❊❦⑥❆▼❁✌❈❵❦⑥❁②❥❙❸✣✼❵➁✍❋❙❈❊❥✩⑧➄✼●❱❪❝ ❳✕❾▼✼●❋❙❈✱❥❙❆✧❫✍✼❻❝❣❆✌❜●❳❩❝❶✸✟✝♠❁✢❳❉P✌P❼❦⑥❥❊❦❣❆❉❁❃❂✽❦➂⑧ ❳☎⑧➄✼✕❳❲❥❙❀✣❋❙✼✰P◗✼●❥❊✼❍❜✐❥❙❆▼❋❻❦❣❈✫❀✌❈❊✼✐⑧➄❀✌❝☛❆▼❁✢❆❉❁✌✼☎⑤✍❳❲❋❊❥✫❆❩⑧✑❥❙❸✌✼✧❦❣❧✲❳❩❷❉✼❉❂✏❦⑥❥✱❦⑥❈❵❝❣❦❣▲❉✼●❝❣❾✰❥❊❆④❫✻✼✖❀✌❈❊✼✐⑧➄❀✌❝☛❆▼❁ ❆❉❥❊❸✌✼●❋✜⑤✍❳❲❋❊❥❙❈✜❆❲⑧❃❥❊❸✌✼✑❦❣❧✲❳❩❷❉✼⑨❳❩❈⑨❱✜✼●❝❣❝♠✸ ✑❅❁✌✼✩❋❙✼❍❳❩❈❙❆▼❁❻⑧➄❆▼❋❴❥❙❸✌❦❣❈❴❦❣❈❴❥❊❸✍❳❲❥✜❥❙❸✣✼✑❈❬❳❩❝❣❦⑥✼❘❁❛❥✚⑧➄✼❍❳❲❥❊❀✌❋❙✼❘❈⑨❆❩⑧✮❳ P◗❦❣❈❙❥❊❆❉❋❊❥❙✼❍P☎❜❙❸✻❳❲❋❬❳▼❜✐❥❙✼❘❋⑨❧☎❦⑥❷▼❸◗❥❃❫✻✼✑P◗❦❣❈❙⑤✌❝⑦❳❉❜❹✼❍P❵❈❙❝❣❦❣❷❉❸◗❥❙❝❣❾✩⑧❋❙❆❉❧❪❥❙❸✌✼❘❦⑥❋❇⑤✍❆▼❈❙❦❣❥❙❦❣❆❉❁❻❦❣❁❻❳✩❥♠❾◗⑤✌❦⑦❜●❳❲❝✻❜❙❸✻❳❲❋♠⑩ ❳❉❜❹❥❙✼●❋✕✸ ✑❅❁✌✼✑❈❊❆❉❝❣❀✌❥❊❦⑥❆▼❁❻❥❙❆❵❥❙❸✌❦❣❈✜⑤✌❋❙❆▼❫✌❝❣✼●❧✠❦❣❈❴❥❊❆✫❈✐❜●❳❲❁✧❥❙❸✣✼✑❦❣❁✌⑤✌❀✌❥✜❦❣❧✲❳❩❷❉✼✜❱☛❦⑥❥❊❸✖❳✿❈❊❦❣❁✌❷❉❝❣✼☛❁✌✼●❀✣❋❙❆❉❁ ❥❙❸✻❳❲❥✜❸✍❳❲❈☛❳✿❝❣❆✌❜●❳❩❝✍❋❊✼❍❜✐✼●⑤✌❥❊❦❣➎❉✼❅➁✍✼●❝⑦P✽❂✌❳❩❁✍P❵❈❙❥❊❆❉❋❙✼✩❥❊❸✌✼✩❈❙❥❬❳❩❥❙✼●❈☛❆❩⑧❃❥❙❸✌❦❣❈✜❁✌✼●❀✌❋❊❆❉❁✧❦❣❁✖❜✐❆❉❋❊❋❙✼●❈❊⑤✍❆▼❁✍P◗❦❣❁✌❷ ❝❣❆✌❜●❳❲❥❊❦⑥❆▼❁✌❈✜❦⑥❁✧❳✿❝⑦❳✕❾▼✼●❋⑨❜●❳❩❝⑥❝❣✼❍P✧❳✡✠☞☛✐➥✍✌✏✎◗➧✑☛✡✒✹➥❹➦✧➜➄❈❊✼●✼✩➁✍❷▼❀✌❋❙✼✩q❉➝●✸ ❡❸✣❦⑥❈✜❆❉⑤✻✼●❋✐❳❲❥❙❦❣❆▼❁✖❦❣❈☛✼❍❽▼❀✌❦❣➎❉❳❲❝❣✼●❁◗❥ ❥❙❆②❳✰❜✐❆❉❁◗➎▼❆❉❝❣❀✌❥❙❦❣❆▼❁✰❱☛❦❣❥❙❸✝❳✧❈❊❧✲❳❲❝❣❝➊❈❙❦❣➅●✼❻▲▼✼●❋❙❁✣✼●❝♠❂❤⑧➄❆❉❝❣❝❣❆❲❱⑨✼✕P✰❫◗❾✢❳✖❈✐❽▼❀✍❳❲❈❊❸✌❦❣❁✌❷✧⑧➄❀✌❁✻❜✐❥❙❦❣❆❉❁❤✸ ❡❸✌✼ ⑤✌❋❊❆❼❜❹✼●❈❙❈❻❜●❳❲❁②❫✻✼✫⑤✻✼●❋♠⑧❆❉❋❙❧☎✼❍P ❦⑥❁ ⑤✍❳❲❋✐❳❲❝❣❝❣✼●❝➊❫❛❾✧❦❣❧✖⑤✣❝⑥✼❘❧✖✼●❁◗❥❙❦❣❁✌❷❵❥❙❸✣✼✿⑧➄✼✕❳❲❥❙❀✣❋❙✼❻❧✲❳❩⑤✰❳❲❈✿❳❻⑤✌❝⑦❳❩❁✌✼ ❆❲⑧☛❁✌✼●❀✣❋❙❆❉❁✣❈✫❱☛❸✌❆▼❈❙✼☎❱⑨✼●❦❣❷▼❸❛❥✩➎▼✼❍❜✐❥❙❆▼❋❙❈❻❳❲❋❊✼✖❜✐❆❉❁✌❈❊❥❙❋✐❳❲❦❣❁✌✼❍P➀❥❙❆✰❫✻✼❻✼❍❽▼❀✍❳❲❝♠✸ ❡❸✍❳❲❥✱❦⑥❈✕❂❃❀✣❁✌❦❣❥❙❈✿❦❣❁✢❳ ⑧➄✼❍❳❩❥❙❀✌❋❊✼✑❧✧❳❲⑤✫❳❩❋❙✼✑❜✐❆▼❁✌❈❙❥❊❋❬❳❲❦❣❁✌✼✕P❻❥❙❆✱⑤✍✼●❋⑧➄❆❉❋❊❧ ❥❙❸✌✼❅❈❬❳❲❧☎✼⑨❆▼⑤✍✼❘❋❬❳❲❥❊❦❣❆❉❁❻❆❉❁✖P❼❦✓✽✼●❋❊✼●❁◗❥❴⑤✍❳❲❋❊❥❙❈✜❆❲⑧✽❥❙❸✌✼ ❦❣❧✲❳❩❷❉✼❉✸ ❞❁❻❦❣❁◗❥❙✼❘❋❙✼●❈❊❥❙❦❣❁✌❷❵❈❙❦⑦P◗✼✐⑩♠✼✔✓✽✼❍❜✐❥✜❆❲⑧❃❥❙❸✣❦⑥❈✡✓✔☛●➨✖✕☞✗✘✌✚✙✛✗❼➥❩➧●➨✢✜✣✕✜❥❙✼❍❜❊❸✌❁✌❦⑦❽▼❀✌✼❉❂✌❳❩❝❣❋❙✼❍❳▼P◗❾✫P◗✼●❈✐❜✐❋❙❦❣❫✻✼❍P ❦❣❁✧➜❶❚❖❀✌❧✖✼●❝❣❸✍❳❩❋❙❥✕❂❉◆☛❦❣❁◗❥❙❆▼❁✖❳❲❁✍P❵❭↕❦❣❝❣❝❣❦⑦❳❲❧❻❈❍❂◗➇✕➋❉➠ ✄▼➝●❂▼❦❣❈❴❥❙❆✿❋❊✼❍P◗❀✍❜❹✼✿❥❙❸✌✼❅❁◗❀✌❧❻❫✍✼❘❋✮❆❩⑧✽⑧➄❋❙✼❘✼✵⑤✻❳❲❋❬❳❩❧❻⑩ ✼●❥❊✼●❋❙❈✿❫◗❾✧❳❻❝⑦❳❲❋❊❷❉✼✿❳❲❧☎❆❉❀✌❁◗❥❍❂◗❈❊❦❣❁✍❜✐✼❻❳❻❝⑦❳❲❋❊❷❉✼✩❁◗❀✌❧❻❫✍✼❘❋☛❆❲⑧⑨❀✌❁✣❦⑥❥❊❈✑❈❊❸✍❳❲❋❊✼✿❥❙❸✌✼❵❈❬❳❩❧✖✼✩❱✜✼●❦❣❷❉❸◗❥❙❈✕✸✤✝♠❁ ❳❉P✣P◗❦❣❥❙❦❣❆❉❁❃❂➊❳✰❜✐✼❘❋❙❥❬❳❩❦❣❁ ❝❣✼●➎▼✼●❝✜❆❲⑧✑❈❙❸✣❦➫⑧❥✿❦❣❁❛➎❉❳❩❋❙❦⑦❳❲❁✻❜✐✼❻❦❣❈✫⑤✌❋❊✼●❈❙✼❘❁❛❥❻❦❣❁✢❥❊❸✌✼✖❈❊❾❛❈❊❥❙✼●❧ ✢❻❈❙❸✣❦➫⑧❥❙❦❣❁✌❷✧❥❙❸✌✼ ❦❣❁✌⑤✌❀✌❥❇❱☛❦⑥❝❣❝◗❈❙❸✣❦➫⑧❥✮❥❊❸✌✼✑❋❊✼●❈❙❀✣❝⑥❥❇❆❉❁❵❥❙❸✌✼❖⑧➄✼❍❳❲❥❊❀✌❋❙✼☛❧✧❳❲⑤❃❂✕❫✌❀✣❥❴❱☛❦❣❝⑥❝◗❝❣✼❍❳✕➎▼✼☛❦⑥❥➊❀✌❁✻❜❙❸✻❳❲❁✌❷▼✼❍P❵❆❉❥❊❸✌✼●❋❙❱☛❦❣❈❊✼❉✸ ✝♠❁✧⑤✌❋❬❳▼❜✐❥❙❦⑦❜✐✼▼❂✍❦❣❥☛❱☛❦❣❝❣❝✍❫✻✼✿❁✌✼✕❜✐✼●❈❙❈✐❳❲❋❊❾✰❥❙❆❻❸✍❳✕➎▼✼✩❧❻❀✌❝❣❥❙❦❣⑤✌❝❣✼✜⑧➄✼❍❳❩❥❙❀✌❋❊✼✿❧✲❳❩⑤✌❈❍❂▼✼✐➟✌❥❙❋✐❳❉❜✐❥❊❦⑥❁✣❷✲P◗❦✩✓✽✼●❋❙✼●❁◗❥ ⑧➄✼❍❳❩❥❙❀✌❋❊✼●❈☛⑧❋❙❆❉❧ ❥❙❸✌✼✿❈✐❳❲❧☎✼✑❦❣❧✲❳❩❷❉✼❉✸