Figure 5:Atypical data.The network classifies these correctly,even though they are quite unlike any thing in the training set. of the image,while the DSP performs the size-normalization and the classification. The overall throughput of the digit recognizer including image acquisition is 10 to 12 classifications per second and is limited mainly by the normalization step.On normalized digits,the DSP performs more than 30 classifications per second. 6 CONC2 USION Back-propagation learning was successfully applied to a large,real-world task.Our results appear to be at the state of the art in handwritten digit recognition.The network had many connections but relatively few free parameters.The network architecture and the constraints on the weights were designed to incorporate geo- metric knowledge about the task into the system.Because of its architecture,the network could be trained on a low-level representation of data that had minimal preprocessing (as opposed to elaborate feature extraction).Because of the redun- dant nature of the data and because of the constraints imposed on the network,the learning time was relatively short considering the size of the training set.Scaling properties were far better than one would expect just from extrapolating results of back-propagation on smaller,artificial problems.Preliminary results on alphanu- meric characters show that the method can be directly extended to larger tasks. The final network of connections and weights obtained by back-propagation learn- ing was readily implementable on commercial digital signal processing hardware. Throughput rates,from camera to classified image,of more than ten digits per second were obtained. Acknowledgments We thank the US Postal Service and its contractors for providing us with the zip- code database.We thank Henry Baird for useful discussions and for providing the printed-font database. References Bottou,L.-Y.and Le Cun,Y.(1989).SN2:A Simulator for Connectionist Models. Neuristique SA,Paris,France.￾✂✁☎✄✍✆✞✡✠✁￾✣✌ ❞❥♠❾◗⑤✌❦⑦❜●❳❩❝❃P✌❳❲❥✐❳✌✸ ❡❸✣✼✿❁✌✼●❥♠❱✜❆❉❋❊▲✲❜✐❝⑦❳❲❈❊❈❙❦➂➁✍✼●❈❅❥❙❸✌✼❘❈❙✼❻❜✐❆❉❋❊❋❙✼❍❜✐❥❊❝❣❾❛❂✣✼●➎❉✼●❁ ❥❙❸✌❆▼❀✌❷❉❸✧❥❙❸✣✼●❾ ❳❲❋❊✼✿❽❉❀✣❦⑥❥❊✼✩❀✌❁✌❝❣❦❣▲❉✼✵❳❩❁◗❾❛❥❊❸✌❦❣❁✌❷❵❦⑥❁✧❥❊❸✌✼✩❥❙❋❬❳❩❦❣❁✌❦❣❁✌❷✫❈❊✼●❥❍✸ ❆❲⑧➊❥❙❸✌✼✩❦❣❧✲❳❩❷❉✼❉❂❩❱☛❸✌❦❣❝⑥✼☛❥❊❸✌✼✿❑✩❏✌➑✈⑤✍✼●❋⑧➄❆❉❋❊❧✖❈✜❥❙❸✌✼✩❈❊❦⑥➅❘✼✐⑩❶❁✣❆❉❋❙❧✧❳❲❝❣❦❣➅❍❳❲❥❊❦❣❆❉❁❻❳❲❁✍P➞❥❙❸✌✼✿❜✐❝⑦❳❲❈❊❈❙❦➂➁➃❜●❳❩❥❙❦❣❆❉❁❤✸ ❡❸✌✼✿❆❲➎▼✼●❋❬❳❩❝❣❝✽❥❙❸✌❋❊❆❉❀✌❷▼❸✌⑤✌❀✌❥❅❆❲⑧❴❥❙❸✌✼❵P◗❦❣❷❉❦❣❥☛❋❙✼❍❜❹❆❉❷❉❁✣❦⑥➅❘✼●❋✩❦⑥❁✻❜✐❝❣❀✍P◗❦❣❁✌❷❵❦⑥❧✧❳❲❷▼✼✵❳❉❜❘❽❉❀✣❦⑥❈❊❦❣❥❙❦❣❆❉❁✧❦❣❈✿➇❍♦❵❥❊❆ ➇ ✤☎❜✐❝⑦❳❲❈❊❈❙❦➂➁➃❜●❳❩❥❙❦❣❆❉❁✌❈✩⑤✻✼●❋✿❈❊✼❍❜✐❆❉❁✻P✢❳❲❁✍P✧❦❣❈✩❝❣❦❣❧✖❦❣❥❙✼❍P☎❧✧❳❲❦❣❁✌❝❣❾❻❫◗❾✲❥❊❸✌✼❻❁✌❆▼❋❙❧✲❳❩❝❣❦⑥➅✕❳❲❥❙❦❣❆▼❁✖❈❙❥❊✼●⑤❃✸✒✑❅❁ ❁✌❆▼❋❙❧✲❳❩❝❣❦⑥➅❘✼❍P❻P◗❦❣❷❉❦❣❥❙❈✕❂◗❥❙❸✌✼❵❑✵❏✣➑✢⑤✻✼●❋♠⑧➄❆▼❋❙❧☎❈☛❧✖❆❉❋❊✼✑❥❊❸✍❳❲❁✰q❉♦❵❜✐❝⑦❳❲❈❊❈❙❦➂➁➃❜●❳❩❥❙❦❣❆❉❁✣❈☛⑤✍✼●❋❅❈❙✼❍❜❹❆❉❁✍P✽✸ ✂ ③✢→➔③✁❵↔➌✉ ➓→➔ ❄✜❳❉❜❊▲▼⑩❶⑤✌❋❊❆❉⑤✍❳❩❷❛❳❩❥❙❦❣❆❉❁❻❝❣✼❍❳❩❋❙❁✌❦❣❁✌❷✿❱☛❳❲❈❇❈❙❀✍❜●❜❹✼●❈❙❈⑧➄❀✌❝❣❝❣❾✲❳❲⑤✣⑤✌❝❣❦⑥✼✕P❻❥❙❆❵❳✿❝⑦❳❲❋❊❷❉✼❉❂▼❋❙✼✕❳❲❝➂⑩❶❱✜❆❉❋❊❝⑦P❵❥❬❳❲❈❊▲✽✸✟✑❅❀✌❋ ❋❙✼❘❈❙❀✌❝❣❥❙❈❵❳❲⑤✌⑤✻✼❍❳❲❋✩❥❊❆✲❫✻✼❻❳❲❥✩❥❙❸✌✼❵❈❙❥✐❳❲❥❊✼❻❆❲⑧⑨❥❊❸✌✼✖❳❲❋❊❥✵❦❣❁✰❸✻❳❲❁✍P◗❱☛❋❊❦❣❥❙❥❙✼❘❁❿P◗❦❣❷▼❦⑥❥❅❋❙✼❍❜❹❆❉❷❉❁✣❦⑥❥❊❦❣❆❉❁❃✸ ❡❸✌✼ ❁✌✼●❥♠❱✜❆❉❋❊▲✢❸✍❳❉P➀❧✲❳❩❁❛❾✢❜✐❆▼❁✌❁✌✼❍❜✐❥❊❦❣❆❉❁✌❈❻❫✌❀✣❥✫❋❊✼●❝⑦❳❲❥❙❦❣➎▼✼●❝❣❾②⑧➄✼●❱ ⑧➄❋❊✼●✼✧⑤✍❳❲❋✐❳❲❧☎✼●❥❙✼●❋❊❈❍✸ ❡❸✣✼✲❁✣✼●❥♠❱⑨❆▼❋❙▲ ❳❲❋✐❜❊❸✌❦❣❥❙✼❍❜✐❥❊❀✌❋❙✼✧❳❲❁✍P➆❥❙❸✌✼❻❜✐❆❉❁✣❈❙❥❙❋✐❳❲❦❣❁◗❥❙❈✿❆▼❁✰❥❙❸✣✼❻❱⑨✼●❦❣❷▼❸❛❥❊❈✑❱✜✼●❋❙✼✖P❼✼●❈❙❦❣❷❉❁✣✼❍P✰❥❊❆✧❦⑥❁✻❜✐❆❉❋❊⑤✍❆▼❋❬❳❲❥❊✼✫❷▼✼●❆❲⑩ ❧✖✼❘❥❙❋❙❦⑦❜✩▲◗❁✌❆❲❱☛❝❣✼❍P❼❷❉✼✿❳❲❫✻❆❉❀✌❥✩❥❊❸✌✼✿❥❬❳❩❈❙▲✰❦❣❁◗❥❙❆❻❥❊❸✌✼❵❈❙❾◗❈❙❥❊✼●❧④✸✩❄❴✼✕❜●❳❲❀✌❈❊✼❻❆❲⑧❴❦❣❥❙❈✩❳❲❋❬❜❊❸✌❦❣❥❊✼❍❜✐❥❙❀✣❋❙✼❉❂❤❥❙❸✌✼ ❁✌✼●❥♠❱✜❆❉❋❊▲❿❜❹❆❉❀✌❝⑦P②❫✻✼✖❥❙❋✐❳❲❦❣❁✌✼❍P②❆▼❁✝❳✧❝❣❆❲❱✜⑩❶❝❣✼●➎▼✼●❝❴❋❊✼●⑤✌❋❙✼❘❈❙✼●❁◗❥❬❳❩❥❙❦❣❆❉❁✈❆❲⑧✵P✌❳❩❥❬❳☎❥❙❸✻❳❲❥✿❸✍❳▼P②❧✖❦❣❁✌❦❣❧✲❳✕❝ ⑤✌❋❊✼●⑤✌❋❙❆✌❜✐✼❘❈❙❈❙❦❣❁✌❷➀➜❶❳❲❈✩❆▼⑤✌⑤✍❆▼❈❙✼❍P ❥❙❆☎✼●❝⑦❳❲❫✻❆❉❋✐❳❲❥❙✼✱⑧➄✼❍❳❩❥❙❀✌❋❊✼✫✼❹➟❼❥❊❋❬❳▼❜✐❥❙❦❣❆❉❁✻➝●✸✿❄❇✼❍❜●❳❲❀✣❈❙✼❻❆❲⑧✜❥❙❸✌✼❵❋❙✼❍P❼❀✌❁◗⑩ P✌❳❩❁❛❥❇❁✍❳❲❥❊❀✌❋❙✼❅❆❲⑧✽❥❙❸✌✼✩P✌❳❲❥✐❳✿❳❲❁✍P❵❫✻✼❍❜●❳❩❀✌❈❙✼❅❆❲⑧❃❥❊❸✌✼✵❜✐❆▼❁✌❈❙❥❊❋❬❳❩❦⑥❁◗❥❊❈⑨❦❣❧✖⑤✻❆❉❈❊✼❍P✱❆❉❁❵❥❙❸✌✼❅❁✌✼●❥♠❱✜❆❉❋❙▲✽❂▼❥❙❸✌✼ ❝❣✼❍❳❲❋❊❁✌❦❣❁✌❷✖❥❊❦❣❧✖✼✩❱❨❳❩❈✩❋❙✼●❝⑦❳❲❥❊❦❣➎❉✼●❝❣❾✧❈❙❸✌❆▼❋❙❥✿❜✐❆▼❁✌❈❙❦⑦P◗✼●❋❊❦❣❁✌❷✧❥❙❸✌✼❻❈❊❦⑥➅❘✼✫❆❩⑧⑨❥❙❸✌✼❻❥❊❋❬❳❩❦⑥❁✣❦⑥❁✣❷❻❈❙✼●❥✕✸✹❏✣❜●❳❲❝❣❦❣❁✌❷ ⑤✌❋❊❆❉⑤✻✼●❋❙❥❊❦⑥✼❘❈✑❱✜✼●❋❙✼❅⑧❶❳❩❋⑨❫✻✼●❥❙❥❊✼●❋✑❥❊❸✍❳❲❁✧❆▼❁✌✼✑❱✜❆❉❀✌❝⑦P➞✼✐➟✌⑤✍✼❍❜❹❥✮➍✐❀✌❈❊❥✜⑧➄❋❙❆▼❧ ✼✐➟✌❥❙❋❬❳❩⑤✍❆▼❝⑦❳❲❥❙❦❣❁✌❷❵❋❙✼❘❈❙❀✌❝❣❥❙❈❅❆❲⑧ ❫✍❳▼❜❙▲▼⑩♠⑤✌❋❙❆▼⑤✍❳❲❷◗❳❲❥❊❦⑥❆▼❁✰❆❉❁②❈❙❧✧❳❲❝❣❝❣✼●❋❍❂✽❳❩❋❙❥❙❦➂➁➃❜❹❦ ❳❩❝❴⑤✌❋❊❆❉❫✌❝❣✼●❧☎❈❍✸❻➑➊❋❙✼❘❝⑥❦❣❧☎❦⑥❁✻❳❲❋❙❾☎❋❙✼●❈❊❀✌❝❣❥❙❈❵❆❉❁✢❳❲❝❣⑤✌❸✍❳❩❁◗❀◗⑩ ❧✖✼❘❋❙❦⑦❜✑❜❊❸✍❳❩❋❬❳❉❜❹❥❙✼●❋❊❈✵❈❊❸✌❆❲❱➌❥❙❸✍❳❩❥☛❥❙❸✌✼✩❧✖✼❘❥❙❸✌❆✌P✖❜❘❳❲❁✧❫✍✼❵P◗❦❣❋❙✼❍❜✐❥❊❝❣❾✖✼✐➟✌❥❙✼❘❁✍P◗✼❍P✧❥❊❆❻❝⑦❳❲❋❙❷▼✼●❋☛❥❬❳❩❈❙▲◗❈❍✸ ❡❸✌✼✩➁✻❁✍❳❲❝❤❁✌✼●❥♠❱✜❆❉❋❙▲☎❆❲⑧⑨❜✐❆▼❁✌❁✌✼❍❜✐❥❊❦❣❆❉❁✌❈✩❳❲❁✍P☎❱✜✼●❦❣❷❉❸◗❥❙❈❅❆❉❫✌❥✐❳❲❦❣❁✌✼❍P☎❫◗❾✖❫✍❳▼❜❊▲❉⑩♠⑤✌❋❊❆❉⑤✍❳❩❷❛❳❲❥❊❦❣❆❉❁✖❝❣✼❍❳❩❋❙❁◗⑩ ❦❣❁✌❷✰❱☛❳❲❈✿❋❊✼❍❳❉P❼❦⑥❝❣❾②❦❣❧✖⑤✌❝❣✼●❧☎✼●❁◗❥❬❳❲❫✣❝⑥✼✱❆❉❁✝❜✐❆▼❧✖❧✖✼❘❋❬❜✐❦⑦❳❲❝❴P◗❦❣❷▼❦⑥❥✐❳❲❝❴❈❊❦❣❷❉❁✍❳❩❝❴⑤✌❋❙❆✌❜✐✼❘❈❙❈❙❦❣❁✌❷✢❸✻❳❲❋❬P❼❱❨❳❩❋❙✼❉✸ ❡❸✌❋❊❆❉❀✌❷▼❸✌⑤✌❀✌❥❻❋✐❳❲❥❙✼❘❈❍❂✜⑧➄❋❊❆❉❧ ❜❘❳❲❧✖✼❘❋❬❳②❥❙❆✝❜❹❝ ❳❩❈❙❈❊❦➫➁✻✼❍P✈❦⑥❧✧❳❲❷▼✼❉❂➊❆❲⑧✿❧✖❆▼❋❙✼☎❥❙❸✍❳❩❁ ❥❊✼●❁ P◗❦❣❷▼❦⑥❥❊❈✫⑤✻✼●❋ ❈❙✼✕❜✐❆❉❁✍P➆❱⑨✼●❋❊✼✿❆❉❫✣❥❬❳❲❦❣❁✌✼✕P✽✸ ✄✆☎✞✝✠✟☛✡✌☞✝☎✠✎✍✄✑✏✠✒✟✔✓✖✕ ❭✢✼✩❥❙❸✍❳❩❁✌▲✧❥❙❸✌✼❵➏✑❏✰➑❤❆❉❈❊❥❬❳❩❝✚❏❼✼●❋❙➎◗❦⑦❜✐✼❻❳❲❁✍P☎❦❣❥❙❈✩❜✐❆❉❁◗❥❙❋✐❳❉❜✐❥❊❆❉❋❊❈✑⑧❆❉❋✑⑤✣❋❙❆❲➎◗❦⑦P◗❦❣❁✌❷✫❀✣❈✑❱☛❦❣❥❙❸✰❥❊❸✌✼✿➅●❦❣⑤◗⑩ ❜✐❆✌P◗✼✫P✣❳❲❥❬❳❩❫✍❳❲❈❊✼❉✸✜❭✢✼✑❥❊❸✍❳❲❁✌▲✰◆❖✼●❁✌❋❙❾✰❄✜❳❲❦❣❋❬P➞⑧➄❆▼❋☛❀✌❈❙✼❹⑧➄❀✌❝✮P❼❦⑥❈✐❜✐❀✌❈❊❈❙❦❣❆❉❁✌❈✩❳❲❁✍P➞⑧➄❆▼❋☛⑤✌❋❙❆❲➎◗❦⑦P◗❦❣❁✌❷❵❥❙❸✌✼ ⑤✌❋❊❦⑥❁◗❥❊✼❍P▼⑩ ⑧➄❆❉❁◗❥✑P✌❳❩❥❬❳❲❫✻❳❲❈❙✼▼✸ ①✘✗✠✙✚✗✔✛✜✗✔✢✤✣✠✗✦✥ ❄❇❆❉❥❙❥❊❆❉❀❃❂✌✺❇✸ ⑩➄✷❻✸❉❳❲❁✍P☎✺✽✼✩✾✜❀✌❁❃❂▼✷✹✸✣➜❬➇✕➋❉➠▼➋❉➝●✸★✧✌✩✫✪✭✬✯✮✰✧✻➨✢✒✁✎✒✱❣➥✍✌❙➩❲➧ ✠❍➩❲➧✳✲⑨➩ ✜ ✜ ☛✵✴ ✌♠➨➄➩ ✜✌➨✖✙✑✌★✶➩✞✷ ☛✸✱✙●✸ ♥❖✼●❀✌❋❙❦❣❈❊❥❙❦⑦❽▼❀✌✼✫❏❞ ❂✌➑✮❳❩❋❙❦❣❈❍❂ ✡✌❋✐❳❲❁✍❜❹✼❉✸