3D photography on your desk Jean-Yves Bouguett and Pietro Peronatt California Institute of Technology,136-93,Pasadena,CA 91125,USA Universita di Padova,Italy [bougueti,peronal @vision.caltech.edu Abstract A simple and inexpensive approach for extracting the three- dimensional shape of objects is presented.It is based on 'weak structured lighting';it differs from other conventional struc- tured lighting approaches in that it requires very little hard- ware besides the camera:a desk-lamp,a:pencil and a checker- board.The camera faces the object,which is illuminated by the desk-lamp.The user moves a pencil in front of the light source casting a moving shadow on the object.The 3D shape of the object is extracted from the spatial and temporal location of the observed shadow.Experimental results are presented on three different scenes demonstrating that the error in reconstructing Figure 1:The general setup of the proposed method: the surface is less than 1%. The camera is facing the scene illuminated by a halogen desk 1 Introduction and Motivation lamp (left).The scene consists of objects on a plane (the desk). When an operator freely moves a stick in front of the lamp (over One of the most valuable functions of our visual sys- the desk),a shadow is cast on the scene.The camera acquires tem is informing us about the shape of the objects that a sequence of images /(,y,t)as the operator moves the stick surround us.Manipulation,recognition,and naviga- so that the shadow scans the entire scene.This constitutes the input data to the 3D reconstruction system.The variables tion are amongst the tasks that we can better accom- x and y are the pixel coordinates (also referred to as spatial plish by seeing shape.Ever-faster computers,progress coordinates),and t the time (or frame number).The three in computer graphics,and the widespread expansion dimensional shape of the scene is reconstructed using the spatial of the Internet have recently generated much inter- and temporal properties of the shadow boundary throughout est in systems that may be used for imaging both the the input sequence.The right-hand figure shows the necessary geometry and surface texture of object.The applica- equipment besides the camera:a desk lamp,a calibration grid and a pencil for calibration,and a stick for the shadow.One tions are numerous.Perhaps the most important ones could use the pencil instead of the stick are animation and entertainment,industrial design, archiving,virtual visits to museums and commercial on-line catalogues A number of passive cues have long been known In designing a system for recovering shape,differ- to contain information on 3D shape:stereoscopic ent engineering tradeoffs are proposed by each appli- disparity,texture,motion parallax,(de)focus,shad- cation.The main parameters to be considered are: ows,shading and specularities,occluding contours and cost,accuracy,ease of use and speed of acquisition.So other surface discontinuities amongst them.At the far,the commercial 3D scanners (e.g.the Cyberware current state of vision research stereoscopic dispar- scanner)have emphasized accuracy over the other pa- ity is the single passive cue that gives reasonable ac- rameters.These systems use motorized transport of curacy.Unfortunately it has two major drawbacks: the object,and active (laser,LCD projector)lighting (a)it requires two cameras thus increasing complexity of the scene,which makes them very accurate,but and cost,(b)it cannot be used on untextured surfaces expensive and bulky [1,15,16,12,2]. (which are common for industrially manufactured ob- An interesting challenge for computer vision re- jects). searchers is to take the opposite point of view:em- We propose a method for capturing 3D surfaces phasize cost and simplicity,perhaps sacrificing some that is based on 'weak structured lighting'.It yields amount of accuracy,and design 3D scanners that de- good accuracy and requires minimal equipment be- mand little more hardware than a PC and a video sides a computer and a camera:a pencil (two uses),a camera,by now almost standard equipment both in checkerboard and a desk-lamp-all readily available in offices and at home,by making better use of the data most homes;some intervention by a human operator, that is available in the images. acting as a low precision motor,is also required. 433D photography on your desk Jean-Yves Bouguett and Pietro Peronat$ t California Institute of Technology, 136-93, Pasadena, CA 91125, USA $ Universiti di Padova, Italy { bouguetj ,perona} @vision.caltech.edu Abstract A simple and inexpensive approach for extracting the three￾dimensional shape of objects is presented. It is based on ‘weak structured lighting’, it differs from other conventional struc￾tured lighting approaches in that it requires very little hard￾ware besides the camera: a desk-lamp, a pencil and a checker￾board. The camera faces the object, which is illuminated by the desk-lamp. The user moves a pencil in front of the light source casting a moving shadow on the object. The 3D shape of the object is extracted from the spatial and temporal location of the obsened shadow. Experimental results are presented on three different scenes demonstrating that the error in reconstructing the surface is less than 1%. 1 Introduction and Motivation One of the most valuable functions of our visual sys￾tem is informing us about the shape of the objects that surround us. Manipulation, recognition, and naviga￾tion are amongst the tasks that we can better accom￾plish by seeing shape. Ever-faster computers, progress in computer graphics, and the widespread expansion of the Internet have recently generated much inter￾est in systems that may be used for imaging both the geometry and surface texture of object. The applica￾tions are numerous. Perhaps the most important ones are animation and entertainment, industrial design, archiving, virtual visits to museums and commercial on-line catalogues. In designing a system for recovering shape, differ￾ent engineering tradeoffs are proposed by each appli￾cation. The main parameters to be considered are: cost, accuracy, ease of use and speed of acquisition. So far, the commercial 3D scanners (e.g. the Cyberware scanner) have emphasized accuracy over the other pa￾rameters. These systems use motorized transport of the object, and active (laser. LCD projector) lighting of the scene, which makes them very accurate, but expensive and bulky [l, 15, 16, 12, 21. An inteiesting challenge for computer vision re￾searchers is to take the opposite point of view: em￾phasize cost and simplicity, perhaps sacrificing some amount of accuracy, and design 3D scanners that de￾mand little more hardware than a PC and a video camera, by now almost standard equipment both in offices and at home, by making better use of the data that is available in the images. Figure 1: The general setup of the proposed method: The camera is facing the scene illuminated by a lialogen desk lamp (left). The scene consists of objects on a plane (the desk). When an operator freely moves a stick in front of thle lamp (over the desk), a shadow is cast on the scene. The camera acquires a sequence of images l(z,y,t) as the operator moves the stick so that the shadow scans the entire scene. This constitutes the input data to the 3D reconstruction system. The variables x and y are the pixel coordinates (also referred to as spatial coordinates), and t the time (or frame number). The three dimensional shape of the scene is reconstructed using the spatial and temporal properties of the shadow boundary thr xighout the input sequence. The right-hand figure shows the necessary equipment besides the camera: a desk lamp, a caliibration grid and a pencil for calibration, and a stick for the shadow. One could use the pencil instead of the stick. A number of passive cues have long b,een known to contain information on 3D shape: stereoscopic disparity, texture, motion parallax, (de)focus, shad￾ows, shading and specularities, occluding contours and other surface discontinuities amongst them. At the current state of vision research stereoscopic dispar￾ity is the single passive cue that gives reasonable ac￾curacy. Unfortunately it has two major drawbacks: (a) it requires two cameras thus increasing complexity and cost, (b) it cannot be used on untextured surfaces (which are common for industrially manufactured ob￾jects). We propose a method for capturing 311 surfaces that is based on ‘weak structured lighting’. It yields good accuracy and requires minimal equipment be￾sides a cowputer and a camera: a pencil (two uses), a checkerboard a& a desk-lamp - all readily available in most homes; somb intervention by a human operator, acting as a low precision motor, is also required. 43