Image-based modeling for object and human reconstruction

Abstract

The topic of this research is the investigation ofthe image-based approach for the 3D modeling of close-range scenes, static objects and moving human characters. Three-dimensional (3D) modeling from images is a great topic of investigation in the research community, even if range sensors are becoming more and more a commonsource and a good alternative for the generation of 3D information. The interest in 3D modeling is motivated by a wide spectrum of applications, such as video games, animation, navigation of autonomous vehicles, object recognition, surveillance and visualization. In particular, the production of 3D models from existing images or old movies would allow the generation of new scenes involving objects or human characters who maybe unavailable for other modeling techniques. Techniques for 3D modeling have been rapidly advancing over the past few years although most of them focus on single objeets or specific applications such as architecture or city mapping. Nowadays the accurate and fully automated reconstruction of 3D models from image data is still a challenging problem. Most of the current approaches developed to recover accurate 3D models are based on semi-automatic procedures, therefore the introduction of new reliable and automated algorithmsis one of the key goals in the photogrammetric and vision communities. In fact fully automated image-based approaches generally do not work under certain image network configurationor are not reliable enough for some applications, like cultural heritage documentation. Automated image-based methods require good features in multiple images and very short baselines between consecutive frames to extract dense depth maps and complete 3D models. But these requirements are not satisfied in some practical situations, due to occlusions, illumination changesand lack of texture. Automated processes often end up with areas containing too many features that are not all needed for the object modeling and areas with very few features to produce a complete and detailed model. Automated reconstructio nmethods generally do not report good accuracy, limiting their use for applications that require only nice-looking 3D models. Furthermore, post processing operations are often required, which means that the user interaction is still needed. Therefore fully automated procedures are generally limited in finding point correspondences and camera poses while for the surface measurement phase the user interaction is generally preferred, in particular for architectures, The image-based modeling of an object should be meant as the complete process that starts from the equisition system and ends with a virtual model in three dimensions visible interactively on a computer. The photogrammetric modeling pipeline consists of few well known steps: calibration and orientation, surface measurement and pointcloud generation, structuring and modeling of the objeet geometry, visualization and analysis. Different efforts have been done to increase the level of automation within these steps and broaden the use of the image-based modeling technology. So far, however, the efforts to completely automate the processing,from the image acquisition to the ou put of a 3D model, are not always successful or not applicable in many 3D modeling projects. In this dissertation different techniques developed to analyze existing sequence of images and partially automate the process of constructing digital 3D models of static objeets or moving human characters are reported. In particular the work investigates if automated and rnarkerless sensororientation is feasible and under which conditions, if it is possibleto recover complete and detailed 3D models of complex objects using automated measurement procedures, which kind of (3D) informationcan be retrieved from existing image data as well as the capabilities or limits of photogrammetric algorithms in dealing with uncalibrated images and zooming effects. For the investigations, sets of available or self-acquired images, as well as frames digitized from existing monocular Videos are used. The possibility to automatically Orient an image sequence heavily dependson the type of images, acquisition and scene. Compared to other research approaches, the developed method for the automated tie point extraction and image orientation relies on aecurate feature location achieved using least Squares matching measurement algorithm and a Statistical analysis of the matched and adjustment results. The reported examples demonstrate its capabilities also for the orientation of images acquired under a wide baseline. A photogrammetric bündle adjustment is always employed to recover the camera parameters and the 3D objeet coordinates. On the otherhand, the analysis of moving human characters using a monocular video is based on a deterministic approach together with constraints and assumptions on the imagcd scene as well as on the human's shape and movement. The developed photogrammetric pipeline can accommodate different input data and different types of human motions. The resulted 3D characters and scene information can be used for visualization or animation purposes or in biometric applications with medium accuracy requirements. For the automated tie point extraction phase, programs for the feature extraction and the relative orientation between image pairs and triplets were implemented,together with a graphical tool to display the recovered correspondenecs and epipolar geometry. Concerning the human reconstruction from monocular videos, programs were developed to recover 3D models from single images and combine them under the same reference system in case of image sequence analysis.