PhotoCloud: interactive remote exploration of joint 2D-3D datasets

This paper introduces PhotoCloud, a realtime web-based system which allows end-users to interactively explore and visualize large mixed datasets composed of (up to) several thousands of calibrated 2D photographs joint together with a high-complexity 3D description of the scene. The system is not tailored to any specific data acquisition process, but aims at generality and flexibility. One major focus in our design is scalability in terms of geometric complexity and number of photographs, tackled by means of multiresolution and dynamic hierarchical representation of the data. The dataset is remotely stored and is accessed by the client through a cache system built over a multi-resolution 3D/2D structure. 2D and 3D data are presented to the user in an integrated seamless fashion. % allowing the adoption of either image-driven or 3D-driven navigation controls. %In the former case, an advanced image-browsing interface represents images in meaningful clusters, while the latter is adopted as the main image selection and presentation criteria. Visualization is completed with smooth transitions between 2D and 3D spaces, iconic representations of the images in the 3D space and, finally, by on the fly projections of the selected images on the 3D scene. Navigation of the image dataset is driven by either an advanced image-browsing interface, or a 3D model which is adopted as the main image selection and presentation criteria. Presentation of 2D images is implemented with smooth transitions between the two spaces, adopting clustering and visual presentation techniques to ensure readability of iconic visualization of the image set in the 3D space and, finally, by projecting on the fly the selected image on the 3D scene. As any system paper, the proposed one is an adaptation of techniques already present in literature, or known in the field. The main contribution of this paper consists in their concertation into a multi-resolution system, which effectively allows a high level of integration of generic 2D and 3D datasets. The paper also discusses several implementation choices and the resulting system is made available as an open source project. A preliminary user study conducted with differently skilled users confirm PhotoCloud main contributions in this visualization context.