The choice of camera lens can impact upon either spatial and visual quality or tem- poral quality. Wide angle lenses lead to image distortion, which left uncorrected will result in spatial and visual quality of the reconstructed output suffering. Cor- rection of the image distortion takes processing time, and will therefore add to the latency and consequently the temporal quality of a 3D reconstruction system. Camera trigger and distribution method can both impact upon spatial, visual and temporal quality in a 3D reconstruction system. Using a hardware trigger will en- sure that images from cameras are perfectly synchronised, resulting in temporally consistent inputs. A software trigger system cannot guarantee such temporal con- sistency on inputs, and can lead to spatial and visual quality degradation. Camera distribution method can also lead to temporal inconsistency on inputs in the case where frame transmission is delayed, or the order is not guaranteed by the trans- mission protocol.
Camera placement impacts on the spatial quality of the visual hull, and moving objects can make careful camera placement a pointless exercise. Considering con- tributions to the visual hull in terms of their visibility within a subset of cameras can greatly alleviate the impact of camera placement on spatial quality. Objects partially visible in a particular camera will no longer be clipped at the camera edge; objects invisible from a particular camera can still be reconstructed by the cameras from which the objects are visible.
Camera calibration accuracy determines the spatial accuracy of the system, and in turn impacts upon spatial and visual quality. Cameras with high calibration error will result in inaccurate projection of the silhouette contour and texture pixels
from the camera image plane into three dimensions. This leads to errors in both form constraint and texturing.
Background segmentation is critical for high spatial quality in shape-from-silhouette based 3D reconstruction. Poor segmentation from a variety of common causes can result in reconstruction errors in the form of erroneous objects, holes through the reconstructed form, eroded or expanded edge of objects. Lighting and shadows are particularly difficult to control in terms of their effect on background segmen- tation.
Camera exposure and colour balance impact upon visual quality. The camera set needs to be carefully adjusted for even colour and exposure balancing across the set. Poor balancing leads to stripy texturing of the reconstruction, adversely affecting visual quality. Real-time correction of exposure and colour balance takes processing time and can therefore lead to temporal quality degradation.
4.6.3
Temporal consistency of inputs
Use of unsynchronised cameras in a 3D reconstruction system is an attractive option because it allows for a much wider choice of cameras. Those featuring a hardware synchronisation input are few and far between, and tend to exist only at the high end of the market. The lack of hardware trigger leaves two options at the camera acquisition stage: use of a software trigger, use of free running cameras. A software trigger takes the form of a signal originating from a single computer requesting that a new image be acquired from all cameras. This signal is likely to be received or processed by individual cameras at slightly different times, and hence gives rise to a temporal inconsistency between frames from each camera. Free running cameras are completely unsynchronised, and therefore the temporal offset between frame starts for each camera can be anywhere up to a whole frame period apart.
CHAPTER 4. QUALITY OF 3D RECONSTRUCTION 109 inputs is important for 3D reconstruction of moving humans. In the case of a free running system, or a system in which the distribution method can result in frames arriving a whole frame period late, progressive degradation in spatial and visual quality of moving humans was demonstrated as more cameras became unsynchro- nised. A software triggered system was shown to exhibit relatively small temporal frame offsets compared to those possible from a free running system. However, it is clear that spatial quality degrades proportionally to the speed of movement. Therefore in the context of a 3D telepresence system, frame offsets would need to be guaranteed to be small enough for human movement within that time period to be minuscule in terms of their pixel representation.
Improving performance of VBR
This chapter discusses temporal quality of 3D reconstruction, specifically the per- formance of the algorithm and approaches to improving it. In the previous chap- ter the necessity to balance visual, spatial and temporal qualities was discussed, and it was determined that increasing camera count or resolution could increase spatial and visual quality, but at the cost of temporal quality. Consequently, by reversing the logic of this statement - Improving the performance of the 3D recon- struction algorithm will enable higher camera counts or resolutions to be used, leading to higher spatial, visual and temporal qualities in real-time. The focus of this chapter is therefore on improving the performance of the 3D reconstruction algorithm. A parallelisation strategy for the EPVH algorithm that is adapted for execution on modern multi-core computation hardware such as GPUs and CPUs, which could remove the requirement for network distributed processing to achieve interactive frame rates. During the course of this parallelisation a number of op- timisations that further increase the algorithm’s performance are presented. The performance of our implementation is compared to the published performance of EPVH, and also between execution on a GPU and CPU.
The work presented in this chapter has been published in [34].
CHAPTER 5. IMPROVING PERFORMANCE OF VBR 111