This section is supposed to represent a brief overview on calibration approaches that are characterized by some specializations, which usually require certain setups. The described examples are systems, where the camera needs to be in motion relative to the refractive interface, non-submerged systems or approaches that depend on special calibration devices.
Camera Motion Relative to the Refractive Interface. The focus of this thesis is on optical imaging systems that are characterized by flat refractive interfaces and cameras, which form a fixed unit. On the contrary, there are also systems that do not form fixed units and which are characterized by cameras in motion relative to a refractive interface. An exemplary calibration approach for such a system is proposed by Chang and Chen [CC11]. The authors use a single camera moving around a scene with the water surface in an aquarium being the refractive interface. The camera is equipped with an inertial measurement unit (IMU) to provide necessary information, which is incorporated within a structure from motion framework.
Another exemplary approach is proposed by Mulsow et al. [Mul10; MM14]. The authors make use of calibration objects, which are fixed to the refractive interfaces. They experiment with a flat refractive interface [Mul10] and a more complex one [MM14]. The known calibration objects are fixed to the interface between water and glass as well as to the interface between air and glass. Multiple images of the system need to be taken by moving the camera in front of it.
Calibration of Non-Submerged Systems. Chen et al. [Che+11; Che+12] propose an approach to recover the orientation of a flat refractive interface between a camera and an object in air. Therefore, the scene is captured twice, once with and once without the flat refractive interface. Since the setup contains only an air-glass-air transition, there is just a depth-independent shift due to the refraction in glass. Therefore, the distance to the interface can not be estimated.
Closely related is the calibration approach for so-called reflection stereo systems of Shimizu and Okutomi [SO08]. In such a system, a single camera is imaging through a transparent flat refractive medium (air-medium-air transition). The authors perform experiments with a double-sided half-mirror plate with a 75% reflection ratio to generate the needed, so-called complex images, which contain stereo information from multiple reflections.
Huang et al. [HL14] develop an approach for the calibration of the intrinsic camera parameters in the case of an existing perpendicular flat refractive interface, but not for the refractive parameters.
3.3 Explicit Modeling of Refractive Effects Special Calibration Devices. Narasimhan and Nayar [Nar+05; NN05] propose a calibration approach in the context of light stripe range scanning. Therefore, they need a camera and a DLP projector outside a water filled aquarium and two known planes with a number of a priori measured 3D points on them placed vertically and at known distances in the aquarium. Their approach is based on the raxel model of [GN05], which means that a pixel on the image is connected to its corresponding ray in water. Therefore, no explicit calibration of refractive parameters is performed. The projector is used to sweep a light stripe over the planes to connect the image points with the 3D points. The remaining points have to be interpolated.
Yau et al. [YGY13] extend the work of Agrawal et al. [Agr+12] by considering the dispersion of light. Therefore, they use a special calibration device for the calibration of a
FRS with an arbitrary number of layers. The device is described as a watertight acrylic
enclosure that contains a number of LEDs. The LEDs are arranged in a grid pattern with known coordinates. Point light sources with a measurable amount of dispersion are generated by one half of a LED emitting light of blue color and one half emitting red color simultaneously. The so-called dispersion constraint is used for estimation of the system axis. For the estimation of the layer thicknesses, the so-called wavelength triangulation constraint is defined. The estimated parameters are refined by non-linear optimization by minimizing the reprojection error. For the approach to work, the refractive indices of the participating media have to be known, the intrinsic parameters of the camera have to be calibrated, the images have to be corrected for chromatic aberrations and the image noise must be reduced in a pre-processing step.
Motivated by the approach of Yau et al. [YGY13], Chen and Yang [CY17] propose an approach for the calibration of a FRS , which makes use of triple wavelength dispersion. They determine the system axis in the same way as Yau et al. [YGY13]. Their major contribution is the derivation and proof of a solution for the determination of the air layer thickness that can be applied after the system axis is determined. They do not need any calibration objects. However, to generate the required triple wavelength dispersion, they need a device to illuminate the scene consecutively with red, green and blue light.
Findings
All of the described approaches require special setups. Cameras that are not fixed to the refractive interface can be seen as a special form of SFRS . Equipping the camera with an IMU, however, depends on an actual movement to gather the required information. Approaches that require to fix calibration objects to the refractive interface are a common alternative for the calibration of refractive systems, as will be seen in the following sections. However, this is not always realizable. Non-submerged systems could be used to calibrate a refractive system before underwater deployment, but the presented approaches are too restricted by their necessary setups. The utilization of special calibration devices represents a similarly severe restriction.
3 Related Works
Figure 3.3: Schematic representation: Calibration object (CO) fixed to the refractive interface
Φ1. A transformation (R, t) links the CO and the FRS .