We now test the system substituting the laser and the single-photon detector with a radio-frequency impulse RADAR transceiver to demonstrate the cross-modality of the proposed 3D imaging ap- proach. Figure 7.12 shows the experimental setup. The cross-modality of the proposed 3D imaging paradigm opens new routes to 3D imaging platforms using a wider range of sensors such as acous- tic or radio waves.
In this case the RADAR chip (Novelda XeThruX4) is a single transmitter and receiver channel operating at 7:29 GHz frequency with a bandwidth of 1.4 GHz and pulse duration of 670 ps with a sampling rate of 23x109samples/s. The radar chip emits RADAR pulses towards the investigated scene and collects the return signal in a temporal histogram form, while the conventional ToF 3D camera synchronously acquires the corresponding 3D image. We then train the NN using the same NN model discussed in Section 6.5. The RADAR return signal is the input of the NN, while the 3D is the output of the NN. We acquire 6500 temporal histogram-3D image pairs of which 5580, 420 and 500 examples are respectively used in the training, validating and testing process. Once the ANN has been trained using pairs of the return signal and of the corresponding 3D image, we test the retrieval algorithm using only the temporal return data. The return signal is collected in a temporal histogram of 91 time bins of 293 ps each. As in the optical equivalent, the ToF camera is
acquiring 64x64 pixels images of the scene by a colour-encoded depth map. The data are normal- ized and randomized to guarantee the generality of the predicted solution and to compare features covering a wide range of values at different scale.
The RADAR transceiver has a frequency bandwidth of 1.4 GHz and a pulse duration of 670 ps measured as the temporal IRF of the RADAR emission. The IRF (Fig. 7.13) has been measured as the FWHM of the temporal return signal of a centimetre-dimension target. According to the formula in Eq. (7.2), the temporal resolution of the chip induces a spatial resolution of 20 cm and 90 cm at 2 metres of distance along the depth and the transverse spatial dimension. Since the system has a lower temporal resolution than the optical equivalent, the quality of the 3D retrieval is poorer than the previous optical counterpart.
Figure 7.14 shows the obtained experimental 3D images predicted from the NN using the return RADAR signal. In this case the scene to be recovered is composed by an isolated person moving back and forward within the investigated scene. The first column represents the temporal trace of the return RADAR signal produced by the person moving in the scene for five investigated ex- amples ((a)-(e)). The second column is the 3D image predicted by the learning algorithm from the temporal trace in the first column. The third column represents the 3D image ground truth to evaluate the performance of the 3D retrieval. Since the experimental setup is affected by the spatial ambiguities produced by the transit-time symmetry of the system, in this case the target is moving only along the z axis dimension of a 3x3x4 m3scene. A full video of the 3D retrieval is available in the Appendix E.
Although the quality of the 3D retrieval is lower than the optical equivalent, the suggested ap- proach is still able to recover the 3D location of the target along the depth and the transverse dimensions. Using a data-driven approach, the proposed method represents a compact 3D imaging paradigm from a temporal trace and an optical trained ANN algorithm, transforming an ubiquitous RADAR sensor into a 3D imaging device.
The suggested approach is therefore able to recover the 3D image in cross-modality beyond the op- tical domain. However, the suggested approach is affected by some limitations such as the spatial degeneracy of the temporal symmetrical 3D scene as discussed in the next section.
Time of Flight 3D camera
RADAR
3 cm
Figure 7.12: Cross-modality 3D imaging experimental setup by a RADAR chip. We test the paradigm using a RADAR transmitter-receiver chip and a conventional 3D camera to demonstrate the cross-modality 3D imaging of the proposed method. The RADAR chip emits RADAR pulses towards the scene and collects the return RADAR signal. The 3D camera simultaneously acquires the corresponding 3D image of the scene. Synchronizing the chip and the ToF camera acquisition, we then apply the same learning approach used in the optical equivalent scenario. The entire system is mounted on a 21x9 cm2breadboard.
time (ns) counts (a.u.) 670 ps 0 6 12 18 24 32 1 2 3 4 10-3
Figure 7.13: Temporal IRF of the RADAR transceiver. The temporal resolution of the RADAR transceiver is measured as the FWHM of the temporal signal scattered from a centimetre dimension target.
Counts (
a.u.)
Depth (m)
4 m
0 m Temporal histogram Ground truth
ToF camera Retrieval 0 0.2 0.4 0.6 0.8 1 Counts ( a.u.) Depth (m) 4 m 0 m 0.2 0.4 0.6 0.8 1 Counts ( a.u.) Depth (m) 4 m 0 m 0.2 0.4 0.6 0.8 1 Counts ( a.u.) Depth (m) 4 m 0 m 0.2 0.4 0.6 0.8 1 Counts ( a.u.) Depth (m) 4 m 0 m 0.2 0.4 0.6 0.8 1 0 0 0 0 (a) (b) (c) (d) (e) 1m 1m 1m 1m 1m
Figure 7.14: Cross-modality optical 3D imaging with Radar. Each row ((a)-(e)) represents an input-predicted output example of a person moving back and forward within a 3x3x4 m3 scene. The first column represents the return RADAR signal from the entire scene and acquired by the RADAR transceiver at 7.29 GHz radiation. The second column is the 3D retrieval predicted by the learning algorithm from the RADAR return in the first column. The third column represents the ground truth of the scene acquired with a standard ToF camera. Due to the wide pulsed duration of the RADAR emitter of 670 ps, the 3D retrieval of the scene has a poorer reconstruction quality than the optical equivalent.