Análisis de Requerimiento

Ho Chi Minh City University of Technology

Ho Chi Minh City, Vietnam [email protected]

Abstract  — Three-dimensional perception has gained more and more importance in robotics as well as other fields for the great improvement in effectiveness recently. In this paper we present a simple but efficient system to calculate the 3D data from the 2D images to construct the object’s surface as a point set. This point set is created by a method that combines several laser scans captured from a single camera. Our work employed the Moving Least Squares (MLS) derived from the Point Cloud Library (PCL) as the underlying representation of an object’s surface.

The Greedy Projection and Poisson algorithms were also used to create a triangulation where the triangle’s edges  respect the geometrical relations of the surface instead of the sampling density of the range scans. This application can be integrated to many fields such as hybrid rendering, physical parameters, laser scan registration and object recognition.

Keywords  — surface structure; surface texture; reconstruction algorithms

I. I NTRODUCTION

Surface reconstruction is one of the most pivotal trends in computer vision and computer graphics. Our main work is to reconstruct a highly accurate model of the object from an unorganized set of points that best fits the model. As one of the most commonly accepted graphic representation with many supporting libraries, polygonal meshes are used to represent the object‟s three-dimensional model after scanning and reconstruction process. To assemble the laser scans into a 3D model, the problem of calculating the point set ha s to deal with the unknown geometrical parameters of the model and the interference that affects our measurements. The unavoidable  presence of noise and outliers is the main challenge and once

these factors are well handled, the reconstruction progress can  produce great results.

Many processes in computer graphics such as object identification, surface reconstruction, separation plane, 3D model comparison, require information about the direction of reference points in order to orient the surface in three-dimensional space. The sensor scans the surface of 3D objects and extracts discrete points which contain an amount of noise.

During the sampling process, information about the direction and curvature of the surface can lost for some extent. The  process of estimating the orthogonal vector (normal estimation)

will restore these values for each sampled point using its neighbors. There are two methods to estimate the orthogonal vector: the averaging method and optimization-based method.

We also used an implicit technique to demonstrate the 3D surface to help reduce noise in the later process of calculating and estimating polygons. Implicit technique is a notion to construct the surface from mesh‟s n odes. A discrete surface is

defined by a finite number of parameters. We only consider here polygon meshes. Polygon meshes are composed of geometrical and topological connections. Clearly spoken, the geometry of a connection includes vertex coordinates, normal vectors while its topology is represented in various ways. The Poisson and Greedy projection algorithms are employed to approximate the mesh from points with normal information without the need for spatial partitioning heuristics and  blending by using a hierarchy of locally supported basis

functions. This reduces the problem to a sparse linear system which can be solved efficiently. After the triangular mesh is formed, the surface can be colored using the data of the object‟s images from camera. This coloring process is often inaccurate for the points on edge because of incorrect calibration and light interference around the object.

There are many kinds of sensor that can produce the three-dimensional data as Kinect, LIDAR, and stereo camera. That yields a solution to increase the accuracy of point cloud by fusing these data from different sensors. In our work, a single camera and laser scanner were used for scanning and capturing  process to decrease environment‟s noise. The camera will capture a set of images and convert this into two-dimensional coordinates. The Point Cloud Library [1] (PCL) is a standalone, large scale, open project for 2D/3D image and  point cloud processing. It implements a set of algorithms

designed to help manipulating three-dimensional data, in  particular point clouds. The backbone for fast k-nearest neighbor search operations is provided by FLANN [2] (Fast Library for Approximate Nearest Neighbors). All the modules and algorithms in PCL pass data around using Boost shared  pointers, thus avoiding the n eed to re-copy data that is already  present in the system. Laser scanner is one of the simplest ways to reconstruct an object‟s three-dimensional surface with high accuracy and efficiency.

II. THEMATHEMATICS OF TRIANGULATION

This part describes the underlying mathematics to add a new dimension to the 2D images using the model‟s calibration  parameter. The angle between the camera and laser beam is the main parameter for triangulation geometric model [3]. Pinhole model is a simple and popular geometric model for a camera.

In this model, every 3D point determines a unique line passing through the center of projection. The intersection of a plane of light with the scanned object contains many curved segments.

We assume that the locations of camera and laser are available  by calibration process. Thus, the equation of projected planes

and rays, as well as the equations of camera rays corresponding to the center points are created by the hardware parameter, which can be measured. Under this assumption, the equations

of projected planes and rays, as well as the equations of camera rays corresponding to reference points, are defined by  parameters which can be measured. From these measurements, the location of reference points can be recovered by intersecting the planes or rays of light with the camera rays corresponding to the reference points.



Equation (1) is used to convert the 2D image to 3D point cloud. From a two-dimensional coordinates of the laser‟s point

_ , (1) calculates the 3D coordinates P3D(x,y,z) based on the identified parameter. Fig. 2 shows the triangulation geometry of equations (1). ^C  x  and ^H o  are the coordinates of the point   of the ^Ox and ^Oz   axes. We can also combine the  pinhole camera model to identify the projected points of the object on the image plane. Base on this model, denote _,

_ to be the projections of_ on the plane. Equation (1) will use these values of _, _ and the other  parameters such as the distance to the camera, the laser scan and reference point to calculate the coordinates of

   The object will be rotated for 1.8 degree after determining the coordinates of the points on the laser scan.

P3D

Fig 1. First orthogonal projection plane Oxy. The parameters L1, L2 are the distances between the camera, the laser device and the origin point on Oxy plane.

CAMERA

Fig 2. Second orthogonal projection plane Oxz Geometryical triangulation parameters looking sideway. The parameters L3

is the distances between the camera and th e origin point on Oz axis. cloud. The image processing algorithm uses the OpenCv APIs to extract the set of cordinates P2D(i,j) of the points on the laser scan. After completing this conversion, the point cloud will become input for pre-processing process.

Fig 4.Actual scanning chamber with the sensors positioned inside.

III. POINTCLOUDPRE-PROCESSING

After conversion, the raw point cloud data will contain several thousands of noisy points. This brings difficulties for the reconstruction algorithm. First, point cloud preprocessing [3] method will remove noise, down-sample and reorganize the point cloud to reduce processing time and increase the [4]. This is a method to divide and reorganize the point cloud data in k-dimensional space. Using this technique the time for searching and processing the point cloud will be significantly reduced. For each point in the point cloud, a k-neighborhood is selected. This neighborhood is created by searching the nearest k-neighbors of the reference point within the sphere of a defined radius. A Kd-tree is similar to a decision tree except that a branch is created from the parent by the median value along the dimension with the highest variance. Each node in the tree is defined by a plane through one of the dimensions that partitions the set of poin ts into left/right (or up/down) sets, each contains half of the points of the parent node. These children are again partitioned into equal halves, using planes through a different dimension. The partitioning process will stop after log(n) levels, with each point being in its own leaf cell. Kd-trees are known to work well in low dimensions but seem to fail as the number of dimensions increase beyond three. The algorithm below describes the intuitional steps for arranging the point cloud into Kd-tree.

Algorithm 1 Nearest Neighbors Searching

1/ Locate the location where the point would be located if it were added to the tree :Starting with the root node, the algorithm moves down the tree recursively, taking decisions to follow left or right node depending on whether the point is less than or greater than the current node in the split dimension.

2/ Once the algorithm reaches a leaf node, it saves that node  point. As the tree is traversed the distance between the point

and the current node should be recorded.

3/ The algorithm goes back up the tree evaluating each branch of the tree that could have points within the current minimum distance.

4/ Search completes when the algorithm reaches the root node.

Moving Least Square  (MLS) [5] filter is a method to remove noise and smooth out the point cloud based on estimating and comparing the normal vectors. These filters will access each point in three-dimensional data and create a

simulated plane, which contains k-neighbor points defined by Principal Component Analysis (PCA) [6]. An advantage of this approach is that it works on general, unorganized point clouds, so moving least squares (MLS) smoothing can be applied as a  post-processing step on the global point cloud (concatenation

of registered point clouds) in order to smooth out any remaining noise or registration imperfections. Moreover, MLS  provides a set of completely tools to solve problems about

smoothing and filling the holes caused by failed scanning  process.

Fig 6. The result after using MLS filter on the point cloud.

Optimization-based  method estimates the normal vector  by optimizing the min value in (2) using PlanePCA [7]

technique. In (2), J is the optimal value function, pi  is the reference point and ni  is the normal vector. PlanePCA will minimize the deviation of the points around an estimated axis and find out the axis with the smallest deviation. In computational terms the principal components are found by calculating the eigenvectors and eigenvalues of the covariance matrix. This process is equivalent to fin ding the axis system in which the co-variance matrix is diagonal. The eigenvector with the largest eigenvalue is the direction of greatest variation, the one with the second largest eigenvalue is the (orthogonal) direction with the next highest variation and so on. Each Eigen value [8] corresponds to an Eigen vector. For this project the maximum Eigen value is being used to find its corresponding Eigen vector. This is a column vector which stores the x, y, z coordinates of the normal position.

i

i i i n

min J(p ,Q ,n ) ^

IV. SURFACER ECONSTRUCTIONPROCESS

Greedy Projection algorithm [9] is based on surface growing rules by maintaining a list of points from which the mesh can be grown and extend until all possible points are

connected. The algorithm‟s input is the point cloud with  the sets of normal vectors.

Algorithm 2 Greedy Projection Algorithm

1/ Nearest neighbor search: For each point „p‟ in the point cloud, a k-neighborhood is selected. This neighborhood is created by searching the reference point‟s nearest k -neighbors within a sphere of radius r.

2/ Neighborhood projection using tangent planes: The neighborhood is projected on a plane that is approximately tangential to the surface formed by the neighborhood and ordered around p.

3/ Pruning: The points are pruned by visibility and distance criterion, and connected to p and to consecutive points by edges, forming triangles that have a maximum angl e criterion and an optional minimum angle criterion.

Fig 7. Object‟s surface using Greedy Projection algorithm Marching Cube algorithm [10] is a popular algorithm in the field of surface reconstruction. This algorithm assumes that each point is a vertex of the cube and creates polygons to connect points in each cube. Each edge and vertex of the cube is weighted to create a cube‟s table.  After determining the edges or polygon vertex that can intersect, the algorithm connects cubes and forms the surface of the primitive objects.

Poisson algorithm creates a stable surface for object using implicit functions to approximate the surface and split them into smaller surface using Marching Cube algorithm.

Fig 8. Object‟s surface using Poisson algorithm V. CONCLUSIONS

In this paper we have presented a productive method for object surface reconstruction using a single camera and laser scanner. We have successfully applied the mathematical triangulation model to convert 2D data to 3D and reconstruct the object‟s surface. As low-cost camera and laser enter the market, we expect students and hobbyists to begin incorporating them into their own 3D scanning systems. Such laser-camera systems have already received a great deal of attention in recent academic publications. Finally, some ideas that will guide our future work: We increase the laser‟s power and the camera‟s speed [11] to improve accuracy of the point cloud.

NOISE

Fig 9. Removing noise using MLS algorithm.

Fig 10. Above : The result of Greedy Projection algorithm.

Below: The result of Poisson algorithm. The Poisson algorithm has a better result than Greedy Projection algorithm but take a lot of time: approximate 25 minute with 80000 points.

ACKNOWLEDGMENT

We would like to send many thanks to the Club for Science Research of Faculty of Electrical and Electronics Engineering for their tremendous contribution and support  both morally and financially towards the completion of this  paper.

R ^EFERENCES

[1] Alexandru, E. Ichim, “PCL Toyota Code Sprint 2.0”, 2013.

[2] J. Wang, “Suface Reconstruction from Imperfect Point Models”, 2007.

[3] Alexandru, E. Ichim, “RGB-D Handheld Mapping and Modeling”, 2013.

[4]  N.B.F. Fernandez, “3D Registration for Verification of Humanoid Justin‟s Upper Body Kinectics”, 2012.

[5] Y. Li, “Low Cost 3D Scanner Background segmentation and Visual hull obtaining”, 2012.

[6] F.S.I. Paniagua, “Object Recognition using Kinect”, 2011.

[7] D. Lanman, G. Taubin, “Build Your Own 3D Scanner 3D Photography for Beginners”, 2009, pp.65-68.

[8] A. Wetzler, “Low Cost 3D Laser Scanning Unit with Application to Face Recognition”, 2005.

[9] J. Hyvarinen, “Surface Reconstruction of Point Clouds Captured with Microsoft Kinect”, 2012.

[10] M. Bertshe, “Recurring Adaptive Segmentation and Object Recognition”, 2012.

[11] F. Engelmann, “FabScan Affordable 3D Laser Scanning of Physical Object”, 2011.