Point Cloud Merging for Complete 3 D Surface Reconstruction

Extracting a point cloud of a real object from 3D surface measurement is a common task in various applications of computer vision, computer graphics, reverse engineering and etc. Usually, a single view is not enough to describe the whole object and multiple views of the surface are necessary. Typically the views are obtained from multiple 3D scanners or from a single scanner positioned at different locations and orientation. Each view is represented as partially overlapping dense point cloud. A problem of surface registration arises, i.e. the different views have to be located in a same coordinate system. In other words, the objective is to determine a rigid transformation for each partial view, in order to find the optimal alignment among all of them. In real cases, the point-to-point correspondences for the overlapping regions are calculated. When only two-views are present, we are required to find the transformation that minimizes the distance between the two clouds. Two views registration is a well studied problem in the literature. The Iterative Closest Point (ICP) algorithm is the most common solution [1]. It iteratively revises the transformation, usually composed of scale, rotation and translation, needed to minimize the distance between the points of the two clouds. The point correspondences are extracted considering the closest point in the other view. In most cases ICP and other (i.e. [2]) algorithms require specifying three or four corresponding points on each of the two views. To obtain a quality results corresponding points has to be selected with high accuracy, this is the main disadvantage of such kind of algorithms. An automatic registration method was proposed in [3]. The algorithm works well on partially overlapping point clouds, but it fails in most of the cases when the point clouds of different views are similar (i.e. two views of a cylinder parts). In this paper we present an automatic method of point clouds merging for complete surface reconstruction. This technique is applied during scanning procedure, as an output we get pre-aligned views of the point cloud. Afterwards fine registration is applied. Later results will show that this technique works great as on largely overlapping point clouds as on partly overlapping point clouds. Our method does not have any special requirements for point cloud acquisition hardware; stereo vision system [4, 5] can be used instead of 3d laser scanner.


Introduction
Extracting a point cloud of a real object from 3D surface measurement is a common task in various applications of computer vision, computer graphics, reverse engineering and etc. Usually, a single view is not enough to describe the whole object and multiple views of the surface are necessary.Typically the views are obtained from multiple 3D scanners or from a single scanner positioned at different locations and orientation.Each view is represented as partially overlapping dense point cloud.A problem of surface registration arises, i.e. the different views have to be located in a same coordinate system.In other words, the objective is to determine a rigid transformation for each partial view, in order to find the optimal alignment among all of them.In real cases, the point-to-point correspondences for the overlapping regions are calculated.
When only two-views are present, we are required to find the transformation that minimizes the distance between the two clouds.Two views registration is a well studied problem in the literature.The Iterative Closest Point (ICP) algorithm is the most common solution [1].It iteratively revises the transformation, usually composed of scale, rotation and translation, needed to minimize the distance between the points of the two clouds.The point correspondences are extracted considering the closest point in the other view.
In most cases ICP and other (i.e.[2]) algorithms require specifying three or four corresponding points on each of the two views.To obtain a quality results corresponding points has to be selected with high accuracy, this is the main disadvantage of such kind of algorithms.
An automatic registration method was proposed in [3].The algorithm works well on partially overlapping point clouds, but it fails in most of the cases when the point clouds of different views are similar (i.e. two views of a cylinder parts).
In this paper we present an automatic method of point clouds merging for complete surface reconstruction.This technique is applied during scanning procedure, as an output we get pre-aligned views of the point cloud.Afterwards fine registration is applied.Later results will show that this technique works great as on largely overlapping point clouds as on partly overlapping point clouds.
Our method does not have any special requirements for point cloud acquisition hardware; stereo vision system [4,5] can be used instead of 3d laser scanner.

3D laser scanning system
In point cloud acquisition process we used our own implementation of 3d scanning system presented in [6], however some minor modifications had to be made in order to meet our requirements.
Basic idea of the 3d scanning system (Fig. 1.) is quite simple.Laser ray is expanded to a line by cylindrical lens.If we make an assumption that laser source is a single point in 3D space, then we can state, that this point and laser line projected on some priori known reference geometry forms a laser plane.Point cloud of the 3D object is computed by calculating intersection of the laser plane with a boundary of the object.
The main problem is that we do not hold any priori information neither on a location of the laser plane, neither on a location of the object or camera.
Relationship between camera and laser plane can be obtained during calibration process.
First step in whole system calibration is to calculate intrinsic parameters of the camera.Camera calibration is Fig. 1.Scanning system and its main components, like laser source, world (reference) coordinate system -X w , camera coordinate system -X c , and relationship between them -R, T http://dx.doi.org/10.5755/j01.eee.113.7.616 done using the ideal pinhole camera model [7] and some calibration pattern which geometry is known in advanced, for example chessboard pattern.
Intrinsic parameters of the camera depend only on optics of the camera and the camera itself.Once intrinsic parameters are calculated they do not change unless, focal length or aspect ratio is changed.
Next step is to determine cameras extrinsic parameters.Extrinsic parameters are rotation matrix (R) and translation vector (T).Extrinsic parameters define relationship between camera coordinate system and world (reference) coordinate system here X c -point coordinates in camera's coordinate system, X w -point coordinates in world (reference) coordinate system, R and T -rotation and translation between camera's and world's coordinate systems.
Computing matrix R and vector T, two calibration planes (patterns) aligned to form a right angle were used (Fig. 1).
Knowing cameras intrinsic and extrinsic parameters, point cloud can be calculated by finding intersection between laser plane and camera ray (Eq.2).Such intersection gives us 3D coordinates of the object points in world coordinate system   here P w -object's point coordinates in world coordinate system, c  -camera ray, laser  -laser plane and t notes that position of laser plane changes in time.
Camera ray is defined as a ray from origin of the camera's coordinate system passing thru point of laser line projection in image plain (Eq.3).Laser line's structure in image is detected with sub-pixel accuracy using shifted Gaussian kernels approach presented in [8] v here c  -camera ray, p 0 -camera's center, coordinates, v  -direction vector of the ray,  -positive scale factor.
Laser plain (Fig. 2.) can be calculated from its projection on image plane.Laser plane is defines by one of its points p 0 and a normal vector.Any point p belongs to the laser plane if and only if vectors   here, laser  -laser plane, n  -normal vector of the plane, p 0 -reference point of the plane, p -any other point on the plane.
To find the point which belongs to the laser plane we calculate an intersection of two laser lines projected on known geometry of the background (the same which was used in cameras extrinsic calibration): and

  
here P(t) -intersection point of two lines, n(t) -normal vector, -laser lines projected on right and left reference planes.
Equations of lines can be found applying Hough transform for line detection and taking two points lying on each of the lines.At this point lines might not intersect at all (Fig. 3.), so we are trying to define approximate intersection as the point p which minimizes the sum of the square distances to both lines Main disadvantage of describes scanning system is that we cannot get the point cloud for complete surface representation.Instead we get multiply point clouds of the same object in world coordinate system with different angles of view which are not priori known.Merging such point clouds in one unique representation is a challenging task.

Point cloud merging using marker patterns
Most of the point cloud or 3D surface merging and registration algorithms require manually specify 3 or 4 corresponding points on each part of the different point cloud representing the same object.After that in most cases iterative closest points algorithm or its modifications are applied to calculate final point cloud representation.
This approach works fine if point clouds overlap largely.In our case point clouds should overlap only at their boundary regions.
First step in point cloud merging process is to find initial rotation and translation between different point cloud views of the same object.Our proposed method could handle scaling as well, but in current implementation this is left to fine registration step.
The solution used in our method is close to one used  3.The case when two lines are not intersecting.Approximate intersection as the point p which minimizes the sum of the square distances to both lines is calculated in augmented reality applications [9], where marker tracking is used for real-time pose estimation.
The idea is to place the object on marker plane and to transform all coordinates of the point cloud from world coordinate system, to markers plane coordinate system, which we will call objects coordinate system (Fig. 4.).Relationship (rotation R and translation T) between world and object coordinate system can be computed using POSIT algorithm [10].
During the scanning process object is rotated and translated together with the marker plane and every time we can track and evaluate these pose changes.
Our marker plane is composed of two markers, to avoid cases where one of the markers is covered by the object.Any combination of simple geometrical shapes can be used as marker as long as this combination is not rotation-invariant (Fig. 5.).
As an output of such scanning process we get prealigned scanning views of different parts of the surface.
For fine registration step we chose algorithm presented in [11].The algorithm iteratively minimizes a cost function considering all the views simultaneously.

Results
3D scanner and point cloud merging method was implemented using C++ and OpenCV library.From technical point of view, web camera and line laser module are the only hardware used in described scanning system.
It is important that our laser scanner works in real time.Pre-aligned point clouds are obtained as an output of scanning process, which means, that only few iterations are needed for fine registration step.
To evaluate accuracy of the system, simple objects (cube, cylinder and pyramid) of different scales were scanned.The average error of the merged point cloud is close to 1%.The largest error rate is at the regions of sharp edges and corners of the object; here error rate varies about 2-3%.
Figure 6 shows two scans of a candle.Each point cloud is made of more than 200000 points.Figure 7 shows merged point clouds using simple ICP algorithm applied automatically and using our approach.
ICP algorithm tries to minimize point-to-point distance function.Our object has cylindrical shape, so point clouds of different object views are similar to each other and automatic ICP outputs wrong results.Our method evaluates relative position between these views and merges them in to correct representation.
Final point cloud was subsampled and filtered, to reduce level of noise, by applying octree subdivision.

Fig. 4 .Fig. 5 .Fig. 6 . 7 .
Fig.4.Marker plane, with its own coordinate system is placed in world's coordinate system.Any point X w from world coordinate system can be transformed to the point X o in the markers (objects) coordinate system applying rotation and transformation