Development of Georeferenced 3D Point Cloud in GPS Denied Environments Using Backpack Laser Scanning System

1 Abstract —This paper presents a method to generate a Georeferenced 3D point cloud of GPS denied built structures using custom made backpack laser scanning system. An orthogonal combination of 2D Hokuyo laser scanners has been used on the backpack system to generate a 3D point cloud of the surveyed environments. The data logging of scanners and simultaneous localization and mapping (SLAM) of the scanning system have been carried out using Robot Operating System (ROS). The developed local SLAM based 3D point cloud solution has been transformed into global Georeferenced 3D point cloud using observed geographic coordinates of nearby GPS visible vicinities. Multiple indoor environments have been scanned and 3D point clouds have been developed which have been found accurate when compared to the ground truth. In comparison to available surveying solutions present in the local market, the developed system has been found more accurate, faster, and user friendly to generate structural results of the surveyed vicinities in detail. The efficacy of the system has been witnessed by local surveying companies by delivering the unique global coordinated solutions on affordable rates.


I. INTRODUCTION
Surveying and inspection of built structures and their neighborhoods are routine practices which have been carried out by concerned public and private organizations. The structural information gathered during inspection of already furnished building is considered very essential for various tasks. Formerly the surveying job had been accomplished using manual measurement techniques which were tedious, slow, and erroneous procedures. Since the last two decades, the use of optical laser scanners has been started to replace old manual practices and different innovative mechanisms have been introduced on applications of scanners in engineering surveying [1]. There are various multi-dimensional laser scanners available commercially with varying measurement ranges and angular scanning capacity. Amongst them, the most popular for mapping applications are 2D Hokuyo and SICK laser scanners [2]. Researchers have successfully utilized them in different indoor and outdoor mapping activities using simultaneous localization and mapping (SLAM) techniques [3].
A USA based research group, Indoor Reality, introduced its hand held mapping product IR500, comprising a single depth sensor with external 360-degree camera to generate a point cloud [4]. Researchers have presented the evaluation of the indoor mapping results using trolley based scanning system containing on board sensing and computing facilities [5]. A Canadian company has introduced a scanning trolley system named "TIMMS" having laser scanners with cameras and the system constructs 2D and 3D indoor models of plain surfaces and floors [6]. Generally, the wheeled scanning systems face problems on an uneven surface, such as slope or stairs, and may not accurately scan the environment. Therefore, many research works have fused together several laser scanners along with additional dead reckoning sensors on backpack systems to generate a 2D/3D model of the environment [7].
In contrast to available global scanning and surveying solutions, the regional market has been standing far behind to adopt the latest trends in surveying engineering. This research work introduces a custom made prototype of the backpack scanning system as shown in Fig. 1. The design and fabrication of the backpack have been carried out using simple mechanical procedures, and the surveyor can wear the system and perform the scanning task easily. The efficient 2D Hokuyo 30LX scanners have been selected, analysed, and then installed on the backpack system.

Development of Georeferenced 3D Point Cloud in GPS Denied Environments Using Backpack Laser Scanning System
This laser scanner lies in class I as per laser safety standard, making it feasible for scanning operations in presence of humans and pets [8]. It uses a semiconductor laser source with the wavelength of 950 nm, which is sufficient to scan the surrounding objects present within a radius of thirty meter. Researchers have presented many indoor mapping applications using Hokuyo UTM 30LX inside various types of surroundings [9]. A robotics group has presented scanning and mapping application in an academic environment and presented probabilistic models using Hokuyo scanner [10]. The developed models are usually referred in local scanner frames, and some research works have provided solutions to represent the generated model in a global coordinated system [11]. Another research group has transformed the local image coordinates into the Georeferenced coordinates for the emergency assistances [12].
In continuation to these contributions, this research work presents the method of transformation of the local scanned 3D point cloud into the global referenced point cloud to represent GPS denied vicinities in standard georeferenced coordinates. Systematically the paper has been arranged in multiple sections. In Section II, a brief reference to the related research works is presented. Section III describes the placement criteria of scanners, mechanical design, and instrumentation scheme of the backpack system. Section IV provides details of the point cloud generation and the georeferencing procedure. In Section V, the real test results are described. Finally, the conclusions and acknowledgment are presented.

II. RELATED WORKS
The typical indoor mapping systems can be categorized into three main classes, including hand held, trolley, and backpack systems as shown in Fig. 2. All these systems have been in operation in multidisciplinary applications, ranging from construction engineering to industrial engineering [13].

A. Hand Held and Trolley Based Mapping Systems
Depending on the nature of the task, surveyors have been utilizing any one kind of scanning system, such as hand held ZEB REVO [14]. This system has been considered a simple and effective mapping solution and it has been in use globally for indoor and outdoor applications. Due to its single scanner operation, generated point cloud density is a little bit less, due to which for precise operations, customers have selected trolley systems containing multiple scanners, such as Viametris iMS3D [15]. Some of their characteristics have been summarized in Table I. These systems produce denser point cloud, but may face problems on rough surfaces [16]. Therefore, the backpack scanning systems have become the natural choice for surveying applications in indoor vicinities having slopes or improperly levelled floors.

B. Backpack Scanning and Mapping Systems
Backpack systems have been designed to overcome the challenges of the surveying indoor and outdoor environments with stairs and an uneven terrain, respectively. They have started emerging since the last decade. A French company "Viametris" has introduced a scanning backpack system named "bMS3D LD5+" having 3D laser scanner along with cameras mounted on it and able to construct 2D and 3D indoor models of surfaces and floors [17]. This product has been vastly selected for various surveying and modelling tasks. A detailed analysis on the performance of the system has been presented by researchers [18]. In addition, the product has been efficiently applicable for surveying jobs in rigorous terrain, such as dense forests. Many researchers have also presented tremendous modelling results with a new georeferencing technique of such terrains with compact backpack system [19]. Several other companies have also started working on the backpack based laser scanning technology and have come up with multiple solutions. The Indoor Reality has also introduced its backpack system IR1000 equipped with five laser scanners and some other sensors [20]. The orientation of multiple laser scanners has been adjusted precisely to capture various planes of the surveyed vicinity and has provided a transformed 3D slice of the area at any instant. Some of the characteristics of these backpack systems have been summarized in Table I. A compact scanning system has been deployed by researchers on backpack and on aerial vehicle and presented multiple models of the surveying environment [21]. Though testing of both mechanisms has been performed in relatively less dense regions, but the system is able to develop a model of interior and exterior of the surveyed vicinity using bird view. Another well-known backpack system has been developed by Leica Geosystems [22]. Using its 3D scanning mechanism, the system is applicable in different indoor and outdoor environments. Some researchers have developed a local point cloud using custom made backpack system and using image based georeferencing, they transformed the local point cloud into the global coordinated system [23].
A continuous research and development is underway related to designing and fabrication of the backpack scanning system. In this research work, a custom made design has been tested and provided to the local industry at economical rate for improving their working efficiency using the latest technology.

III. DESIGNING OF BACKPACK MOBILE SCANNING SYSTEM
Designing of a backpack mobile scanning system involves the selection of suitable laser scanner, the appropriate placement of multiple scanners at its core along with the development of mechanical and instrumentation hardware as discussed in the following sections.

A. Selection of Scanner
There are several 2D/3D laser scanners available for indoor mapping applications. Considering financial limits, only 2D laser scanners listed in Table II have been tested and analyzed for the final selection of the scanner. The main features, which have been required in the scanner, are its accuracy and scanning speed to link it with the motion profile of the surveyor. So, the research team has picked Hokuyo 30LX scanner by finding better stated characteristics values from the manufacturer as summarized in Table II. In this course, the comparison tests have been executed with other scanners. A test for the performance evaluation has been conducted inside a lab where all three scanners have been uninterruptedly run for 30 minutes and their scanned values have been monitored for a particular object placed in front of them as demonstrated in Fig. 3. These values have been plotted and compared for all scanners and presented in Fig. 4. For initial 15 minutes, all three scanners have been sensing the range of the object placed at 4.25 m quite accurately though Hokuyo scanners have been found more accurate. After this time, comparatively more errors of 1 cm have appeared in the range values of RPLidar. So, it is concluded that Hokuyo 30LX provides accurate readings for longer ranges and it can be used for mapping tasks where accuracy is the key requirement.
Furthermore, to check the scanning performance of the Hokuyo scanner on different materials which may be present in the surveying vicinity, another test has been performed for 35 minutes where Hokuyo has been used to scan three stationary objects of wood, acrylic, and metal placed at different locations. All range values of the scanned objects have been plotted as shown in Fig. 5 and quite stable and satisfactory results have been observed for all materials. Therefore, the selection of Hokuyo 30LX has been finalized for the development of the backpack system.

B. Placement of Scanners
A single scanner is only able to scan the XY plane of the surveying environment and can produce 2D pose and map information if distinct geometrical features are available in surroundings [3]. However, for building 3D map of the environment, another scanner is needed to attach with the system that can scan XZ plane. A possible angular arrangement for connecting two scanners on a sensor box is shown in Fig. 6(a). The local frame of the horizontal scanner is considered as the reference of the scanning system and is used to define the local frame of the other scanner. The horizontal scanner scans XY plane and its yellow scan line on a wall surface is shown in Fig. 6(b). The second scanner has been connected at translational (x 2 , 0, z 2 ) and rotational (0, θ y , θ z ) values as shown by its local frame 2 in the geometrical representations. To keep X2 axis upside, angle θ z will fix to a constant value of 180 degree w.r.t. reference Z1 axis. However, the angle θ y between Z2 and Z1 axis needs to be set properly and initially assign some value such that 0 < θ y < 90°. The second scanner scans the wall surface as shown by inclined blue scan 2 line. If angle θ y sets to the orthogonal value, then the second scanner will scan the wall surface vertically as shown in Fig. 7(a). The geometric relationships for a new placement scheme are shown in Fig. 7(b). The second scanner is now scanning the wall plane vertically as shown by vertical blue scan 2 line. The blue line only holds ∆z component and having no ∆x component which was significantly present in angular placement scheme. So the horizontal scanning region is drastically reduced by scan 2 line. To justify which possible orientation scheme suits the surveying applications, a physical box placement is shown on a backpack frame in Fig. 8(a). The respective axis definition of the surveyor wearing the backpack system and the axis of the scanning system are shown in Fig. 8(b). The surveyor may move in X s direction to scan the region, and during the motion, possible unintentional angular disturbances of (θ a , θ b ) may appear on the system due to the shoulder movements of the surveyor as shown in Fig. 8(b).
These disturbances force both scanners to scan the region from different views. Figure 9(a) indicates the angular placement scheme where both scanners exhibit the variations occurring on their scans. The horizontal yellow scan 1 line may acquire new possible scan positions shown by red and cyan lines when θ a or θ b may change, respectively. Similarly, the angled blue scan 2 line may reach new places shown by red and cyan lines, respectively. Therefore, it is clear that small angular disturbances caused by surveyor's movement may change the effective scanning region of the second scanner significantly as depicted by ∆X b component. However, in the orthogonal placement scheme as shown in Fig. 9(b), a limited variation of blue scan 2 line can be seen by cyan line in response to the motion profile of the surveyor. Numerically, the horizontal projection ∆X b can equate to R 2 Cos(θ y + θ b ) where R 2 is the total length of scan 2. Setting the smaller value of θ y in angular placement scheme causes the greater ∆X b projection that makes a larger scanned region effected by the angular θ b noise. At this moment, its 3D registration process using noisy horizontal scan based pose estimation causes in accurate placement of the scan exceeding the actual wall surface as shown in Fig.  10(a) through front and top views of the wall. However, setting the orthogonal value of θ y causes a very limited noisy scanning region as shown in Fig. 10(b) and produces better 3D registration. Hence, the orthogonal combination of scanners has been finalized for this work due to its better performance against the procedural noises. An additional third scanner has been attached with the system to scan the XZ plane specifically the floor surface as shown in Fig. 11(a). The scanner has been connected with fixed translational (0, y 3 , z 3 ) and rotational (90 °, 0, 90 °) values w.r.t. reference frame and represented by green color in Fig. 11(b). During the normal surveying, the third scanner measures the range R 3 of the floor surface. In the case of abrupt angular noise, the view point of the third scanner may also change along with the range measurement R 3a as shown by the red scan line on the floor. Using the measured range observations per scan of the surveyed floor plane, a motion profile of the surveyor may predict and interpret the accuracy of the generated poses during the scanning operation. To estimate the range R 3 of the floor surface from position 1 during the scanning operation, a series of scanned values have been measured for the surface and labeled as P 1 , P 2 , P n till P m as shown in Fig. 12(a). Each measured value has been multiplied with the respective scanning angle to get a new measurement for R 3 and to establish the vector y pt as shown in (1) Here H p is the unity measurement Jacobian matrix. The same procedure can be performed to estimate the range value on position 2 using the respective scan values and can further be applied for all surveying poses. A range profile plot can also be developed as shown in Fig. 12(b). If the surveyor moves smoothly on the plain floor surface, then the resultant curve will be almost a straight blue line, while in the presence of angular disturbances, a red line plot may appear indicating some deviated range values than the normal ones. Physical angular and orthogonal placement setups, as shown in Fig. 13, have been testified during short surveying experiments inside a portion of corridor. The surveyor has moved twice inside the region by carrying each setup once and deliberately creating motion noises. As a result, two separate 3D maps have been formed. To see the accuracy of maps, a small wall plane has been extracted as shown in Fig. 14. It is observed that the proper wall plane has not been formed through the angular setup and some empty spaces have appeared. While in orthogonal setup, much better wall plane has been observed with quite accurate window openings. To quantify errors in the developed maps, the same wall portion has been scanned using stationary scanning and its standard plane model n w .X pn = -d w has been developed, where n w is the normal vector and d w is the distance of the wall plane from origin. Later, the extreme points of each axis present in the individual developed map have been identified. Each extreme point X pn has been placed in the developed model to determine its absolute plane fitting error e n as shown in (3) .. The calculated RMSE values for angular and orthogonal scanned maps have been found 0.176 m and 0.124 m, respectively, indicating more stable performance of the orthogonal setup. Using the scan values of the third scanner observed during both tests, the range estimation profiles have been determined using (2) and plotted in Fig. 15. Both green and blue plots are indicating the estimated range variations due to the appearance of motion noise during the testing of angular and orthogonal setups, respectively. This study has been found to be quite effective to do scanning jobs accurately where smooth profiles have been observed for developing the maps as presented in the result sections.

C. Mechanical Model
To manufacture the desired backpack hardware for indoor mapping, at first stage, a mechanical model of the backpack has been designed in the SolidWorks software as shown in Fig. 16. The backpack system has been comprised of two metal plates and its supportive frame connected firmly with each other. On the top plate, the scanner's mounting box has been attached. While on the bottom plate, the placement for the electronic circuitry has been performed. After finishing the simulated design in solid works, the manufacturing of the backpack has been initiated.
The base plate has been manufactured using a stainlesssteel sheet of thickness 0.6 cm, which gives support, and the wooden cardboards has been placed on the top of it. In addition, another plate of the same stainless-steel sheet with back pillar supports at height 28.5 cm has been fixed on the top of the main base plate. The top plate has been utilized to integrate the scanner's mounting wooden box and the wearing strap. The bottom plate has been used to hold battery, electronic circuitry, and laptop for interfacing scanners and other sensors. All three scanners have been placed on the wooden box using small metal plates. First Hokuyo scanner has been mounted horizontally, while the other two have been mounted vertically on the front and side planes of the box. An additional inertial measurement unit (IMU MPU 6050) has been placed at the center of the cubic wooden box to record the orientation updates of the system. One HD camera has been connected on the front plane to visually perceive the environment. The backpack design has been kept simple, low cost, and if needed, extra sensors can possibly interface with it.

D. Instrumentation System Design
The designing of the electronic instrumentation circuitry comprises of interfacing multiple sensors with ROS running on the laptop. In this sensor fusion work, all three Hokuyo scanners have been linked to the network port of the ROS laptop as shown in Fig. 17. The reference horizontal scanner has been used to scan XY plane of the vicinity and its live data have been utilized to run the open source Hector SLAM ROS package to produce 2D pose and the map [26]. All the online data streams of all scanners have been saved along with 2D pose and map information using ROS bag package during the surveying operation of the respective vicinity. In the instrumentation scheme, some additional sensors have been interfaced in the system including, IMU and HD camera. IMU has been connected to the Arduino embedded board which has been serially transmitting the online data to the ROS for data logging. The HD camera has also been interfaced to ROS through the USB port and live images have been viewed online in RVIZ package and in parallel stored in a respective bag file.
After finishing the surveying and scanning task, all the stored ROS bag data file has been processed in offline mode using the Matlab coding. The processing comprises of 3D registration of horizontal and front vertical scan points, extraction of building planes, and transformation of 3D SLAM results into georeferenced coordinates. The left vertical scans have been analysed to observe the motion profile of the surveyor and the smoothness of the floor at which the surveyor has been moved inside the vicinity. The IMU data have been used to visually compare the generated SLAM 2D poses along with the motion profile of the surveyor. For all the experimental tests which have been presented in this work, the accuracy of the SLAM results has been found far better due to the availability of the distinct geometric features inside the surveyed vicinities. Moreover, no uneven floor or significant motion noise has been identified through the analysis of left vertical scans. Therefore, the SLAM based poses have been found sufficient for the 3D registration of scans.

IV. 3D POINT CLOUD GENERATION AND GEOREFERENCING
The development of the 3D point cloud and its georeferencing have been presented in this section. The procedure for generating 3D scanned points of the surveyed environment involves at its core an accurate 2D pose estimation during the scanning operation. The actual idea of the SLAM is to estimate the most probable map m and pose x by factorizing the complete SLAM posterior p(x 1:t , m | z 1:t , u 0:t-1 ) into the factored form as shown in (5) Here the 2D pose of the scanning system is shown by x, environmental map by m, horizontal laser observations by z, and controls (odometric updates) by u. The Hector SLAM ROS package has been used to generate an on line 2D pose and map information. The Hector SLAM does not utilize u by odometry rather does it through scan matching and it generates occupancy grid map of the environment. In the first exploration of the scanning setup, a long corridor vicinity of 50 meter has been surveyed containing pillars and walls as shown in Fig. 18. The surveyor has been moved slowly in a straight direction by wearing the backpack scanning system. The online data of the horizontal scanner have been utilized to compute a 2D pose and a map of the environment which are shown by red and cyan colors, respectively, in Fig. 19. Later, the front vertical scan data have been transformed into the referenced horizontal scanner frame in offline mode using the MTALAB scripting. Here S 2 S 1 is the transformed scan of the vertical scanner into the referenced scanner frame and it has been accumulated with the horizontal scan. Since each scanner has 1081 scan points in a single scan, so total points in the complete scan S c have been reached to 2162 count as depicted in (7) 1 2 1 .
Now using the distinct recorded poses at the respective time, the complete scan S c can be registered into the SLAM coordinated system to incrementally generate a consistent 3D point cloud. The final transformation of the complete scan into these coordinates has been shown in (8) , .
Here the respective translational (x n , y n , z n ) and rotational (θ n ) pose values related to the movement of the surveyor have been estimated earlier in the mode using the SLAM. By utilizing all scans recorded at different time stamps and their respective pose information, the final 3D point cloud has been generated in the offline mode as shown in Fig. 20. The generated point cloud represents the surveyed vicinity of 50x40 m 2 dimension and shows the detailed information of structure, vegetation, and open spaces. The time taken for scanning the environment was approximately fifteen minutes and the time consumed for developing the point cloud was roughly thirty minutes. So, the overall time of scanning and mapping is quite small in order to gather quickly the comprehensive information of the surveyed vicinity. Furthermore, the Random Sampling Consensus (RANSAC) based segmentation of the developed point cloud has been carried out in Matlab [28]. It is the process to group the scanned points belonging to the respective planes present in the vicinity, such as floor, ceiling, and wall planes. To visualize some inner structural planes, the floor and ceiling segmented planes have been removed from the original point cloud, and the remaining planes, such as wall and pillars, are shown in Fig. 21.
The dimensions of the developed point cloud have been found accurate as compared to the ground truth. Finally, the developed point cloud needs to transform into the standard georeferenced coordinates. The surveyed vicinity has no access to GPS signals like the other indoor environments. Figure 22 shows such a representation of the surveyed corridor where the red dots indicate the possible movement of the surveyor in the GPS denied region. The GPS reference values have been observed from the nearby road where markers A, M, and B are known GPS locations. The surveyor's starting location is shown with the point C making a perpendicular with the line segment AB at the marker M.
The distance between M and C has been physically determined, so the vector MC can be decomposed into its components of d x and d y by first identifying θ 3 using (10) Finally, the decomposed components have been subtracted from the GPS marker M to determine the GPS marker for C as shown in (12) So, every surveyed point along and nearby to the segment CD can be measured, which in turn completely represents the surveyed vicinity through the georeferenced coordinates. Figure 24 shows the georeferenced point cloud of the corridor. V. RESULTS The backpack system has been further tested in various real environments. The second real test has been carried out inside a long square shaped corridor as shown in Fig. 25. The surveyor has scanned the environment, and the developed 2D map along with the surveyor's movement is shown in Fig. 25(b). The 3D point cloud of the corridor has been developed as shown in Fig. 26. The complete point cloud represents floor, ceiling, walls, pillars, and even vegetation around the surveyed vicinity. The developed point cloud has been compared with the ground truth as shown in Table III and has been found nearly accurate. The RMSE value of the scanned wall w.r.t. model wall plane has been determined and has been found satisfactory. The developed point cloud has been transformed into the georeferenced coordinates as shown in Fig. 27. The third test has been carried out inside an under construction building to see the performance of the system as shown in Fig. 28 along with its respective 2D SLAM result. The 3D point cloud of the surveyed region has been developed as shown in Fig. 29. The complete point cloud represents floor, ceiling, walls, windows, and open spaces around the vicinity. Though the surveyed region was under construction, but still an accurate point cloud has been formed, indicating the applicability of the scanning system. Finally, Fig. 30 shows the georeferenced coordinates of the surveyed vicinity. The developed point cloud has been compared with the ground truth as shown in Table III and has been found nearly accurate along with the acceptable RMSE. So, by observing the quantitative results of both tests, it has appeared that the scanning system has performed very well in real environments even on the construction site.
However, the presence of distinct geometric features has been observed in all surveyed environments. Therefore, further testing of the system has been planned to perform in those regions where similar structures are present.

VI. CONCLUSIONS
This research work presented the operation of the efficient orthogonally configured backpack scanning system. The developed mapping results of the system have been found 99 % accurate if compared with the actual dimensions of the surveyed vicinities. The time taken for generating the surveying results has been minimized to 80 % or more with ease of operation at affordable rates if compared with the available surveying techniques in the regional market. In addition, the procedural transformation of the local mapping results into the georeferenced coordinates has been presented to exhibit the structural information of the GPS denied vicinities as per global standards. To improve the localization and visualization of the surveyed regions, the use of the recorded monocular vision information has been planned in the future works with latest classification and segmentation techniques.