Mining Shaft Inspection by Laser Photogrammetry

One of the most important tasks in the area of mining safety is the inspection of vertical mining shaft. Geological processes in the rock mass rise problems with all used mining components over a long-term period. There is a possibility of occurrence of an equipment deformation or destruction in all directions (there are installed pipelines, cables, long vertical linear guidance, etc . inside a shaft). Periodical inspection of mining shaft profile and its analysis are necessary to ensure a safe transport both of the material and especially staff. Base of the solution of this problem lies in the use of contactless measurement methods of the profile, such as photogrammetry described in this article. First it gives a description of photogrammetry problem, pros and cons of proposed implementation with solid state LED or LASER LED used as light sources (instead of xenon flash tube) as well as flash timing and control and spectral distribution of the light. Then computer analysis of mining shaft profile as a part of rapid diagnostics is described. DOI: http://dx.doi.org/10.5755/j01.eee.22.1.14109


I. INTRODUCTION
Mining activities result in creation of large areas inside rock massif which is impossible to map or to analyse due to unsafe access to them.The same problem occurs with rather accessible areas but with very extreme conditions (possibility of measurement device destruction or lifethreatening environment) regarding measurement process.Accuracy of the shape, size and position of these areas are very important for safety and economic factors of mining activities.
There are many methods describing how to prepare a map of the areas on the ground but less number of methods for dangerous areas in mines due tough condition mentioned above.One from very interesting up-to-date methods is photogrammetry.
Photogrammetry is a measuring technique that allows triangulation of multiple photos, resulting in creation of 3D model and surface texture within a post-processing job.
Manuscript received 5 January, 2015; accepted April 13, 2015.This work is supported by project SP2015/184, named "Control of technological systems with OAZE providing an independent sustainable development of complex systems" of Student Grant Agency (VSB -Technical University of Ostrava).
Photogrammetry is one of suitable methods for this type of measurement but it has some limitations in case of using in the mines due several reasons.The first set of reasons lies in specific condition of mining spaces namely the availability and quality of light, presence of the water, fog, mud, material of massif, camera sensitivity, possibilities of post processing, etc.Other reasons are connected to safety and security of regulations.It is impossible to use any measurement device within mines.
Advantage of photogrammetric method is staff safety and economic benefit of rapid re-measurement key parts of mines but on the other hand this method requires higher technical and technological investment to achieve its successful implementation.
Time of the measurement is another complicated part of the measurement in mining areas which could be connected with regulations, for example, people exposed to danger environment or technical and technological time scheduling of mine operations.This could be a crucial point of selection of measurement technology for periodic tasks in full-time running industrial operations in mine.Correctly adapted photogrammetric method could be a good solution of this time-based problem in given measurement problematics.We decided to design a new device for inspection of mining holes with the use of photogrammetry method motivated by reasons described in previous paragraphs.

II. SINGLE SHOT PHOTOGRAMMETRY
Single shot photogrammetry is one of photogrammetric methods [1], [2] suitable for measurement under dynamic conditions, especially useable in motion.Photogrammetric tool consists of two parts.The first part is a camera which takes the pictures of the measured scene.The second one is the light source.
Light source is mounted under camera container and the distance between camera and light source depends of used objective and maximum radius of the scene.In our system there are two modules connected by steel rope and these measurement modules are self-stabilized due to the gravity.Both of these modules are hanging on the mine lift and they are moving with his motion and scanning mining hole.Basic scheme and picture of complete device for photogrammetry implementation are shown in Fig. 1.Simplified description of the measurement is clarified in chapter Experimental verification.

III. SAFETY OF OPERATION
The entire equipment used in mines, especially in deep coal mines, has to fulfil conditions and regulations which are very specific for this industry.Basically we can view this equipment as an electric device and that's why it must be safe for operators from this point of view.
There exists a solution for both sources of the lightxenon tubes and LED.LED diodes simplify the light source circuit because it uses a low voltage power supply, for example NiMH batteries, without any DC/DC converters of generally voltage multipliers.Xenon flash (stroboscopic) tube needs high voltage for its operation and specific high voltage impulse for an ignition of flash discharge.This part must be well isolated and also mechanically protected.
The second important task is intrinsic safety.There are many limitations and restrictions for this category of devices.Generally the prospective quantum of the energy is limited, which is the energy that could be released in one moment.Other limit values are related to maximum impedance, capacitance, battery capacity or minimum battery resistance, etc. Xenon flash light used for experiments needs energy about 60 W per second.It is enough for methane gas ignition in case of malfunction or damaged device.There are some ways to protect these kinds of devices and make them acceptable for operation as intrinsic safety devices but all of them bring negative final effect -they add a mass to final measurement device.This is the reason why we decided to change the xenon flash tube light source to different technology with LED light source.However, nowadays our solution with LED light source is not yet classified as the intrinsic safety device.It is a future plan connected (not only) to solution of certain technology problems.

IV. CONTROL AND DRIVE LED LIGHT SOURCE
LED diodes have several advantages in comparison with xenon flash tube.But there was a necessity of a total redesign of control system for operation with LED diodes.We designed a new control system based on 8-bit microcontroller for this purpose.Primary function is control of the light in captured scene and communication with camera.Our system supports control of the following attributes of light source [3]- [6]:  control of LED segments,  light intensity,  periodic repeated flashing,  length of flash,  continuous light (unable by Xenon Flash source).
Synchronization of operation is provided by one wire between standard camera and light source interface with optical galvanic isolation of both working parts of this system.It is also possible to replace metallic wire by optical cable in order to reduce an interference using a long cable (distance between camera body and light source could be more the ten meters).
Main cons of this solution are longer time of most short flash due the lower energy per peak.This could cause some problems with longer exposure time for camera and distortion of pictures captured at higher rate.

V. EVOLUTION OF LASER SYSTEM
Although the LED system improves existing solution significantly, we decided to keep developing of next generation of light source system.At the beginning there were experiments with thickness of the light beam reduction and its divergence because this was a big problem in our optical system which was focused for one diameter of mining shaft only.Design of focusable optical system for light cutting plane is very difficult.Especially this holds good if this device is supposed to work under tough conditions of the environment like mining industry.The movable component in optic system is unwanted due to the reason mentioned above.We decided to try to use a laser system with fixed optic.However, a single laser was unable to cover 360 degree of cutting plane which is necessary for proper function for measuring system.The next step of development was using the system with more laser sources where each semiconductor laser module included its own optic subsystem.This solution also significantly reduces the price of lasers and optic subsystem because it allows using standard optic component instead of custom parts.Simple model of laser arrangement in the prototype is shown in Fig. 2. We decided to protect intellectual property of described devices by registration at the Czech Industrial Property Office because we designed a new unique device for the mining shaft measurement.This registration was finished by patent under number CZ303699.The main part of new measurement system (Fig. 3) is shown in Fig. 4. In present measurement system there is a disclosed laser measuring system consisting of eight laser modules (7) working as light sources provided by integrated optics in order to form a narrow light trace (6).At least three laser modules (7) are disposed between two base plates ( 5) thus forming a sectional plane of total angle of 360 degrees.A power supply unit ( 4) and an inertial system module for measuring incline are disposed in one of the base plates (5).A digital camera ( 2) is suspended on steel wire ropes in a suspension (1) above the all laser modules (7) at a suitable distance.The entire laser measuring system is then connected to a control unit.Next innovation lies in adding a measurement of inclination of light cutting plane and then the camera is able re-compute cutting plane to real horizontal plane.For this purpose we designed a module of inertial system on both main parts (camera and light source part).It an integrated three axis acceleration sensor, three axis gyroscopes and three axis magnetic sensors.Also there is a possibility to add a pressure sensor to measure approximate depth of the shaft.Prototype of the sensors board is shown in Fig. 5. Wireless communication between the units is also aim of future work.

VI. EXPERIMENTAL VERIFICATION
Two calibrated digital cameras FujiFine Pix S2 Pro (6.1 MP) and Olympus E-20 (5.2MP) were used for testing of accuracy depending on the camera's resolving power.The beginning of the blind shaft bottom was always taken as the first measured horizon.This profile dimensions were measured using a steel tape.The 20 m segment of the blind shaft was measured twice using the FujiFine Pix S2 Pro camera.The first series of images was acquired bottom-up, the second one from the top down.An example of the captured images of the laboratory blind shaft is shown in Fig. 6.The FujiFine Pix S2 Pro camera was connected to the control unit (notebook) via the high-speed IEEE 1394 (FireWire) interface, allowing use of a special program (Camera shooting software) to trigger the exposure automatically as well as to store the images to the hard disk of the computer.Our measuring application for controlled photogrammetric measuring chain use RealVNC software.At the moment of exposure there was a possibility to inspect the captured images immediately and make a shift to a horizon at higher level provided that the captured image was a high quality.The third series of images was captured bottom-up using the Olympus E-20 camera.During measurement with Olympus E-20 the function of autosensing at the interval of 1 minute was used and the captured images were stored directly to the CompactFlash card [7].
All captured images were first pre-processed using the FOTOM software developed by the Department of Informatics at the VSB-TU Ostrava.Rectangular ELEKTRONIKA IR ELEKTROTECHNIKA, ISSN 1392-1215, VOL.22, 1, 2016 coordinates X, Y of all evaluated points were obtained from the images using this program.
There were created applications to facilitate the graphical display of the blind shaft profiles.The Profiles application provides a graphical representation of blind shaft profiles viewed according to height horizons.For accuracy comparison it is possible to plot the matching profiles from individual series on each other and distinguish among them by colours, to plot the profiles of the entire blind shaft on each other or to create a wire model of the shaft (Fig. 7).
The Verticality application allows display of the representation of the profile verticality based on the points of interest of the blind shaft.Figure 8 shows the verticality profiles based on individual measurement series.For better comparison of accuracy they are plotted and colour-marked.Because there are no different objects in the laboratory there were chosen corners as the objects of interest.These profiles can be graphically illustrated in either X or Y directions in the measured horizons from the vertical which go through the first profile point of interest.The numeric value next to the curve is the size of deviation from the vertical line, expressed in millimetres.The profile representation of the blind shaft straightness is shown in Fig. 8.These graphs are plotted after Straightness application is launched.The whole figure differs from the blind shaft verticality picture so that instead of deviations from verticals the deviations from the line connecting points of the first and last profile are illustrated.The deviations are also expressed in millimetres.For a classical circular shaft the dimension (radius) of the shaft target would be also further determined in several steps.First, the local dimension of the blind shaft is calculated, followed by the calculation of radius on the horizon, and finally the calculation of the shaft radius.Further, it is also possible to track deviations in the shape of the shaft target.

VII. CONCLUSIONS
The use of microcontroller-driven Laser LED flash in photogrammetry, especially for mining shaft inspection, could bring many advantages in comparison with a conventional solutions like solid state or bulb less light source, possibility of controlling intensity and time of flash or continuous operation mode, lower energy consumption and shift of devices mass from payload to enclosure make possible to build robust and reliable measuring equipment that can operate in rough environment with different light conditions which is usual in mining industry.This device can also simplify and speed up photogrammetry method and it brings more important and affordable of daily used based inspection of critical part of mine and its environment [8].Also there is prepared intrinsic safety variation of this design for daily coal mine shaft scanning.This could provide high performance diagnostics of changes or deformation of mining shaft equipment.
In additional there is possible to collect data for statistical analysis and rock massif movement's prediction.It is very important especially in case of mine conservation.Due restrictions there exist many coal mines closed today.Automation of monitoring process could bring significant cost reduction associated with mine maintenance.
Finally the commercial device is planned for the future due to interesting results of experiments and progress in LED technologies.

Fig. 7 .
Fig. 7. Illustration of profiles using the profiles application.