Wireless Monitoring System for Fireman ’ s Competence Objective Assessment

Developing technologies associated with tracking human movement and behaviour enable new applications for competence assessments from training results of professionals, such as medical staff, sportsmen or emergency servicemen. This article considers a methodological approach to design a system for firefighter’s skills and competence assessment. Assessed training features such as in-building behaviour and tasks execution are analysed based on data gathered with wireless Ultra-Wideband Real-Time Location System, UWB RTLS, and Inertial Measurement Unit, IMU. The assessment is based on the predefined required training tasks, the expert’s expertise and results of the trainee’s test. The Unity game engine is used for data processing and visualization. As the comprehensive final map of the trainee’s skills, the spider diagram is applied and the single score method provides the conclusive statement. The proposed solution was verified experimentally in real environment.DOI: http://dx.doi.org/10.5755/j01.eie.23.4.18723


I. INTRODUCTION
The problem of an objective assessment of professional skills and competences, or precision and progress in rehabilitation is a research topic that gains interest due to the capacity of new technologies, especially wireless sensor networks, Internet of Things and advanced algorithms.However, because of the variety of professionals' performance that might be assessed, it is still an open-ended problem.The comprehensive evaluation of professional competences is even more problematic because of a need to map the different skills into the final score.
Currently, such evaluation is commonly done by means of observation and eventually simple measurements, which are limited and subjective.Moreover, most trainings, like emergency serviceman training often performed under conditions of limited visibility, consist of many elements whereas one observer is able to focus only on a few training features.Therefore, an automated assessment system can reduce the number of evaluators, shorten the time of the evaluation process, and what is probably the most crucial, Manuscript received 11 November, 2016; accepted 19 April, 2017.
the assessment is objective.This article, using the methodical design, resolves the problem of objective assessment of training skills and competences of a fireman as a case study.Based on an analysis of both stakeholders' and future users' needs and requirements, the wireless monitoring and assessing system using a Real Time Location System, RTLS, and Inertial Measurement Unit, IMU is proposed.
The designed system applies location and orientation sensors to acquire data about the completed training path, trainee's Field of View, FoV, execution time, range of explored area, and a number of accomplished tasks.The data, compared with the expert's pattern, provides an objective assessment of the trainee's professional skills and his/her competence.Moreover, using the modern game engine and visualization method along with a spider diagram, the evaluation is presented in a comprehensive way and the training competences are objectively scored.
The simulated results and real field training data validated the design idea and its implementation.

II. SURVEY OF RELATED WORKS
Surgery, rehabilitation, sportsmen, military, fire brigade and other secure services are the professions where objective assessment of people's competence is crucial.However, each profession may have very specific skills that are important and which need to be assessed.
The key features of assessment systems applied in medicine are high precision, repeatability and speed.Such systems are used to assess surgeons e.g. in performing laparoscopy.Taffinder et al., to objectively evaluate surgical dexterity use Imperial College Surgical Assessment Device, ICSAD, consisting of an electromagnetic field generator and two sensors attached to the surgeon's dorsum [1].They assess the surgeon's skills based on measurements of hand movement speed, the distance travelled by his/her hands during the task, along with execution time of the operation.The authors of [2] developed an objective assessment device for speed and accuracy evaluation of the laparoscopic surgery.Their solution consists of laparoscopic simulator and computer-based skills assessment software.Another approach is considered in [3] where a virtual reality is used.The assessment of surgical skills is done using laparoscopic simulator Minimally Invasive Surgical Trainer-Virtual Reality, MIST-VR, where laparoscopic novices perform six tasks and their results are compared with results of experienced surgeons.
In [4], an UWB RFID system by Zebra Technologies has been installed at football stadiums.It identifies players and localizes them with 15 cm accuracy at the frequency of 25 Hz.This solution is used for game statistics for broadcasters and post-game analysis by the coaching staff.The authors of [5] propose the system, which uses multiple cameras to track each of players on the court or field in single player sports for the statistical analysis of covered distance, as well as average and instant speed.The usage of multiple cameras in order to analyse behaviour of players is also described in [6].The authors use them for localization of players in soccer games and display virtual offside lines, players' positions and motion patterns.Similar applications for soccer and basketball games are presented in [7] and [8] respectively.
To objectively assess motion of a person's body or its parts' location is needed in many cases.Most of such solutions require an accurate location, tracking and orientation, which lead to high price systems.However, IMU devices are the reasonable alternative to expensive solutions.In [9] a wearable device for motion detection based on tri-axial accelerometer and two gyroscopes using expert's knowledge is presented.This system compares pattern movements of an expert with the motion of rehabilitated patient during an exercise to correct his/her body position.The average motion estimation error varies from 0.69° up to 1.75°, depending on the speed and exercise difficulty.
To assess soldiers' competences, the authors of [10] use UWB RFID technology.They track the localizations of soldiers engaged in a training in multi-floor buildings, then analyse their movements, actions and interactions to each other, for post-action review lessons.Some researchers go a step further and work with recognition of specific behaviours based on location and orientation data.In [11], Rossi et al. track human movements with humanoid robot and apply the Haar-like features method, to detect human faces.Their solution, in the best scenario, obtains performance of 55px.Ruan in [12] recognizes humans' activity by means of a wireless RFID sensor network.In the proposed method, the received signal strength indicators, from the passive RFID tags array, are used to estimate the subject's activity pattern.The disadvantage of this method is an extensive infrastructure needed to cover all considered area.
The authors of [13], based on information about spatial location of three parts of human body derived from Ultra-Wideband Radio Frequency IDentification, UWB-RFID, tags and Ubisense RTLS, recognize six human behaviors.Applying the data to Artificial Neural Network, the authors obtained activity recognition accuracy of 82 % in the worst case scenario.

III. PROBLEM STATEMENT
The reported studies indicate that the existing solutions of the training assessment of emergency serviceman e.g.firefighters are not rewarding due to the subjectivity factor of the most of applied methods.Therefore, there is a need for a development of the smart multi-sensor system, which would objectively and holistically assess the trainees' skills.
Hence, the objective of this research is to methodically design, implement and validate a system, which would objectively assess trainees' skills and provide a comprehensive map of their skill profile.The skills of interest include, among others their speed, in-building behaviour and tasks execution.The system consisting of handy convenient devices should operate wirelessly even under conditions of limited visibility.
The proposed solution consists of the wireless multisensor system, which, based on RTLS and IMU devices, yields information about trainees' movement in the test-field in real-time.Moreover, using the modern game engine, the particular elements of the training are visualized.The judgment of trainees' skills is based on a template created by experts.The trainees' individual skill characteristic, as a complex variable, is visualized in graphical way using the spider diagram.The trainee's competence is assessed using a singular score, which is related to the expert's pattern.
The main contribution of this paper is applying methodical tools to design, model and prototype the system supporting assessment of trainees when firefighters are the application case study.Moreover, the holistic mapping of the results is proposed and visualized using a suitable chart.Furthermore, the developed system's performance is validated and verified by means of simulation and real measurements from training scenarios.

IV. PRIMARY STEPS OF SYSTEM DESIGN
The design approach used is based on User Centered Design methodology, UCD, where both stakeholders' and future users' needs and requirements are considered, albeit the designer's perspective is taken into account [14].The authors propose to divide the design process into two main stages: the problem formulation and product development.Whereas each step is further divided into sub-steps summarized in Table I.The problem formulation step started with the definition of the general need of the system as supporting the evaluation of the firefighter's training.
According to the stakeholders and future users, the final comprehensive score of the trainee's performance should be based on the evaluation of five features of the training performance, which constitute the system functionalities: 1. a number of examined checkpoints, a, 2. the examined area coverage, c, 3. a number of examined objects, o, 4. the execution time, t, 5. an average speed, s.

1) Number of examined checkpoints
The first feature concerns how many of the predefined checkpoints are examined by the trainee.In most cases, a single room is considered as a singular checkpoint.However in a case of large space corridor, the space could be arbitrarily divided into several smaller areas as checkpoints, which is illustrated in Fig. 1.To assent an area as checked, the trainee has to enter it, which can be verified from localization data and the Unity's feature, which indicates when one object interacts with another one -in this case one object is the trainee and the second is the defined checkpoint.The score, a, from this feature is the ratio of the number of checked areas to the number of all required areas for the training.

2) Area coverage
The second key feature concerns the area coverage, which indicates to what extent the room is combed out.It complements information whether the trainee just entered the room or he/she also examined it carefully by looking around.Each area is divided into a number of subzones as illustrated in Fig. 1.However, some subzones could be overshadowed by obstacles such as furniture etc., which requires from the trainee an entry into the area and examine the space behind the obstacle.The area coverage assessment is done by combining location and field of view data along with the colliders built in the Unity game engine.The score, c, is based on the ratio of how many subzones are passed to a number of all subzones in the training field.

3) Number of examined objects
The third key feature of training assessment deals with a number of examined objects.It corresponds to the cases when somebody hides or lies e.g.unconscious.The example objects, which need to be examined, are depicted by white cubes in Fig. 1.It is assumed that the object is examined if at least 80 % of its area appears in the trainees' field of view, however not farther than one meter from the observer, for at least 0.5 second.
In Fig. 2, the light grey colour indicates the estimated FoV, the darker rectangles are the objects, which are considered as properly examined and the lighter grey rectangles indicate objects, which are not checked.Similarly as in the previous cases, the assessment score, o, depends on the ratio of the number of examined objects to the number of all objects, which should be examined.

4) Execution time
The fourth feature contributing to the final assessment is a total time of the training, t.The system starts measuring the time when the trainee begins moving and ends when a button on the IMU board is pressed.Then the system compares the performance time with an expert's guidelines and assesses the performance.

5) Execution average speed
The fifth feature of the assessment is average speed, which is estimated from the length of the trainee's path and execution time.The length of the path is calculated from location system data.This feature gives the combined information about the ability to execute all tasks in required time.
The complementary requirements consist of the wireless nature.The system should be wearable and operate even under conditions of limited visibility.Moreover, it is crucial to present the comprehensive results in a user-friendly manner.
The stated needs, functionalities and requirements have been evaluated as feasible and the product development step could be continued.

V. PRODUCT DEVELOPMENT A. Technology and Algorithms Selection
The Ubisense Real Time Location System using UWB technology was evaluated as the sufficient trainee localization platform, fulfilling the stakeholders' and future users' needs and requirements.This system consists of sensors located outside the training facility and throughout the building rooms and corridors along with small wearable tags.The localization process is based on the analysis of the Received Signal Strength, RSS, and Time Difference of Arrival, TDoA, from the UWB active tag to the system's sensors.With these technologies, the system is able to estimate a tag's position in 3D with accuracy of about 30 cm at 8 Hz sampling frequency.
To check which areas and objects in the training field have been examined by the trainee, a direction measurement system based on Adafruit's BNO055 Absolute Orientation Sensor and Pro trinket board, are applied.These devices, placed on the trainee's head, give information where the trainee's face is directed, to estimate his/her field of view FoV.The dynamic accuracy of a system's roll, pitch and yaw estimations equal 5º and its weight is less than 150 g with dimensions of 5 cm × 4.5 cm × 2 cm.The operation frequency is at least 50 Hz, and the built-in battery enables at least 2-hour training data logging.The system is equipped with USB interface.
The data processing of the trainee's performance is carried out in the application built in the Unity3D game engine [15], [16].The engine is useful because of its capabilities, offering Integrated Development Environment, IDE, with a built-in editor, scene builder, scripting, physics engine, networking and more.Its deployment ability of the application for different platforms and operating systems was also an important reason for choosing this software.
To visualize the results of skills evaluation in a userfriendly and objective manner, the spider diagram is chosen.This comprehensive method shows all considered training aspects on a single diagram to help the evaluator easily judge the trainee's skills levels.Moreover, this solution can be used to graphically compare the trainee's results the expert's or with the average results of all trainees.
The modelling, prototyping and verification steps are introduced in the following chapters.

B. Modelling
The main goal of the system is to support an assessment of firefighters' training.As mentioned, the two monitoring systems are responsible for data collection.At the hardware layer, these two systems work independently.At the software layer, information from the systems are synchronized.From a perspective of operating principles the evaluation process consists of two phases: online and offline, Fig. 3 and Fig. 4 respectively.When the online phase begins, both systems are independently calibrated, and then their data are gathered and saved.The calibration of the localization system is done by means of the calibration tag placed at the known position, and RSS and TDoA signals are measured using all sensors.The system parameters are adjusted to the actual system state and environment.The calibration of the direction system requires calibrations of gyroscope, magnetometer and accelerometer.For calibration, the gyroscope has to be placed vertically for at least one second.To calibrate the magnetometer, the device needs to be rotated in a figure eight-like pattern.The accelerometer calibration is accomplished after the device is rotated around its axis while stopping for one second for every ~45°.In such manner the direction angle and FoV are adjusted and upgraded.After the calibration the system is ready to be used to monitor the training performance.
To be able to synchronize data of the two independently operating systems, there is a need to establish a timestamp of the training start.To do so, the trainee is asked to perform the starting procedure, which is as follows: go to the starting point; stand upright for one second, lean forward for one second and straighten up again for one second.
Location of the trainee is found by using the wearable tag, RTLS and Location Platform.Orientation of the trainee's head is monitored by means of the IMU device installed on the top of his/her helmet.During the training the data are saved in an internal system memory.
After finishing the training, the data are transferred to the Unity application, where the offline phase takes place.The offline phase starts with synchronization of the data from the two systems.The two synchronization timestamps in each data set are recognized, and then the procedure adjusts a suitable offset of the both systems' time control.
A disadvantage of the used localization system, which is the low sampling frequency, can be overdriven by an algorithm duplicating a number of localization samples.Besides duplication, the algorithm smoothes out the sharp turns on trainee's path and reduces measurement noise.To double the number of localization samples the Chaikin algorithm [17] is used.The advantage of the algorithm is its adjustment by means of two curving factors.The algorithm set of equations is: where ( ) To further reduce noises in the estimate of the trainee's path, a follower filter is proposed.
The key features extraction, comparison with expert's pattern, data visualization and final assessment are described in following chapters.

C. Visualization and Final Assessment
The final assessment can be established and visualized using a spider diagram, where all assessed features are represented, as shown in Fig. 5.The darker part of the figure depicts the specific scores of the trainee and the lighter part represents the expert's results or the average results from all assessed trainees.The visualization provides easily readable information, which trainee's skills have to be improved, and which skills are satisfactory.In a case of five features, the data representation forms a pentagon, whose surface areas depicts the trainee's comprehensive score s T and can be calculated from the formula [17] 1 sin(72 ) ( ), 2 s T st to oa ac cs where s, t, o, a and c represent scores of each of the five key features of the training.In this manner, it is also possible to assess trainee skills in comparison with an expert's ones 100 %, where s F is the final relative score of training and s T and s E are trainee's and expert's scores respectively.

VI. PRODUCT DEVELOPMENT -IMPLEMENTATION AND VERIFICATION A. Implementation
Localization data are gathered using Localization Platform provided by Ubisense.It consists of a few interrogators located throughout the building and wearable tags, see Fig. 6(a).The system was verified and it was proven that the position measurement is very precise and it has about 15 cm-20 cm accuracy.The trainee's FoV is measured using the designed and prototyped IMU device and authors' own application written in C++ programing language based on the Adafruit's BNO055 Absolute Orientation Sensor and Pro Trinket board, see Fig. 6(b).IMU sensor's data are processed by the dedicated Arduino program using the open source library provided by the board's manufacturer.Several different tests for both axes that are crucial in our application, which are yaw and pitch, confirmed that the system meets all defined requirements.The designed system is implemented on Lenovo ThinkPad T440s with i5-4200u 1.6 GHz CPU and 8 GB of RAM with Windows 7 64-bit operating system.
Analysis of the trainee's performance data is carried out in the application built into the Unity3D game engine, which consists of the integrated development environment, IDE, with built-in editor, scene builder, scripting, physics engine, networking and more.

B. System Verification
System verification was preceded by validation of location data preprocessing consisting of the Chaikin algorithm combined with the follower filter.Fig. 7(a) presents an example of the recorded training path.One can observe that the path consists of many sharp turns and some inconsistencies, which could be an effect of RTLS accuracy and precision.These effects could cause a problem when the location data are merged with data of FoV.To avoid the issue, the path is smoothed using the Chaikin algorithm and then the follower filtering is applied.The combination of these two algorithms leads to results shown in Fig. 7(b).The path smoothness after preprocessing is sufficient to synchronize location and direction data.The verification scenario includes three phases.First, the training path has to be defined including a number of areas and objects, which need to be examined.Then a performance pattern of the test run by an expert is established.After that, the system is ready to use.The second step comprises trainee's training attempt when the real data are recorded.The last step includes visualization and assessment of the trainee's performance.
To verify the designed system, the training field was implemented in a lecture hall at Gdansk University of Technology in Poland, where the Ubisense RTLS system is installed.Figure 8 presents the used lecture hall with localization sensors.The desired training path including defined areas, objects and the expert's test run need to be entered into the application.As the test field, the front part of the lecture hall and first two rows of seats were used, see Fig. 10.The chosen space is divided into six areas, in the figure separated with dotted lines, where there are five objects, depicted by the grey rectangles, which have to be examined.In addition, to make the test more realistic, two artificial walls are added.To avoid vagueness of the figure, the subzones of the area coverage estimation are not displayed.However, in reality, each area is divided into a nine equal subzones as it is in Fig. 1.
Figure 10 and Fig. 11 present expert's and trainee's test run respectively.From Fig. 11 one can see, that the assessed trainee missed one area.Moreover, the trainee did not sufficiently examine all areas and thereby missed two objects required in the desired training path.
Table II summarizes the trainee's performance in comparison with the expert.It shows that the trainee's execution time was longer than the expert's and therefore the average speed was also not up to mark.   Figure 12 shows the final trainee's skills evaluation using the spider diagram.The lighter pentagon indicates the expert's result and darker pentagon depicts the trainee's performance.The graph visually depicts that trainee needs to improve precision of field examination.The final score of trainee's competence s F equals 60 % of the expert's one.

VII. CONCLUSIONS
The presented study proves that the wireless multi-sensor system, based on RTLS and IMU devices and yielding information about trainees' movement in the test-field in real-time, is capable to objectively assess the trainees' skills and provide a comprehensive map of their skill profile.
The applied UCD methodology, which takes into account the stakeholders' and future users' requests, demonstrates the capacity to facilitate a precise definition of the problem and corresponding requirements.This design approach results in an appropriate selection of the algorithms and technologies.
The proposed solution is dedicated to operate even under condition of low visibility.The used devices are convenient to handle even during very dynamic and intensive firefighters' training and their performance is visualized in a user-friendly manner.
The system implementation based on Ubisense RTLS and dedicated IMU device, performs all required functionalities under given constraints.The trainee's performance is assessed based on five key features, which are: an average speed, a number of examined checkpoints and objects, area coverage, and a tasks execution time.The selection of the features is done by experts, and in their opinions this selection is most relevant for vocational trainings.
Synchronization of the two independent measurement data channels is done by mean of the timestamp established at the training start.The defined calibration procedures of sensor used ensure the require accuracy and precision.
A combination of the Chaikin algorithm along with follower filter is used to smooth the training path, which enables clear estimation of the trainee's FoV.
The Unity game engine software along with the spider diagram are used to visualize trainee's skills and judge his/her advantages and weaknesses in user-friendly manner.Moreover, it provides the final comprehensive score, which facilitates comparison of trainees' results with experts' and other trainees' results.
Future research may concern an extension of the number of analysed training key features.Moreover, it is planned to improve the Unity application to visualize the scores of the assessed people on the background of all trainees participating in training.Another development could focus on a comparative analysis of history of trainee's results.
The proposed method after adequate modifications could be easily adapted to other applications, such as military or miners rescue training.

Fig. 1
Fig. 1 Example of training environment model with three areas and six objects.

nP
 corresponds to new position points, ( ) c P  corresponds to the current position points; 1 f and 2 f are the curving factors.

Fig. 7 .
Fig. 7. Example path measurement: a) without filtering, b) after using Chaikin algorithm along with follower filter.

Fig. 8 .
Fig. 8. View on the simulating training field.To place trainee's location and FoV data into the training field, mapping of the training environment in the Unity application is required.The map of verification training field is shown in Fig. 9.

Fig. 12 .
Fig. 12. Result visualization of expert's and trainee's performance using the spider diagram.

TABLE I .
STAGES OF DESIGN PROCESS.

TABLE II .
PERFORMANCE ASSESSMENT RESULTS.