Intelligent Environmental Recognition Features of “Robosofa”

For more than a decade, representatives of various disciplines investigated the idea of artificial intelligence, which includes such areas of science as "intelligent environments, computer vision, autonomous vehicles and robots. Intelligent environmental adaptations are closely linked to visual information processing in different graphics elements extraction and recognition. Image processing investigates the environment where the camera device is present; by taking and analyzing the captured information. A huge increase in image quality and computational technologies allowed opening of a big range of possibilities in image processing [1]. Nowadays, specifically, image processing can be split into two main branches, which are: positioning [2] and augmented reality [3]. Positioning through image processing techniques is getting a significant boom, due to, not only the possibility of being able to identify the position of certain objects and/or persons in an environment, but as a reinforcement in GPS systems which do not work inside houses ant etc. However, designing an image based positioning system is a complex process because position system inside the surrounding environment relies on many uncontrolled issues (lightening, object displacement etc) [4, 5]. Therefore, positioning task is simplified to the detection of certain visual markers (tag) or patterns in this work. Each marker that is used in research has own semantic value that describes the certain environment (room, corridor etc). The paper is organized in four sections. In the second section the state-of-the-art methods are presented. More about markers, proposed recognition algorithm and experimental results are presented in the third section. The conclusions and discussions are introduced in the fourth section.


Introduction
For more than a decade, representatives of various disciplines investigated the idea of artificial intelligence, which includes such areas of science as "intelligent environments, computer vision, autonomous vehicles and robots.Intelligent environmental adaptations are closely linked to visual information processing in different graphics elements extraction and recognition.
Image processing investigates the environment where the camera device is present; by taking and analyzing the captured information.A huge increase in image quality and computational technologies allowed opening of a big range of possibilities in image processing [1].Nowadays, specifically, image processing can be split into two main branches, which are: positioning [2] and augmented reality [3].Positioning through image processing techniques is getting a significant boom, due to, not only the possibility of being able to identify the position of certain objects and/or persons in an environment, but as a reinforcement in GPS systems which do not work inside houses ant etc.However, designing an image based positioning system is a complex process because position system inside the surrounding environment relies on many uncontrolled issues (lightening, object displacement etc) [4,5].Therefore, positioning task is simplified to the detection of certain visual markers (tag) or patterns in this work.Each marker that is used in research has own semantic value that describes the certain environment (room, corridor etc).
The paper is organized in four sections.In the second section the state-of-the-art methods are presented.More about markers, proposed recognition algorithm and experimental results are presented in the third section.The conclusions and discussions are introduced in the fourth section.

Related works
Interest within smart environment recognition as well as artificial intelligence has been growing for more than decade.Lots of realizations are based on image processing and recognition.There are various ways to extract useful data from image by processing in but only some of them with practical application will be introduced.
The relevant data-processing method is used for identification markers [6].In work, they propose a new localization method which is based on invisible markers that consist of translucent retro-reflectors.In the proposed method, the markers are invisible to the human eye and can be captured using IR camera.There is no need in any power supply and era.To get system more robustness infrared LEDs are flashed on and off continuously and decrease the influence of the infrared and other reflection from retro-reflective markers.Then the images containing the markers are captured then the LEDs flashes and the markers region are extracted using difference between images then LEDs are off.This system can guide a user through an unfamiliar building to a destination room knowing exactly position of the user.
For this system visual markers are graphic symbols designed to be easily recognized by machines (Fig. 1).They are traditionally used in augmented reality [7] and processing algorithms such as ANNs are applied [8].Generalized Hough transform (GHT) is often used for two dimensional marker identification which consist of round objects [9][10][11].GHT detect smooth curves, therefore, it is widely used for text recognition, various forms identification and separation in partly occluded environment [12].Despite of high computational cost GHT method can be used 2-D gray scale object detection in images and to create 3D space map using stereo vision system [13] for mobile robots navigation and obstacle avoidance.Scanning a visual marker through a phone camera, users can retrieve localized information and access mobile services [14].The flexible positioning system can be developed using hybrid solution such as ARToolkit [15] for optical tracking and room geometry information with intuitive user interface for wireless control of home equipment.
The fastest and cost effective image processing method is logical separation.The method is carried out by comparing each frame of gray shades the brightness of the active point to a specific integer value called threshold (0÷255).If the grayness of the pixel is less than the threshold, the value "0" is given; otherwise the point is signed with value "1".Traditionally fixed threshold value is used in image processing which is very sensitive to varying lighting condition.The adaptive threshold selection overcomes mentioned difficulty [16], however a priori information is needed (e.g., size, number of objects etc.) for selection of primary threshold value [17].The logical separation is widely used, were objects or certain markers are in high contrast with the rest of surrounding environment.For example, the color of road markings is clearly distinguishable from the color of roadway, so the logical separation can be applied in the development of driver support systems and autonomous vehicles.The accurate detection of road markings keeping them in proper shape and orientation is necessary for the determination of markings term.It is difficult task, which is solved using not only logical separation but also patter recognition technologies [18].The images of the road markings often are distorted because of camera viewing perspective and varying illumination.The authors of the work [19] have presented inverse perspective transformation (IPT) that performs image transformation from three dimensional space (two image coordinates and viewing angle) in two dimensional plane (image coordinates).IPT method allows getting the top-down viewing effect (Fig. 2).Road markings detection is widely used in creation autonomous cars.The paper [20] presents a system, which uses the logical separation and Gaussian comparison function to extract several lines and markings.The vigilant car can be created by determining the path, markings and road sings.The amount of the work and extensive research which is carried out by many researches shows the relevance and importance of the work.Rapidly growing interest in autonomous vehicles, smart mobile platforms and furniture motivates to create new methods and algorithms suitable for it.

Algorithm of environment recognition
Marker includes three areas: central circle, color segments and outside area.Points from all these areas are used in recognition process.Central circle and outside area are used for premature recognition; by default hose two areas are white.Information about environment is coded using color segments and different color combinations.Thick black centers border enhances marker detection (Fig. 3).Marker detection overall is round object detection.As mentioned before logical separation is suitable for such task.Color image RGB is first converted to grayscale image, such conversion is performed using weighted sum of R, G and B color components.Grayscale image contains pixels that are defined using lightness numeric value (0÷255).According lightness value logical image is transformed, were each pixel is compared to pre-defined threshold value.Each pixel gets new value, according to the comparison result it can be 0 or 1.
Marker detection and recognition algorithm 1 is shown bellow.Before processing logical image, it has to be labeled; it means that all white areas get a numeric label (a 1 ,…,a i ) that helps to calculate their parameters one by one.Center coordinates (), equivalent diameter () and perimeter () are calculated so that all regions can be tested simultaneously.

Algorithm 1: Marker detection and recognition step 1: Logical labeling from the indexed image (bw) regions = (a 1 ,…, a i )  bw, i  Z step 2: The extraction of features of labeled white regions (perimeter(), diameter(), center coordinates () )
 i , i , i   a i ,  i = (xc i , yc i ); step 3: The diameter of each region is checked using  and  limits  Diameter is checked in order to reject noise small areas as well as those sectors that are too big using limits [, ].All areas that passed diameter evaluation (ind1) are then verified to be round objects.The calculation of the proportion in step 3 is used which allow to determine which objects are round.Afterwards the number of round objects is used to determine whether used grayness threshold was of the right value.Marker is recognized using nine pixels color components (Fig. 4).Measured color values are compared with template color values using Euclidian measure.The opportunity to control the location of mobile furniture using only voice command or eye gaze information is desirable for person with disabilities.Untraditional decisions improve efficiency and makes design simpler.Although, low price and simple design is obtained, system won't work properly if detection and recognition doest' succeed.

Experimental investigation of algorithm
The recognition accuracy of the certain room and accurate localization directly depends on the recognition quality of the markers.The markers are placed in the environment were light is uncontrolled.Therefore, the proposed marker recognition algorithm must be evaluated in different lightening conditions.Experiments took place in order to investigate potentiality of detection and recognition algorithm.Processing rate and accuracy tests were performed.The recognition algorithm was tested in three different lighting settings (24 lx, 125 lx and 180 lx) and using different contrast values.Contrast which varies from 0 to 255, is manually set in camera.During experiments the recognition accuracy and processing rate were calculated (Fig. 6).Experimental results show, that the recognition algorithm is able to classify the markers correctly in a room with good lightening (180 lx) and even in a relatively dark room (24 lx).Best recognition accuracy (about 98%) was reached using a contrast level of 38 and a lightening of 125 lx.The processing rate is the main characteristic for all real time systems.The algorithm is able to process 20 video frames per second, therefore, the system can be used for real time applications.

Conclusions and future works
Proposed idea of localization of the mobile smart furniture combines known technologies of optical tracking and standard range sensors.Such system improves the living quality of persons (especially the disabled), because, it allows driver-less control.Intermediate results show that the recognition algorithm was fast and accurate in satisfying environmental conditions.The performance of the algorithm can be improved by changing the lightening conditions.
Even though the experiments show that the system could suitable for real time applications, more tests should be taken.The system is basically based on image processing so the main task is to provide high quality images for the recognition algorithm.Uncontrolled lightness is the main difficulty; therefore, future work will involve testing the utility of using additional light.Also future work will involve the creation of algorithm that recognize the room, based on the objects itself -i.e., without the special color markers used today.The detailed review of state of the art techniques and algorithms is presented in this work, aiming to create the autonomous mobile platform for mobility improvement of disabled persons.The video camera, sixth infrared range sensors and the motorized furniture were used in our mobile system.Presented system automatically localizes the position of the "smart" furniture based on certain color markers.The mobile platform is a part of a bigger associative system which is being developed during the project ROBOSOFA.Therefore, the main focus of this paper is on the self localization problem from visual information, i.e., the images of colored markers.The recognition task of the certain room is limited by the signals from range sensors and the recognition of the certain markers, because each room is coded with different color markers.Intermediate experimental results show, that the proposed algorithm can be applied in real time applications and it is able to process 20 frames per second.Ill.6, bibl.20 (in English; abstracts in English and Lithuanian).Darbe pristatoma detali techninių sprendimų ir algoritmų apžvalga, leidžianti sukurti autonominę mobiliąją platformą, tinkamą neįgaliesiems.Sistemos pagrindą sudaro vaizdo kamera, šeši infraraudonųjų spindulių atstumo jutikliai ir judusis baldas.Pristatomoji sistema autonomiškai nustato "išmaniojo" baldo buvimo vietą pagal specialiai suformuotus spalvinius žymeklius.Autonominė mobilioji platforma yra dalis didesnės asociatyviosios sistemos, kuriamos vykdant projektą "Robosofa".Todėl šiame darbe daugiausia dėmesio skiriama padėties nustatymui (lokalizacijai) naudojant vaizdinę informaciją, t. y. kambarių specialiųjų žymeklių atvaizdus.Kiekvienas kambarys pažymimas tam tikru spalviniu žymekliu, todėl lokalizacija yra apribojama žymeklių atpažinimu ir atstumų iki sienų nustatymu naudojant jutiklius.Pirminiai eksperimentiniai tyrimai parodė, kad taikomas algoritmas yra tinkamas realaus laiko aplikacijoje ir juo galima apdoroti iki 20 kadrų per sekundę.Il. 6, bibl.20 (anglų kalba; santraukos anglų ir lietuvių k.).

Fig. 1 .
Fig. 1.Examples of the markers used in related works

step 4 :
Evaluation of roundness of white region using  and  limits ind1

step 5 : 6 : 7 :
Adaptive threshold formation and evaluation thresh = thresh-st, if m > , thresh = thresh+st, if m > , ,   Z; step Recognition of the room by color codes from the colored marker image () color_code = R c ,G c ,B c     (xc, yc), Room = min kK (color_code,color_tamplate), K -number of markers; step Jump to the step 1.

Fig. 4 .
Fig. 4. The nine pixels which are used for marker recognitionThe configuration of intelligent environment recognition system is shown in the Fig.5.System consists of: 1. Computer, on which all 2D and 1D data is processed.The detection, recognition and motion control is done there; 2. Color web-cam, which is used for acquisition of environment image with markers; 3. Motion controller, to control electrical drives of the mobile furniture; 4. 6 IR range sensors which are used for obstacle avoidance.

Fig. 5 .
Fig. 5.The nine pixels which are used for marker recognition

Fig. 6 .
Fig. 6.The functional relationship between contras value, lightening conditions and recognition accuracy