Orientation Invariant Surface Classification Using Uncertainty Level Estimation

,


I. INTRODUCTION
Automatic industrial sorting systems are widely spread and used for many classification tasks.Such systems are now common in food processing [1], [2], surface defect detection [3], [4], garbage recycling [5], letter sorting [6], etc.Many of them are based on the object's surface optical properties but use a different kind of sensors such as CCD cameras, spectroscopy [7], stereo vision, infrared light, and others.Optical sensors can capture colour, shape, texture, and other optical features; in many cases, it is a multiclass [8], [9] identification problem.Optical properties depend on lighting conditions; therefore, isolating objects from environment and implementing artificial lighting sources may be one of the most important key points of the system to work properly.Such systems usually have strict requirements for their performance.They should be able to process requested number of objects fast enough and as accurate as possible.
In wiring industry, before assembling any type of electrical wire connectors, they should be first marked physically.For this task, a conveyor and ink-jet printer are Manuscript received January 9, 2013; accepted June 8, 2013.used to label small batches of connectors.Each batch is composed only of the same type of connectors.This process has not been automated yet and performed manually.
It is impossible to use any orienting devices with vibratory-bowl feeders, because the number of connectors in each batch is small and each type of connector varies in size, geometry, etc.The system with robot arm and computer vision may be the solution for automation; connectors then could be reoriented fast without human interaction.Such a system should have the ability to identify the side and orientation angle of the connector viewed by a video camera.Then, a robot arm could undertake the pick-and-place task of correct positioning of a connector for ink-jet printing.The presented approach focuses on connector side identification.
To the best of our knowledge, there is no way to estimate the level of uncertainty in classification precisely.The performance of the classifier depends on data; if samples from different classes correlate, misclassification errors may occur.Although there are some attempts [10], [11] to estimate uncertainly level but those solutions still do not give 100 percent accuracy.Our approach proposed is based on idea that some sides of connectors are easily classified correctly by visual properties such as surface area or mean of greyscale intensity values.For the rest sides, we used one of the popular orientation and viewpoint invariant method -Scale-invariant feature transform (SIFT) [12].
The paper contains five main sections.In the second Section, theoretical background is overviewed.The third Section gives more information about used data and the experiments.Experimental results are presented in Section IV.Finally, conclusions are given in Section V.

II. THEORETICAL BACKGROUND
The approach suggested (Fig. 1) is based on assumption that some sides of connectors can be classified with zero misclassification error (MCE) using just a few first order statistical (FOS) features such as mean or standard deviation calculated from greyscale intensity values.FOS features are orientation invariant and fast to compute; however, they are estimated on individual pixel values, ignoring the spatial interaction between image pixels.Decision Tree (DT) was chosen as a classifier, because experiments show that this method was the fastest and still accurate enough.Visually some sides of the same connector are like mirrored version of the same image or differ by tiny details only; therefore, FOS features are not sufficient in distinguishing these sides.In such cases, the advantage of global spatial information obtained by SIFT method is additionally applied for feature extraction.
For each connector, a subset of different features was selected by SFS (Sequential Feature Selection) [13] dimensionality reduction method.Fewer features still give the same accuracy results, but the model is smaller and classification is faster.The most useful FOS features in experiments were surface area, mean, standard deviation, harmonic mean, median, central moments, kurtosis, skewness, mode, and percentiles extracted from the greyscale images.
For the FOS features classification, four different classifiers were compared: ANN (Artificial Neural Network), DT (Decision Tree), KNN (K Nearest Neighbours), and QDA (Quadratic Discriminant Analysis).Accuracy of all the classifiers is around the same, but DT shows the best performance.MCE is evaluated using holdout method.
To make it easier to reproduce our result, the pseudo code is given.The flow of the algorithm is shown in Fig. 1 and functions are given bellow.vl_sift() and vl_ubcmatch() are functions provided by VLFeat source library for Matlab.The first one extracts SIFT features and descriptors; the second matches SIFT descriptors of two images.The actual code was in Matlab; therefore, pseudo code is similar to Matlab syntax and has the same or similar function names.
The initialization procedure starts all preparation needed for calculations such as: defining necessary constants; for featureType in reducedTypes // create a vector 3 X = X U {getFeature(featureType,idx)} 4 return classify X vector using DT The last piece of pseudo code finds samples from training set with similar angle but only in the case if C from DT belongs to the list j.Quantity of matches is used for voting.By using coarse Hough space [16], we find similar SIFT features by orientation, scale, and location.Then, only features from the largest bin of Hough space are selected.This allows excluding many incorrect matches and is essential in the solution proposed.

III. DATA
The core idea of experimental setup is to train the system and to use industrial robot arms in manufacturing process.Two robot-arms (see Fig. 2) were used; however, it's enough to have one, because the second robot can be easily replaced by a simple drive in training phase.By using robots, we had advantages such as automatic sample labelling, fast database preparation, and the ability to gather a decent number of variations for any connector type.This setup provides the ability to prepare the system for a new connector type extremely fast.The first robot follows a spiral scanning path and the second one rotates a plate with a connector on it.The data acquiring algorithm runs 3 minutes at 10 fps frame rate.After excluding repetitive samples, we got around 1500 unique images per each side of every connector.Connectors have 4 to 6 sides; therefore, a database for each connector has 6000 to 9000 samples.
Eight connector types were chosen for experiments (Fig. 3).They differ by shape, number of sides, size, colour, and texture.During training, arms of both robots move along the pre-defined path until a significant number of variations are acquired.The white background of each acquired image had some gradient noise which was removed using edge detection and morphological operations.Then each non-white segment in images was recognized as an area with connector.

A. Classification Using First Order Statistical Features
Training sets for each connector contain 20 percent of random picked samples and the rest 80 percent is used to calculate MCE.This 20/80 ratio threshold for holdout method was found empirically by comparing accuracies for all connectors.Table I provides the number of sides for each connector and number of features selected up by the SFS method.Four classifiers were compared by their MCE and performance (Table II).Results show that all the classifiers give around the same accuracies, but the fastest classifier (DT) was chosen for further experiments.On average the one image computation times are: QDA -0.0069 s, KNN -0.0215 s, ANN -0.0221 s, and DT -0.0046 s.The overall performance for a connector mostly depends on connector's size and number of features extracted; it's enough to use 2 to 7 features to get high enough accuracy.All experiments were performed on an Intel Pentium 4 3 GHz processor with 2 GB RAM.Whatever classifier was used, some sides had zero MCE.The remaining of the sides had some misclassified cases but with a strict pattern (Table III).For example, front and back sides of C1 connector are very similar as well as their left and right sides; therefore, FOS features are not enough to distinguish them.Other sides for this connector are well identified using only a few FOS features by any classifier.4 shows a relation used for voting.Orientation is expressed as an angle between 0 and 90 degrees.It is measured between the x-axis and the major axis of the ellipse that has the same second-moments as the region.All training and testing data have been used for the charts.As it is seen from Fig. 4, if the number of matches is computed on pair of images from the same class, the number of matching points is much bigger comparing to number of matches obtained for the images from different classes.As well as it is obvious to note that the smaller angle of displacement is present between pair of images, the bigger number of matches could be found within the same class data pairs.This relation is valid for all connectors used in experiments; we use it for voting in classification.SIFT may be more sensitive to viewpoint changes than to orientation changes, and number of matches may be lower if the viewpoint is significantly different.Sometimes the proposed system gets into a confusing situation by counting equal numbers of matched features for different classes of connectors.In order to reduce MCE the minimal threshold of matched features is used.
So, it is possible to use the matches' ratio threshold and if the ratio is low, the image is left unrecognized.A high ratio means high reliability, while a low ratio means uncertainty or even misclassification.But it is better to leave the image unrecognized than to identify it incorrectly.Table IV shows connector recognition results obtained with the applied thresholding technique.For all connectors the MCE is equal to zero but because of the applied threshold some acquired images are left unrecognized.V. CONCLUSIONS A novel approach for electrical wire connector sides' classification with uncertainty level estimation is introduced.The uncertainty level estimation proposed allows identifying connector sides with 100 % accuracy, even if they have very similar visual appearance.The proposed system is designed for robot-arm pick and place autonomous operation.The first order statistical features were extracted by SFS method from scale images by the video camera.Four different classifiers were experimentally compared for connectors' data analysis: ANN, DT, KNN, and QDA.The decision tree classifier has shown the best performance.
In order to reduce the MCE, the thresholding technique were introduces on matched points found by SIFT method.Experimental investigation have shown that the number of unrecognized samples slightly increased but overall system performance has lead to zero misclassification error.
In the presented work we have also shown that the samples of connectors' narrow sides have a high unrecognising rate, but this result is not critical in real application, because the connector most likely will land on the bigger flat side in front of the camera as well as the acquired instance will be rejected if the quantity of matches will be lower than the predefined threshold.

Fig. 1 .
Fig. 1.The abstraction of proposed algorithm using pseudo code fragments.

Fig. 3 .
Fig. 3. Connector types used in experiments; back sides are shown.

Fig. 4 .
Fig.4.Two samples of the first connector type were picked, one as a front side example and another as a back side example.Each sample was matched by SIFT features to all samples of the same side (front to front; back to back) and to all samples of very similar another side (front o back; back to front).The quantity of matches for each pair was computed.

TABLE I .
NUMBER OF SIDES AND FEATURES FOR EACH CONNECTOR.

TABLE II .
CLASSIFIER COMPARISON BY MCE.

TABLE III .
EASY AND HARD SIDES FOR RECOGNITION.
B. Classification by SIFTFigure

TABLE IV .
FINAL ACCURACY RESULTS.