TMicroscope: Behavior Perception Based on the Slightest RFID Tag Motion

1 Abstract —Behavior perception with fine granularity can offer much more valuable information, e.g. which and why products people are often interested in will not be purchased in shopping. In this paper, we propose a RFID-based method to perceive slight motion of a target product with RFID tag for mining customer’s behaviors. However, the interference from human movements in the reading zone is a challenge to result in false positives. To address this problem, we first compute virtual displacements of the target RFID tag and then adopt Support Vector Machines to recognize absence or presence of human motion near the tag. Next, we divide the surveillance region into mm-level grids to construct similarity matrixes of the tracked tag before and after human motion

correlate electronic product code (EPC) of each RFID tag with RFID reader ID in the database, which means the tag belongs to the corresponding reader's monitoring area.The RFID reader continually inventories all of tags in the reading zone.Once the reader fails to capture one of RFID tags in an inventory period, the system will determine that the object corresponding to the RFID tag' EPC has been missed or moved by persons [1]- [4].Yet, there are many problems in this way of identifying absence or presence of RFID tags in the monitoring area.(1) Low positioning resolution.The reading range of most of commercial off-the-shelf (COTS) RFID reader is generally within 3~10 meters.Once a reader reads the RFID tag belonging to its predefined tag set in the database, the system will consider that the tag exists in the monitoring area.But it can't meet application requirements with high positioning precision.(2) False negative reads.False negative [5] reads means that a reader fails to read a tag in the surveillance zone because the propagation path from the tag to the reader is partially obstructed by human or some other metal objects, resulting in false positives.(3) Large delay.The large number of RFID tags in the large reading area will prolong the total time that RFID reader inventories all of RFID tags each time, thereby increasing computation delay of the tracking system.(4) High cost and read collision.To improve the real-time performance, the system needs to deploy large amount of RFID readers in the area with high costs.Also, it will also increase the occurrence probability of RFID reader collisions [6].
To address above problems and improve the perception sensitivity to tag movements, we design a method to perceive the slight RFID tag motion.The proposed approach in this paper can be used to monitor valuables and mine customer shopping behaviors for the scenarios where any movement of the objects needs to be accurately captured.For example, when a person is interested in a product on the shelf in shopping, he/she will take the object from the shelf and examine carefully the product information, such as origin, composition, production date and shelf life.If the product meets requirement, he/she will add it to shopping cart.Otherwise, the product will be returned back to the shelf.Retailers can only know what products a customer has bought from the final transaction data, but fail to understand the reason why the customer gives up buying products that he/she is interested in.The motion of the RFID tags attached on the products implies the rich customers' behavior information, so accurate detection of tag motion is of essential importance to help retails further understand consuming habits.
Currently, most of existing methods employ absence or presence fashion to identify RFID tag movements.In [7], the authors present that a large RFID system needs to consume much execution time to frequently read a large number of RFID tags in the monitoring area for avoiding interference from environment changes, and then they introduce a missing-tag identification protocols to reduce the execution time for real-time performance.The method can achieve the detection accuracy of more than 88.9 %.Chiu C. Tan et al. [8] propose two trusted and untrusted reader protocol to accurately monitor a set of tags without collecting IDs from them.Similarly to the above method, Rui Zhang et al. [9] present three novel protocols to reduce execution time for fast identification of the missing tags and indeed can also tell which tags are missing.The main idea of these methods is to collect a short response signal from the RFID tag instead of the integrated EPC for reducing execution time.Although the objects attached with RFID tags exist in the detection range, these systems can't understand whether the tracking tags move even at meter level.To overcome the problem, Lei Yang et al. [10] observe that the signal strength is hypersensitive to tag movement.Then they build the mixture of Gaussian model based on RSSI to perceive the slight tag motion using computer vision technology.There are two drawbacks in this method.The first one is that the measured RSSI parameter reported by COTS RFID reader is often affected by multipath propagation and impedance mismatch and polarization mismatch of RFID antenna, which will be less reliable than another parameter-Phase.The phase values with higher resolution measured by COTS RFID reader are employed in our method.Second, if a person moves continually near the target tag, the changes of RSSI in this case are similar to that influenced by the tag motion.So the system can't distinguish these two scenarios.TagBooth [11] is an effective method based on RFID technology to acquire fine-grained deep shopping data using RSSI and phase features with people moving around and fetching tagged goods, which can recognize customer actions of picking and toggling the clothes hanger.The problem similar to Lei Yang' can't extract different features of the RSSI and phase caused by tag motion and continuous movements of human near the target tag.However, our method could address the complex problem.
In this paper, we propose a method, called TMicroscope, to perceive human behavior based on the slight motion of objects attached with RFID tags.At first, we explore the changes of radio frequency (RF) phase under different scenarios of no interference, human interference and tag motion.Next based on experiment observations, we divide the real scenarios into four categories, no motion, human motion outside the predefined surveillance area, human motion inside the surveillance area and tag motion.We consider the first two cases as the absence of tag motion and regard the latter two cases as the presence of tag movement.Then we build scenario classification model to recognize scenarios using Support Vector Machines (SVM).For the latter two cases, we also formulate tag motion recognition model to distinguish whether the tag has been moved by human in the surveillance area.Finally, we build a prototype to evaluate our method.The results show that our method can achieve the average scenario classification accuracy of 96.3 % and the average slight tag motion perception accuracy of 100 % with the perception sensitivity of 1 mm.
The rest of the paper is organized as follows.The empirical study is described in Section II.Section III introduces how to classify the absence or presence of human motion inside the predefined surveillance area or tag motion.The next Section IV describes the method to distinguish whether the tag occurs to move.The experiment is evaluated in Section V. Finally, we conclude our work.

II. EMPIRICAL STUDY
Due to the phase with high resolution and hypersensitivity to changes of surrounding environment [12], we adopt RF phase to detect the slight tag motion for customer behavior perception.So we conduct empirical studies to better understand how human and tag motion take impact on the measured phase reported by RFID reader.Figure 1 shows the experiment setup, including Impinj R420 RFID reader, 8 dBi antenna and Impinj H47 RFID tag.

A. Impact of Slight Displacement for a RFID Tag
At first, a RFID tag on a plastic bracket is placed at 0.65 m away from the RFID reader antenna.The reader is configured to max throughput model to collect phase values within 5 seconds as much as possible.We conduct experiments 5 times and the displacement each time is 1 cm.The average measured results are 0.4 radians, 0.13 radians, 6.0 radians, 5.7 radians and 5.5 radians respectively, shown in Fig. 2. Since phase is a periodic function with period 2 radians, the phase values will clearly repeat at distances separated by integer multiples of one-half the carrier wavelength [13].From Fig. 2, the RF phase is hypersensitive to the distance change away from the antenna.Even though the tag displacement is only 1 cm, the phase can reflect the tiny change in location.

B. Impact of Human Motion
In this experiment, we collect the reported phase within 10s to study on the impact of human motion.The person keeps in stationary state in the first 2 s and then moves from one side of RFID tag to another in 3 s-10 s.The change of phase is shown in Fig. 3. From the figure, when the person is at rest, the phase values are very stable.However, when the person moves closer to the tag, the measured phase will changes sharply.Especially, when the direct propagation path from the antenna to RFID tag is completely obstructed by the moving person, the amplitude of phase wave is the largest.The tag-centered region with the width of about 2 m shown in Fig. 1 is called strong interference zone where the phase values change more greatly than others.From the Fig. 2, the human motion can significantly affect the reported RF phase.

C. Impact of Tag Motion
In this experiment, we mainly focus on the impact of tag motion.Similarly to above experiment, in the first 2 seconds the RFID tag is in stationary state and then we move the tag randomly within the pallet.The collected phase values, shown in Fig. 4, are very stable under stationary state whereas the phase values change seriously in the range of   0, 2 under tag movements.The larger the tag displacement is, the greater the change of phase wave is.From above experiments, phase values will also be affected by human and tag motion.The change of phase caused by human movements is similar to that caused by tag motion in the small range.

III. THE CONCEPT OF PHASE-AMBIGUITY
At first, we assume that the total distance traversed by the RFD signal from the transmitting antenna to the RFID tag and then back to the receiving antenna is 2R.And considering the impact of the reader's transmit circuits T  , the tag's reflection characteristic Tag  , and the reader's receiver circuits R  on the reported phase, the total phase value   [8] can be represented as where  is the wavelength, i.e., 32cm   .However, the measured phase is a periodic function and need to be estimated modulo 2 , i.e.
 mod 2 , where  is the reported phase by COTS RFID reader.In our experiment, the RF phase contains random noise from device interference, following a Gaussian distribution with a standard deviation of less than 0.05 radians under no human motion.
Given   , we introduce the phase-ambiguity N , then we where the values of N is . The unknown parameter, phase-ambiguity N, can't be reported by COTS RFID reader.
In the tag movement, the phase-ambiguity N will change with the distance between the antenna and RFID.Assume that the time between successive inventories of the same tag are respectively i t and 1 i t  .The total phases of tag at the time i t and The difference of i t and 1 i t  will depend on reader mode, tag population size, and environmental conditions (e.g.interference levels).In our experiment, RFID reader is configured to the Gen2 mode of Max Throughput.Under the number of 20 RFID tags in the reading area, the time interval between the two inventories is about 1 0.03s 32  .In the short interval, we have So we obtain In the next section, we will build the scenario classification model based on (5).

IV. SCENARIO CLASSIFICATION
Thermal noise from the reader is always present but other noise sources, such as human motion interference, can also affect the reported RF phase.Instead of viewing the RFID tag in stationary state, we view that the tag moves in the reading zone to result in the changes of RF phase.In other words, if there is no human interference near the tag, the standard deviation of phase is very small, meaning that the RFID tag moves slightly in the small range.And if a person moves close to the target tag or the tag is moved by the person, the phase will change greatly and it is similar to tag movements in a large region.We formulate the model to compute the displacement of 'moving' tag based on above assumption, thereby exploiting Support Vector Machines (SVM) to classify the absence or presence of moving person near the target tag.
As described before, RF phase is hypersensitive to the change of surrounding.When the person moves very close to the target tag and the direct propagation path between the tag and antenna is obstructed by the person, the standard deviation of phase is larger than the cases of no human interference or human motion far away from the target tag.Thus based on our experiment, we define that the surveillance area with strong interference is a tag-centered square of 2 2 m m  . When persons move outside the surveillance area, the of human motion interference on the measured can be ignored.
. Also, we assume that the distance from the k th antenna to the tag at the (i-1) th and i th reading cycles are respectively However, since the RFID reader adopts TDMA fashion to successfully inventory each of RFID tags to avoid tag collisions [14], it is impossible for all of antennas to read the same tag at the same time.Obviously, the displacements measured by different antennas between successive reading cycles can't be exactly the same.But the time interval between successive inventories of the same tag read by different antennas in a reading cycle is extremely short, so we assume that these antennas collect phase values at the same time in the reading cycle.In other words, the tag displacements from the (i-1) th to i th reading cycle are the same, i.e., And we also assume that the measured phase k ) .
Approximate a two variables Taylor series expansion at the point  and ignore the terms up to second order yields, then we have where For Thus We know that most of COTS RFID readers are often equipped with 4 RFID antenna interfaces.Considering the cost savings, we only use two antennas to collect phase values in the reading zone.So the expression between the coordinates of the tag and phase values collected by antennas 1 and 2 in the (i-1) th and i th reading cycle is formulated as follows:  From Fig. 6, a large part of points overlap around the origin, leading to false positives.So we divide the original reading stream into read frames and exploit data framing method to improve the accuracy of scenario classification.
Assume that the read frame contains the number of L coordinates, i.e.,       .
Thus the coordinate   , Similarly to video frames, each frame is actually a static picture.People will see object movements by playing a series of frames continuously.If we assume that the frame length L = 30 and the frame shift S = 15, the snapshots are shown in Fig. 7. From Fig. 7, there are obvious differences between the before and after two scenarios.Finally, we adopt the SVM based on preliminary measurements collected from several scenarios to classify the absence or presence of moving human in the surveillance region.The LIBSVM [15], a SVM tool, is used easily for classification in our experiment.

V. TAG MOTION PERCEPTION
In the above section, the system can identify the absence or presence of moving persons in the surveillance area, but we could not understand whether the person moves the target RFID tag or not.So in the following, we will discuss how to perceive the absence or presence of tag motion.In shopping, when a customer takes a product from the shelf and then returns it back, the position of the product in the shelf will change a little compared with the original position.However, when the customer only passes through the surveillance area, the product position will never change.So if we can perceive the slight change of tag position before and after human movements in the surveillance area, the products moved will be captured by the system.
At first, we divide the surveillance region into M N  square grids at mm-level (less than 1 cm) and assume that there are visual RFID tags with the same tag characteristics as the real tag on the center of each grid.If the grid width is a , the distance from the center of grid to four vertexes , the real tag maybe exist at the grid.So the probability that the real RFID tag is located within the grid W,L G is denoted by where L represents the frame length, i.e., the number of times that a RFID tag is read by the antenna A k .

 
We know that the reading range of a COTS reader's antenna is generally around 10m, so the large surveillance region will increase the number of grids at mm-level in the region.If we compute the phase difference between the real and visual tag at each of grids, it will jeopardize the real-time tracking capability.Since the phase values will clearly repeat at distances separated by integer multiples of one-half the carrier wavelength, we can assume that the target tag only moves with the range of one-half carrier wavelength.
From Fig. 9, we assume that the tag motion perception region is 20 20 cm cm  with the 0.5 cm width of each grid.And the coordinate of RFID reader antenna in the coordinate system is (10 , 0 ) cm cm .We compute the probabilities of each grid, which is represented by a matrix Z, between before and after tag motion with the displacement of 1 cm. Figure 10 and Fig. 11 show respectively a contour plot of matrix Z, where Z is interpreted as heights with respect to the x-y plane and these two figures can show the difference between before and after tag movement.Due to evident differences between Fig. 10 and Fig. 11, we can compute the similarity between them to determine the absence or presence of tag motion.
Here, we use matrix correlation coefficient to compute the similarity , between the matrix X and Y corresponding to figures.A coefficient of '1' represents a perfect similarity whereas '−1' indicates a total disagreement.In our experiment, the similarity between before and after tag motion with the displacement of 1 cm is closer to 0 and the similarity under no tag motion is closer to 1.So we can adopt the determination threshold method to identify tag motion.If    Note that if the distance of tag motion is just equal to integer multiples of one-half the carrier wavelength, the , X Y  will be closer to 1.But it is a low probability occurrence and can be ignored in practice.

VI. EVALUATION
We present the implementation of TMicroscope system and evaluate the TMicroscope from the aspects of scenario classification accuracy, tag motion perception accuracy, frame length, grid width and accuracy comparison.The evaluation metrics include scenario classification accuracy and matrix correlation coefficient in this section.The former describes the percentage that the scenarios are correctly identified by TMicroscope system.The latter is to determine whether the target tag has been moved by human or not.The larger the matrix correlation coefficient is, the more similar they are.

A. Implementation
We build a prototype of TMicroscope system using the Impinj R420 RFID reader, eight Impinj H47 RFID tags and two 8 dBi RFID antennas.Figure 12 shows the main setup of TMicroscope.The distance between two antennas is 1 m.The size of surveillance near the target tag is 2 2 m m  .We use the Impinj R420 RFID reader without any hardware modification to collect measured phase data and we also depend on LTK Programmers Guide [16] to write RFID applications communicating with the R420 RFID reader in C#.The RFID reader is compatible with EPC Gen2 standard [17].The Gen2 mode of high performance feature in R420 reader, Max Throughput, is configured in our experiment.2) are labeled as no human interference inside the surveillance area whereas the scenarios (3) and ( 4) labeled as human motion inside the surveillance area.We invite 8 human with different heights to participate in the experiment.We collect phase values from the eight humans under different scenarios within 20 minutes.The first 15 minutes of data is used to train SVM model and then the last 5 minutes of data is to test the scenario classification accuracy of our TMicroscope system.The frame length is 30 and the frame shift is 15.The results are shown in Fig. 13.

B. Scenario Classification Accuracy
From Fig. 13, our method under scenarios (1), ( 2) and ( 4) can achieve the classification accuracy of as high as 100 %.However, the accuracy drops a little under the scenario (3) with the mean of 92.4 % and the standard deviation of 11.6 %, because if the person keeps in stationary state in a long period, the phase collected by RFID reader will be stable and then snapshots will also be close to the origin coordinate similar to scenarios (1) and ( 2), resulting in false positives for SVM.

C. Tag Motion Perception Sensitivity
In this experiment, the target RFID tag is moved 20 times within the range of 2 cm away from the original position.The displacement is 1 mm each time.At different positions, a RFID reader collects phase values within 10 s.For the frame length L = 30 and the frame shift S = 15, we respectively compare the probability matrix of tag location at position 1 with that from position 2 to 20, i.e.,   Tag motion perception sensitivity is shown in Fig. 14.As the tag displacement increases, the matrix correlation coefficient decreases.When the tag is moved 1 mm away from the original position, the mean of matrix correlation coefficient is 0.9, and when the displacement is more than 6 mm, the mean drops to less than 0.8.So TMicroscope has the high tag motion perception sensitivity even though the displacement of RFID [18], [19] tag is only 1 mm.In practice, the determination threshold is set to 0.95

D. Impact of Frame Length and Grid Width
In the experiment, we evaluate the impact of frame length and grid width on TMicroscope.
(1) Frame length.In Fig. 15 and Fig. 16, the large frame length will increase scenario classification accuracy and correlation coefficient whereas the perception time will be prolonged.The appropriate frame length should be adopted for different requirement in practice.In addition, in scenario classification, if the frame length 28 L  , the classification accuracy of scenarios (1), ( 2) and ( 4) is as high as 100 %, and the accuracy of scenario (3) is more than 92.3 %.In the tag motion perception, the results of matrix similarity comparison increase slightly from 0.78 to 0.79.As a result, the impact of frame length on scenario classification is more obvious than that on tag motion perception.(2) Grid width.For the 8 mm displacement of tag motion away from the original position, the larger grid width will increase the average matrix correlation coefficient in Fig. 17.If the grid width is 3 cm, the similarity result is 0.987, resulting in more false positives.Therefore, with the increase of grid width, the accuracy of tag motion perception will drop.In practice, the grid width of less than 0.5 cm is introduced in TMicroscope system.

E. Accuracy Comparison
We compare TMicroscope with other two tag motion perception methods achieved by Frogeye and TagBooth.The comparison results are shown in Fig. 18.Frogeye [10] and TagBooth [11] can achieve the motion perception accuracy of 92.3 % and 89.2 % respectively.All of these tag motion perception method only consider the scenario of persons walking nearby the target RFID tags and ignore continuous human activities in the surveillance area between the reader and target tag.However, in our method it can achieve the average scenario classification accuracy of 96.3 % and the average slight tag motion perception accuracy 100 % with the perception sensitivity of 1 mm.Our proposed method works in two steps.At first, we observe from the experiment that the phase drift caused by human activities interference nearby the target tag is hard to differentiate from that caused by tag motion.Unlike Frogeye and TagBooth, our method isn't to eliminate the human interference as much as possible rather than to group the two cases into one scenario of tag movements.We only divide the scenarios into the absence or presence of human activities in the given surveillance nearby target tags, addressed by scenario classification method in TMicroscope.Second, once TMicroscope the presence of human activities, tag motion perception method of TMicroscope can further recognize whether the target tag has been moved by human or not.In summary, because the phase is more sensitive than RSSI, and we first distinguish the absence or presence of human activities and then perceive tag motion, TMicroscope can achieve higher perception accuracy than Frogeye and TagBooth.

VII. CONCLUSIONS
Behavior perception is becoming a hot research field in recent years.Our proposed method based on RFID technology can be used to mine customer shopping data for physical stores.Compared with existing methods, the innovation for our method is to regard the scenarios of human interference inside the surveillance and tag motion as the same one, and then to distinguish the absence or presence of tag motion after human leaves the surveillance.This approach can effectively improve the slight tag motion perception accuracy.In the experiment, we have verified the performance of TMicroscope system.In the future, we will detect complex customer activities.

Fig. 5 .
Fig. 5. Tag motion model.As shown in Fig. 5, we assume that there are   2 m m  fixed antennas deployed to read RFID tags.The coordinate of the k th antenna is ( , ) k k A A X Y in the rectangular plane coordinate system.The tag is read for the i th time by the k th horizontal   and vertical displacements of the tag measured by k th antenna between the (i-1) th and the i th reading cycle are respectively cycles.When there is no human interference in the reading zone or human motion only outside the surveillance region defined before, the values of Tx in Fig.6with points around the origin of coordinates.And when human moves inside the surveillance area or the target tag is moved by the person, the points are far away from the origin of coordinates, shown in Fig.6.

coordinate is 2 2 a. 8 .
and the distances from the grid centroid and the position of real tag to the antenna A k are respectively , Ignoring the phase-ambiguity N, the measured phase collected by A k is Tag  and the phase of visual tag at the center of grid , G W L is described as ,


, the tag isn't moved by the person and if , X Y    , the tag has been moved by the person.

Fig. 13 .
Fig. 13.Scenario classification accuracy.At first, we collect training data from different scenarios for SVM and then verify classification accuracy of SVM by inputting test data.The four scenarios include: (1) No human interference in the reading zone; (2) Human movements outside the predefined surveillance area; (3) Human movements inside the predefined surveillance area; (4) Tag motion.The scenarios (1) and (2) are labeled as no human interference inside the surveillance area whereas the scenarios (3) and (4) labeled as human motion inside the surveillance area.We invite 8 human with different heights to participate in the experiment.We collect phase values from
tag is moved by the person.

Fig. 16 .
Fig. 16.Impact of frame length on tag motion perception.

Fig. 17 .
Fig. 17.Impact of grid width on tag motion perception.