Subjective and Objective MOS Evaluation of User ’ s Perceived Quality Assessment for IPTV Service : a Study of the Experimental Investigations

The digital television providing over the networks, based on the Internet Protocol, gives the exceptional opportunities to the service provider for its broadcasting. However, all Internet Protocol Television broadcasting solutions are directly related with the user’s perception of the service quality. This perception depends on the visible or audible factors, which are observed during the channel change process of Internet Protocol Television. The channel change process of Internet Protocol Television is assessed by its duration. Due to this, it is important to analyze the relation between the channel change process and the user’s perception of the quality of Internet Protocol Television. The experimental investigations for the subjective and objective Mean Opinion Score evaluation of the user’s perceived quality assessment for Internet Protocol Television according to the channel change process are presented. DOI: http://dx.doi.org/10.5755/j01.eee.19.7.5178


I. INTRODUCTION
The digital IP television (IPTV) broadcast is unique due to the fact that requests for the television (TV) channel change are sent to the IPTV HeadEnd from where the IPTV service user requested TV channel broadcast starts.Such a two-way service delivery, when user only receives his requested TV channel, not all available TV channels simultaneously, impacts on TV channel change process and its duration.ITU -T FG IPTV recommendation [1] defines the process of TV channel change from one TV channel to the other as a TV channel zapping.ITU -T G.1030 [2] recommends that digital IP television channel zapping time should not exceed the limit of 2 seconds.However, IPTV service providers record channel change process in several times longer.The longest duration of channel change process is influenced by the main components of the TV channel zapping time and their influencing factors.In this case, during TV channel change process, the user sees dark TV screen longer than he was used to see in the other type of TV service.IPTV service quality could not be Manuscript received December 29, 2012; accepted May 8, 2013.separated from the user's perception.In fact, the user is the main quality estimator.The user perceives quality as the integrated concept, rating not only the technological aspects.IPTV Quality of Experience (QoE) is affected by the TV channel video/audio quality and their synchronization parameters that are rated according to the Quality of Service (QoS) indexes.IPTV QoE parameters and QoS indexes both are correlated with the same lack of IPTV service -the long channel zapping time (Fig. 1).The marking T zapping is the IPTV channel zapping time in Fig. 1.The analysis of various scientific works showed that the TV channel zapping time is the most important criteria which affect the subjective assessment of IPTV QoE [3].This is because the user's visual and audible perception of IPTV quality is based on the evaluation of TV channel change process.It was determined that a black TV screen, seen during the channel change process, affects the user's visual perception of IPTV quality the most [4].It means that the longer user doesn't see the start of the requested TV channel transmission, the more negative attitude is formed towards IPTV service quality and the television channel broadcast.
According to this, the main aim of this study is the experimental investigations of IPTV Quality of Experience, determining the TV cannel change process influence on the users' subjective perception of the IPTV service quality.Following the experimental investigations and its results the

II. IPTV QOE RELATION WITH THE TV CHANNEL CHANGE PROCESS
The main methods used for the subjective assessment of the IPTV Quality of Experience are: Mean Opinion Score (MOS) -the user's perception of the service quality is expressed in a scale from 1 (very low quality) to 5 (high quality) values [5]; Single Stimulus Continuous Quality Evaluation (SSCQE) [6]; Double Stimulus Continuous Quality Scale (DSCQS) [6].The subjective assessment of the IPTV Quality of Experience is based on the users' visual and audible receptors.It's also frequently based not only on the main criteria for the assessment of QoE, but on the others, visible or audible factors.The objective assessment of the service Quality of Experience is based on the analytical methods, where the possible errors of the subjective assessment are eliminated.
The researchers have proposed the QoS and QoE correlation method evaluating the delivery process of IPTV service and the quality parameters of the TV channel flow transmission [7].This method enables the IPTV service provider to objectively predict the Quality of Experience or customize the improvements of IPTV service in the individual interactions, while maintaining as much as possible optimal IPTV QoE evaluation.Such method of IPTV QoS and QoE correlation [7] is expressed in the mathematical formulas below where ) ( X QoS p is the Quality of IPTV service, according to the type of access network; h is the weighting factor of IPTV quality, dependent on the type of the access network for IPTV service; L I is the loss of IP packets; U I is the intensity of TV channel's flows by the requests' intervals for that TV channel; J I is the IP packets' delay variation; D I is the IP packets' delay; B I is the network's bandwidth; I I is the parameter of IPTV service's quality; is the limits of the parameters of IPTV service's quality, depending on the type of the access network.
The objective IPTV service's quality of experience can be assessed by the given quality of IPTV service ) ( X QoS p [7], presented in (2) where Qr w is the factor, that limits the range of IPTV channel's video groups of pictures (GoP), depending on the TV's screen size or resolution; A w is the factor, which describes the importance of IPTV service; GoP w is the factor, which defines the structure of video frames, allowing to the size of GoP.
However, during the analysis of this method it was found that the results based on this method are very different from the users' subjective assessments of IPTV QoE.It means that is important to offer a new solution that allows an objective assessment of the IPTV QoE by the understanding the relation between the digital IP television quality and Quality of Experience assessment according to the TV channel zapping time.

III. THE EXPERIMENTAL INVESTIGATIONS FOR THE SUBJECTIVE ASSESSMENT OF IPTV QUALITY OF EXPERIENCE
The experimental investigations were carried out in one of the IPTV service providers' network.The TV channel zapping times were measured by analyzing the user's behaviour for TV channel search and selection during the experimental investigations.The authors have shown that the evaluation of the user's behaviour to the channel change process is very necessary in their previous studies [8].The two methods of TV channel search and selection were used: 1) TV channel is selected sequentially; 2) TV channel is selected random.The measurements were carried out by changing 57 IPTV channels.The IPTV QoE was assessed by 100 independent respondents.The respondents rated the subjective IPTV QoE using MOS method and expressing the perceived quality of service in a scale from 1 (very long TV channel change process, particularly unacceptable to the user) to 5 (very fast TV channel change process, the user is satisfied with the transmission of IPTV service) values.The results of the experimental investigations are presented in Fig. 2 and Fig. 3.The analysis of the experimental results showed, that the evaluation of the subjective QoE according to the user's quality perception only partially corresponds to the average TV channel zapping time.The average IPTV channel zapping time between 38 th and 39 th television channels is above the ITU -T recommended limiting value for TV channel zapping time (Fig. 2) and the user of IPTV service assess it as a very long time for TV channel change process (Fig. 3).The user assesses such IPTV channel zapping time as unacceptable time in MOS scale ( 3 .

= MOS
).The assessment of the IPTV channel zapping average times showed that if TV channels were changed randomly all 57 IPTV channels zapping times exceeded the recommended limit of 2 seconds.The each experimental investigation was carried out in 570 times.Thus, TV channel zapping time is dependent on the user's behaviour for the TV channel change process if the user is changing IPTV channels randomly.The greater number of the queries for the TV channel change performance in one search period for the TV channel, the longer queries' delivery time in the channel change process.The longer queries' delivery process affects the longer TV channel zapping time.
However, there were noted the discrepancies between the IPTV channel zapping times and the subjective assessment of the perception of IPTV quality in MOS scale in both cases.So, in order to eliminate possible errors in the subjective assessment of the perception of IPTV quality was carried out the processing of the experimental results using the methods of the statistical analysis.

IV. THE PROPOSED MATHEMATICAL EXPRESSION FOR THE OBJECTIVE ASSESSMENT OF IPTV QUALITY OF EXPERIENCE
The correlation coefficients and the regression analysis of the statistical data collected during the experiments were performed in order to evaluate the correlation between the user's perceived quality of IPTV service and TV channel zapping time.The correlation coefficient between the MOS values and TV channel zapping times was 96 .0 − = r , when TV channel was selected sequentially, and 77 .0 − = r , when TV channel was selected random.Thus, it can be stated, that there is a strong relation between the user's subjective assessment of IPTV QoE by MOS values and the measured IPTV channel zapping times.
The correlation between the MOS values and IPTV channel zapping times during the regression analysis was determined using the fifth and third degree polynomials (Fig. 4).These dependencies of IPTV channel zapping times and MOS values are expressed in ( 3) and ( 4).The objective IPTV QoE when TV channel was selected sequentially (3) and random (4): The regression analysis allows assess only the quantitative relation between the analyzed parameters.However, it does not reveal critical thresholds of the changes in user's reactions.The sharpness of the change of the user's reactions can be assessed as a tolerance limit in the analysis of IPTV QoE.This is the limit to which the user is willing to get some changes with IPTV service's transmission.The user reacts to it negatively when the service is provided over the tolerance's limits.So it is important to determine the tolerance's limit according to the IPTV channel zapping time, in what the user's reaction to the perceived quality of service is in a significant change and the change's suddenness of this reaction.The determination of these parameters was carried out by the approximation of the experimental data using the function of arctangent and the optimization (function minimization) using the Nelder Mead method.The function minimization by finding a steepness of the function describes the sharpness of the change of the user's reactions.The results of these investigations are presented in Fig. 5.The analysis of the investigations' results showed that the sharpness of the change of the user's reactions to the IPTV channel zapping time and critical thresholds of this parameter are different using the different methods for the TV channel change.The limit of tolerance for the IPTV channel zapping time is 1.96 s when the user selects IPTV channel sequentially.The change's suddenness of this reaction is 2.4.The limit of tolerance for the IPTV channel zapping time is 2.7 s when the user selects IPTV channel random.The change's suddenness of this reaction is 3.42.
After the analysis of the experimental data, it was found that the standard deviation of the TV channel zapping time was 52 .0 1 = σ , when TV channel was selected sequentially.
The standard deviation of the TV channel zapping time was 9 .0 2 = σ , when TV channel was selected random.The basic mathematical expression of the evaluation of MOS values for the objective assessment of IPTV QoE were carried out by the evaluation of the limits of tolerance and the standard deviations of the TV channel zapping time according to the behaviour of the users for the TV channels selection where zapping T is IPTV channel zapping time; σ is the standard deviation of the experimental data of IPTV channel zapping time.
The mathematical expression for MOS evaluation using the experimental results was carried out depending on the range of the interval of the IPTV channel zapping time and this expression is presented below 5, where 1 MOS is the expression presented in (4) and 2 MOS is the expression presented in (3).
The objective assessment of the IPTV Quality of Experience by the TV channel zapping time is appropriate to carry out in order to eliminate the possible side factors of the IPTV quality' evaluation according to the user.In this case, the experiment was carried out by recording the TV channel zapping time, but not excluding the method of the TV channel search.The subjective assessment of the IPTV QoE in MOS scale was recorded also.The objective assessment of the IPTV Quality of Experience was estimated using the proposed mathematical expression for the MOS values calculation (6).The results of the difference between the subjective and the objective assessments of the IPTV QoE are presented in Fig. 6.The difference between the subjective and the objective assessments of the IPTV QoE in MOS scale.
The proposed method of the objective assessment of IPTV QoE can be used for the predictions of the IPTV service's QoE according to the channel change process.

V. CONCLUSIONS
The analysis of the investigations showed that the current method for IPTV channel change does not provide the given quality of service for the user.The results of the experimental investigations showed that the IPTV channel zapping time exceeds the ITU -T recommended limit of 2 seconds and the subjective assessment of IPTV QoE in MOS scale is lower than the acceptable minimum MOS value (ITU-T MOS = 3.5).The proposed mathematical expression of the objective assessment of IPTV QoE and depending on the TV channel zapping time was carried out using the results of the experimental analysis.It was stated that the average MOS value is 3.56, when TV channel was selected sequentially and the average MOS value is 1.73, when TV channel was selected random, after the analysis of the subjective and objective assessments of IPTV QoE.These results showed that it is a need to offer the new solution for the existing TV channel change process that would increase the attractiveness of IPTV service for the users.

Fig. 1 .
Fig. 1.The influence of the channel change process to the quality of IPTV service.

Fig. 2 .
Fig.2.IPTV channel zapping time using the different methods for TV channel search and selection[9].

Fig. 4 .
Fig. 4. The dependency of MOS values and IPTV channel zapping times and it polynomial approximation.

Fig. 5 .
Fig.5.The relationship between the user perceived quality of service (MOS) and TV channels zapping times using the different methods for IPTV channel search.
Fig.6.The difference between the subjective and the objective assessments of the IPTV QoE in MOS scale.