Suboptimal Solutions for Time Varying Time Delayed MPC Controllers

Model Predictive Control (MPC) with constraints is still an interesting subject and offers many problems to work on. This study basically aims to understand the optimization process and the decrease of convex quadratic costs in a single model predictive controller. For those processes where the system dynamics change so slowly it is essential to obtain the control law as soon as possible to minimize the time delay on the controller side. This study proposes an early termination of the optimization process and the suboptimal solution to the quadratic programming. To define the early termination in the following chapters it is discussed and explained when, where. The implementation of the strategy is also illustrated with a case study and it is compared to the LQR controller for the regulator problem. DOI: http://dx.doi.org/10.5755/j01.eee.20.2.6378


I. INTRODUCTION
The main difference between the conventional control and model predictive control is that conventional control uses a pre-computed control law (offline computation).Model predictive control or receding horizon control uses on-line optimization to solve a finite horizon open loop control problem based on iterative algorithms, applying the plant current state as the initial state.The advantages of model predictive control are that it can handle multivariable or time-varying plants; constraints can be employed directly in the problem [1], [2].
To make the implementation of MPC easy one of the methods that can be used is real-time implementation.Today execution time characteristics of MPC tasks are one of most interesting problems.The time-varying execution times introduce delays to the control systems which are not easy to compensate.The more time needed for the optimization the more the latency has the systems.The latency has an effect in control systems the same as input time delay which affects the control performance negatively unless it is well compensated.From a real time perspective it is held till now only by Cervin et al who considers the MPC tasks as periodic tasks for EDF scheduling based on early termination of iterative algorithms.In these papers, it is claimed that the early termination of the iterative algorithms Manuscript received April 9, 2013; accepted October 6, 2013.This research has been supported by Yildiz Technical University can increase the performance index of MPC and still have efficient results for the prediction of the control signals and fulfil the stability conditions [1], [3], [4].
In the last years, MPC controllers are often studied for time delay in sense of network control [5] where feasible networked MPC scheme is used for discrete time interconnected systems on which time varying transmission delay affects the network [6] via minimizing the upper bound of the cost function it is used as a robust one in discrete-time uncertain systems with time-varying delay, input constraints and bounded nonlinear perturbations.Reference [7] discusses delay compensation of the moving horizon estimator and aims to compensate the loss by updating the covariance matrix for those states if there is incoming data from the sensors for them.Another compensation study [8] takes the time varying time delays introduced by the communication network into consideration and offers a solution by modelling delays as disturbances.Reference [9] proposes MPC algorithm for linear parameter varying systems with unbounded delay and derives the sufficient conditions from linear matrix inequalities using relaxation matrices.Rather than other time varying time delays this study focuses on the delays on controller side.

II. MPC WITH TIME-VARYING TIME DELAY
The basic question in a single MPC with constraints is how long optimization process should go on.It is known that each iteration optimization process goes on further; a better value is achieved for the control signal.However, while the objective function decreases in every each iteration, the total cost depends on delay increase.
Up to now, the previous work which has been done in this field suggests interrupting the optimization when the costs depending on delay begin to increase.At this point, it is suggested to use a simple stop criterion.With this criterion it is aimed to hold the costs depending on the delay at a certain level.One basic illustration to this idea can be seen on Fig. 1.
Here on the figure, line one illustrates the optimization process which leads to a decrease of the objective function in every iteration step.Line two illustrates the costs depending on time-delay which is based on delay time before the control signal is given to the process.The idea in the previous work is to interrupt the optimization at a point on which costs start to increase when the optimization Suboptimal Solutions for Time Varying Time Delayed MPC Controllers T. Veli Mumcu 1 , K. Gulez 1 1 Department of Control and Automation Engineering, Yildiz Technical University, Istanbul, Turkey tmumcu@yildiz.edu.trprocess reaches a sub-optimal value which is not the best but still good for obtaining the control signal.Fig. 1.Simple stop criterion proposed in the previous work [1], [2].
For this figure then point A is the point where the optimization process is interrupted at a sub-optimal value and save time, before the delay dependent costs start increase vigorously.However, it is not declared in the work how this point theoretically can be obtained.The previous work suggests us to use a simple stop criterion based on experienced maximum allowed delay and needs to be searched for every single system.Moreover, it is not known if this point or maximum allowed delay is the best value where the optimization process can be interrupted or since it is based on the experience can there be a better value or a point which is not experienced, yet [3].

III. MATHEMATICAL METHODS WHICH CAN BE USED FOR DEFINING THE PREMATURE OPTIMIZATION IN A SINGLE MPC
In the light of the previous works done in this field it is known that the premature optimization process is studied with the active set methods.For the optimization process when active set methods are used, it is known that the optimum point is obtained by adding and removing the constraints to the active set.With this method it is not clear how the objective function will decrease as the number of the iterations increases in time.Because of working with different constraints inside the active set at different iterations it is also not clear how far the process should go on to achieve a better error and how many iterations it takes for the algorithm to reach the optimum point.All these factors make it not clear not only to decide whether it is logical to optimize further or not, but also theoretically not feasible to reach to the optimum point at a certain time [3], [4].
In the optimization process inside an MPC algorithm, the active set, simplex and interior point methods are the common solvers that can be used.Above it is clarified why it wouldn't be so suitable to use active set methods [10], [11].
For this work basically three reasons make interior point algorithms so attractive to implement to the optimization process for finding the premature optimal point.First the complexity of the total algorithm is always polynomial, has mostly O (n 3 l) complexity.Second, it offers an estimation of how far from the optimal value a solution at a given iteration is.Third, the numbers of the iteration which is needed to find the optimal point is bounded by an upper value bounded, and it is also one of fastest algorithm for the solution of the convex quadratic problem [12], [13].

A. Defining the Cost Function for Convex Quadratic Optimization Problem
In order to define the premature optimization in a single MPC, system equations with a control signal with time delay (Fig. 2) which is caused by solving the optimization problem should be transferred from continuous time to the discrete time.The plant (1) has a continuous-time state space model: Our aim is to discretise (1) with time delay so that the time delay can be transferred from continuous time to discrete time.Thus, the discreted time system equations [15] are seen in ( 2): The matrix form of ( 2 The cost function which is needed for optimization problem is shown below (4).To make the discretization process easier here a classic LQR cost function assumptions (5) will be used.Here after (6), it is assumed that the delay is embedded within the control signal uk to make the following equations easier to follow.The cost function in continuous time is seen in ( 4) .
Here, cost function is chosen as LQR that's why xref and uref will be zero.Then, the final cost function in continuous time is as in (6): 0 , 0, 0.
Once the discretized cost function is obtained, the following operations will be executed in order to transform the cost function so that it fits to the convex quadratic problem formulation.

B. Cancelling Cross Term (Cross Term Elimination) [16]
Cross term Nd is cancelled ( 7)-( 10) since the aim in the objective function to build the costs based on only state xk and control signal uk: The system equations after cross term elimination are (11)-( 13): Matrix decomposition (Cholesky factorization) [16] of the weighting terms of the cost function is ( 14), (15) system equations ( 11)-( 13) after Cholesky factorization are ( 16)-( 18): After these operations the discreted cost function for the delay depended MPC is (19 Now the delay-depended MPC (20) can be built for the regulator case using the transformed system equations , 0 where and xp is substituted in cost function J, and Q is chosen as C T C Then, our objective function for the quadratic programming will become (21

C. Defining the Constraints for Convex Quadratic Optimization Problem
It is known that MPC is solved via quadratic programming in case of constraints.Now the constraints (22) should be defined for the control horizon and introduced to the convex quadratic optimization problem.
All operations applied to the system equations and cost function should also be applied to the constraints.
Cancelling cross term (23)-(26) (cross term elimination): The new constraints after cross term elimination are (27 Matrix decomposition of the constraints (Cholesky factorization) is ( 28)-(31): The new constraints after Cholesky factorization are (32) Thus, it can easily be seen that the constraints for the control signal in this case depended on the state vector again nevertheless this dependence should be removed by introducing another MPC (33) only for control horizon to the system . where The altered constraints after MPC formulation are (34)

V. CONVEX QUADRATIC PROGRAMMING FOR MPC VIA A PATH FOLLOWING METHOD
Once the MPC is built under the constraints, it leads to a quadratic programming problem.Literally this problem can mainly be solved either using an active set algorithm (simplex for quadratic case), or an interior point algorithm.For this MPC structure with constraint(s) it is chosen to have an interior point algorithm since these have the following advantages: Based on the direction criterion it is often possible to prove that the cost function decreases in every iteration, however, this is not possible in active set or simplex methods since the value of the cost function can remain same [10].Interior point methods have an iteration bound which gives information in the current iteration about the distance to the optimum point [11].
Since the iteration calculation is constituted of fixed numbers of equations, iteration has always the same processing time on a microprocessor.
All these advantages help us to define the preoptimization point where the total cost begins to increase after a certain number of iterations because of the timevarying time delay.The more time it takes to calculate the control signal the more the system will become distant to the point where the state variables are measured.
A path following method is chosen here to solve the MPC problem.Today there are several packages which can be used to solve this problem via an interior point algorithm (Newton-KKT [13], Quadprog-middle scale algorithms [17], Yalmip (interior point option) [18], QPC [19], etc).
Here is (35) objective function with constraints again.The quadratic programming solver QPC can handle this problem directly, however for other solvers these constraints should be altered to the form or Ax b Ax b   just as in (36): The constraints (37 In the constraint formula it shouldn't be forgotten that the state variable xk is the initial value of the variable.In order to have the representation seen easier, this representation is expressed in the form of classic quadratic programming [9] in vector representation as in (38)

VI. CONVERGENCE OF QUADRATIC PROGRAMMING
This part deals with solving the quadratic programming problem in a general case.In each iteration, a preoptimization criterion for a delay depended MPC makes only sense in case of decreasing costs.The decrease of the costs in interior point methods is always guaranteed with a safe search direction criterion.This criterion can be found for the solvers in their convergence analysis.Together decreasing costs and iteration bound help us to see how far a given iteration is from the optimum point.
In this work the solver [12] is used.The following proof belongs to the reference [12] and is added here to explain the idea how cost decrease in optimization process.In this solver the regular convergence ratio (39) is defined.The proof of (39) is explained in the following (39)-( 45).Here, some symbol definitions are listed below: n: Number of variables in quadratic programming, λ>0 is the barrier parameter, and chosen as .
Lemma I: Proof: Note that: From ( 41), (42) the following (43) is obtained The last inequality (43) is true since the quadratic term reaches its maximum value at this value   With this proof lasted in (45), the aim is to show that in a path following solver the costs are decreasing through optimization process rather than they decrease with a regular factor.Thus, the general iteration bound belongs to [11].This iteration bound can be updated to the solvers using the right decrease factors.
Lemma II: If the barrier parameter µ starts with the initial value µ0 and updated by 1 -θ, with 0 < θ < 1 After at most iterations nµ   is obtained.
Proof: The duality gap is determined nµ 0 and after the iteration, it is decreased with 1-θ.When k iterations passes, the duality gap is lesser than ε if Introducing the logarithm of both sides of (46), ( 47) is obtained where -log This explains the lemma II in (46).

VII. IMPLEMENTATION OF THE DELAY DEPENDED MPC ON
A PROCESSOR The implementation of the time varying time delay dependent MPC will be as follows: Step 1: Discretise system (1) with a fixed τ which is system (1) +one iter Step 2: Discretization + transformation of system (1) Step 3: Calculate MPC cost function J (11)

and constraints
Step 5: Iteration= iteration+1, update τ and go to Step1 Step 4: If Jk< Jk+1 QP for iteration otherwise stop, actuate u In the first step the time delay is defined as a fixed time delay which is the total time of discritizing system matrices and calculating the cost function and constraints for quadratic programming problem plus time needed to solve the quadratic problem.In each iteration, it is checked either the calculated delay depended cost is higher than the cost one iteration before.If the total cost still decrease, the QP for given iteration steps is solved.Once the total cost starts to increase instead of decreasing then the solver is stopped and actuate the control signal.
This code implementation can be done both offline and online.In offline version the cost curves of a given system can be calculated and the optimum cost is found for optimum iteration steps for the system.After this point there is no need to iterate further since the total costs start to increase instead of decreasing.In the latter one, during the quadratic programming, the time-varying time delay costs can be checked and compared with the help of sub-codes.Once it is realized that the delay depended costs start to increase the quadratic solver can be interrupted and control signal can be actuated.A time-varying time delay effect always makes control signal calculations harder since the system moves to another point once the control signal is calculated and introduced to the system.Here with this process which is introduced above our aim is to minimize the time-varying time delay effect for a MPC with constraints so that the calculated control signal will fit to the system.Thus, it is aimed to interrupt the optimization process in MPC with constraints in order to minimize the time-varying time delay which is needed to calculate the control signal.

A. Case Study
For the given system above, the discretized cost and the iteration for the optimization are introduced as 5 % time delay to the system.This is increased up to 50 % percent time delay over the optimization process which needs ten iterations to find the minimum.It is seen that the discrete cost function which has the time varying time delay effect starts to increase after the third optimization iteration.After the third optimization iteration, the discretized cost function starts to increase instead of decreasing because of the increasing time delay effect.Here the optimization process interrupted at the third iteration in quadratic programming since the calculated costs are minima at this point.The discritized cost values over the iterations can be seen in Fig. 3. Fig. 4 is given to compare the MPC with a traditional one.In the trajectory following overall performance of a MPC with a pre-interrupted optimization algorithm is faster than traditional LQR.However, this can be the opposite based on single state variables.The cost decrease is seen as on Table I.On the table, it is seen that during the optimization process cost starts to increase again because of the time varying time delay effect.Here for this system the optimization algorithm can be interrupted at the third iteration.
For a MPC with initial values the output changes of the classical MPC and the proposed one can be seen on Fig. 6.

VIII. CONCLUSIONS
In this work it is seen that the cost in an MPC start to increase during the optimization process.Therefore, it is better to interrupt the optimization algorithm where the cost starts to increase instead of decreasing.A control found this way may smaller than a traditional one since the interrupted cost will be smaller than the cost noninterrupted.
In the future, our aim is to research the case with several MPCs with different cost and initial values.This way we intend to find an algorithm to schedule several MPCs with different cost values running on a single processor to increase the overall system performance.

Fig. 2 .
Fig. 2. Time model of the control signal with time delay [14].

Fig. 5 .
Fig. 5.The output changes of MPC Classic vs. MPC Pre-optimized for step input.

TABLE I .
COST DECREASE IN OPTIMIZATION PROCESS.