Time-Series Prediction for Amount of Airworthiness Based on Time-Delay Neural Networks

Troposphere and the first stratum of the stratosphere are intensely utilized atmosphere layers for the aviation activities. Due to the different performances, capabilities, designs, and equipment of the aerial vehicles, meteorological weather events that occur in the troposphere affect these vehicles at different levels during their aeronautical activities. Although simple aircrafts are more sensitive to the effects of meteorological events, they are especially preferred by flight training organizations (FTOs) in pilotage training when they are considered in terms of maintenance and equipment costs. In cases where inexperienced pilot candidates and simple aircrafts that are more vulnerable to weather events come together, analysis and prediction of meteorological parameters becomes more important in terms of preventing accidents and reducing risks, as well as proper planning for flight and maintenance. The purposes of this study are, first, to derive flight availability time-series for two different types of aircraft according to visual flight rules by using Meteorological Terminal Air Report (METAR), and then to establish and evaluate a prediction model by using Time-Delay Neural Networks (TDNNs).


I. INTRODUCTION
The flight activities are directly affected by Meteorological parameters and events. So much so that meteorological information plays an important role in planning, such as flight programs, travel times, routes, and flight levels. Non-optimal weather conditions may cause delay, cancel or re-route of the flights for safety reasons. In addition, predicting unfavourable weather conditions and flight times to reduce meteorological risks to ensure flight safety is also a substantial factor in the planning of maintenance activities, which is also one of the flight safety requirements.
Candidates' flight training is often carried out on a Manuscript received 8 February, 20č0; accepted 29 July, 2020. This article has been prepared from the master's thesis "Investigation of Airworthiness in terms of Meteorology and Artificial Neural Networks Approach in the Time Series for Short-term Forecasting Model: Case of Hasan Polatkan Airport" completed by the first author at Anadolu University, Institute of Science, Avionics Department.
training plane according to visual flight rules (VFR) and in the troposphere where meteorological events occur. In terms of equipment and maintenance costs, flight training organizations (FTOs) may prefer aircrafts with simpler designs that are generally more sensitive to meteorological conditions. Flight safety can be adversely affected when both simple structured training aircraft and inexperienced pilot candidates come together with the adverse weather conditions, such as high winds, cross winds or limited visibility. Thus, meteorological assessments made by various reports, such as METAR, SPECI, and TREND, which analyse the current and future meteorological situation, play an important role in the flight/maintenance plans of FTOs [1], [2].
In this study, first, flight availability time-series for Cessna-172SP and TB-20 aircrafts according to VFR were derived by using the data of METAR belonging to Hasan Polatkan Airport in Eskisehir, and then the prediction of daily flight availability models were made using Time-Delay Neural Networks (TDNNs) capabilities on these time-series. Selected aircrafts for this study are in the stock of Eskisehir Technical University, Faculty of Aeronautics and Astronautics and they are actively used for the flight training. The purpose of this study is to introduce a model to predict the airworthiness to support preventing accidents and reducing risks in flight training activities and also to proper scheduling for flight and maintenance. The general structure of the TDNNs with its implementation and its evaluation criteria and the time-series of airworthiness are introduced at the second, third, and fourth part of the study, respectively. Finally, in the last section, the results obtained are discussed and evaluated.
Conventional artificial neural networks provide a model between inputs and outputs by carrying out pattern association. In network models where the time is the subject, it is possible to recognize patterns that evolve throughout time by establishing a model that relates past values to new values. TDNNs are models which are equipped with memory structures by using a conventional network in a way that allows recognition of patterns evolving throughout time. TDNNs have some practical applications in areas such as prediction, classification, signal processing, motor controlling, and voice recognition [3]- [10].

II. DATA SET
Considering that the meteorological aeronautical surveys require a minimum of five years as stated in [11], a timeseries covering airworthiness for pilot candidates' information hourly per day was generated from the METAR and SPECI of Hasan Polatkan Airport, including Jan. 2009-Jun. 2015. Table I shows the general information of this airport, which is used for flight training activities by Eskisehir Technical University. For network training and testing, ten data sets were used that generated from the timeseries. While creating the time-series, crosswinds to be encountered by the training aircraft during take-off and landing were also evaluated from the runway perspective and from the crosswind limits given in the aircraft manuals. Due to the fact that aircraft must take position relative to the heading of runway during take-off and landing, crosswind components exceeding the limits specified in the manuals (depending on the type of aircraft) restrict the use of the runway. In this study, the crosswind components to which Cessna-172 SP and TB-20 aircraft are exposed are determined and evaluated according to Fig. 1, using the prevailing wind direction and speed. Accordingly, the aircrafts can perform their flights safe with a limit of 15 kt and 25 kt, respectively, for Cessna-172 SP and TB-20 [14], [15].
The majority of the educational aviation activities process is completed under VFR, and so FTOs often use the low altitudes of the troposphere. Since the troposphere is the atmospheric layer in which weather events occur, it is necessary to carefully examine the weather conditions for VFR flights. As described in the Turkish Aeronautical Information Publication (AIP), VFR flights cannot be possible if at least one of the following conditions is occurred: the visibility falls below 5 km, the cloud ceiling falls below 1500 ft, and so 30 minutes after sunset, and 30 minutes before sunrise [12].
It is stated in [1] that about 20 % of the aviation sector accidents are based on meteorological events. Therefore, considering the inexperienced pilot candidates in flight training processes, meteorological events need to be taken into account more precisely. Considering this situation, both the VFR conditions and substantial meteorological events that will negatively affect aviation operations are evaluated as a measure of airworthiness in this study. In addition, based on the pilot experience, as well as technical criteria, the following situations are evaluated for flight availability:  Cloud cover should be less than 5/8 octa;  Cloud ceiling should be more than 3000 m;  Temperature should be more than 5 °C;  There should be no thunderstorms, heavy torrential rainfall, clouds of cumulus congestus, and cumulonimbus.

III. TDNN ARCHITECTURE
In the context of artificial neural networks, determining the appropriate network weights in the model, which is created in a way to produce the outputs that are proper for the network inputs, is called the "training process". While establishing the relationship between input and output in the training process, using the time can yield valuable information to the network. Time can be presented to the network in the form of a numerical quantity as an input. Besides, it may be presented by the changing frequency of an input signal to recognize the patterns that evolve throughout time or may be presented as past values applied to the network. With the inclusion of time into the process of the prediction model, it can be possible that statistical variations can be pursued in a non-stationary time-series. In this paper, integration of the time into the network model has been applied both as a part of the network's operation and as a numerical input [3]- [10].
TDNN uses ordinary input time-delays to perform temporal processing, which was described in Lang and Hinton. TDNN models shown in Fig. 2 have memory structures obtained by an input delay mechanism which allows the recognition of patterns that evolve throughout time. Because TDNN has no feedback from output to the input of the network, it is not a recurrent network. According to the model, network inputs are the input matrix x at time t and the past values up to time p. In the input layer of the network, model has a discrete-time structure, and the association between input and output is established through a p dimensional window. TDNN is a conventional feedforward network equipped with a memory structure which delays input and presents as input to the network. The network is trained similarly like the conventional feedforward networks by updating the weights to reduce the error so that the moving window in the time domain shall induce the producing a minimum error with the corresponding observed and desired outputs. The TDNN structure can be given in (1) and (2) as: In (2), the bold characters denotes matrix and superscript (·) T denotes the transpose of a matrix. Weights of the network are indicated by w and inputs are indicated by x. The number of hidden layer cells and output cells are indicated by l and m, respectively. The discrete-time index, hidden layer neuron index, and lag index are given by t, j, and L, respectively. In addition, bm and bj are the biases and denote the output layer and the hidden layer bias neurons. f(·) denotes the activation function which is a differentiable and continuous function. The sum of the convolution with bj forms the input of the activation function.
Because of the TDNN has time-delayed inputs, each time delay in the window is connected to the hidden layer nodes over its own weight and presents input values in past instances in time to network. Considering the input matrix dimension, there are many connections in the input layer compared to the traditional network.

IV. METHOD
The data set included in this study is composed of a timeseries containing 6.5 years of daily available hour information about airworthiness. The first 5-year data are allocated for training and the remaining 1.5-year data are reserved for testing. Two types of data sets are created to predict the amount of hours available for flight which belongs to the day of t. Both of these data sets include previous time series data from t = 1 to t = 5. However, one of the data sets contains an additional input which is the month of the information when the forecasting is made, while the other does not contain any additional inputs. Depending on the model created, the outputs corresponding to the inputs include the number of hours available for the flight tomorrow. In other words, 5 of the 10 datasets includes airworthiness data covering up to the past 5 days, while remaining 5 data sets contain an additional information about the month of forecast, as well as the airworthiness.
Since the best hidden layer cell number is unknown, trials were performed on different cell quantity. Therefore, the quantity of neuron in the hidden layer is increased one by one from 3 to 60 in created network models. In addition, the activation functions of neurons have been changed to be sigmoid, hyperbolic tangent, Elliot sigmoid, and linear functions [16]- [19], whose equations are given with the equations

t p j l t p j t p j t p j t p j
Networks consist of different delay structures. These are four types: without delay, with one delay, with two delays, and with three delays. Thus, in total 9280 networks were trained and tested using 10 data sets as input. The test phase was performed by testing the last 23 percent of data on the network, which was never used in the training phase of the network. The test data sets contain the last 1.5 years. Different from the training success, the test data set which is the continuation of the training time-series was applied in the test phase to the trained networks to determine the success of the forecast. The most consistent network models were determined according to the differences between the network outputs (o) and the actual values (ô) that should have occurred. As the comparison criterion, the Root Mean Square Error (RMSE) is used, which is given in (7) [13],

V. RESULTS
The RMSE for the training and testing phases is given in Table II. Additionally, the table gives the information of in which delay, activation function, at how many hidden layer neurons, and in which data set this RMSE is obtained. The input dataset column represents the dataset content, in other words, the size of the input matrix, and it is one of datasets from 10 datasets prepared. For example, "t -1.5 and month" means a 6-dimensional vector containing 5 past discrete values in the time-series and also the month in which the forecast is made. The values indicated as bold in the table are the minimum RMSE between models that have different delay structure. In respect of the training and testing phases, the underlined values presented in the Table II is the network which has the least RMSE among the trained 9280 networks. The most successful network is obtained with the linear activation function at 5 hidden layer neurons by using the data set covering only {t -1} data. For training and testing stages, the best RMSE is 0.198 and 1.866, respectively. In terms of training and testing, the addition of a delay to the network model has positive result when compared to the memory-less model because of the filtering effect of pointless high-frequency components with delay. However, as input time-delay structures are increased, valuable information from the signal also filtered. And as a result that reduces the training and testing success. Adding a delay does not always mean that it will contribute positively to network success.
The model with the minimum RMSE in the training phase does not always mean that the minimum RMSE results will be obtained in the testing phase. For example, among twodelayed network models, the training with the least RMSE was obtained in the model with the activation function of Elliot sigmoid at 38 hidden layer neurons, using the past two time-series data {t -2, t -1} and the month information as inputs. Of all this, in the training phase, the network with sigmoid activation function at 45 hidden layer neurons, and the inputs containing past five days {t -5 to t -1} could have modeled the test time-series in the least error. This is due to the meteorological chaotic nature of the flight availability time series.
Models that memory-less structure have a similar situation. In the three-delayed network model, although close errors are obtained in different activation functions, the use of month information contributes positively to the training and testing stages.

VI. CONCLUSIONS
Time-delay structures allow the network input data set to be driven by creating a window in the input layer. Therewithal, over time, similar time-related input patterns can be represented more on weights in the input layer. This model practically functions as a finite impulse response (FIR) filter. Therefore, the high-frequency components in the time-series are less influential and making a prediction by using the outlines of the input time-series is possible. Adding delay structure to each network causes detailed information to be more filtered in the time-series. The loss of information as a result of excessive filtering of the signal by increasing the delay structures, also the presence of unfiltered high-frequency components because of adding no delay structure, negatively affect the estimation success of the network.
Besides, the RMSE appears to be higher due to the existence of pointless high-frequency components in the time-series. In addition, using more than one of delay structures leads to valuable frequency components to be filtered, which can negatively affect training and testing success. In this case, network models require more hidden layer neurons to produce fewer RMSE. Due to the small number of hidden layer neurons, the network has fewer weights, and thus the error between the observed and the desired outputs can be easily distributed to the network. In the models of without time-delay (the memory-less models), the requirement of a few numbers of hidden layer neurons verifies this.
If any delay structures are not used, the model transforms into a conventional MLP model. In such a case, to reduce the RMSE, the training data-set of the network must have patterns as diverse as the patterns in the data-set. Thus, the network error decreases as the most common patterns take more place in the network weights.
Close and acceptable results are obtained from the TDNN models prepared using 1 delay. From the experiments, the best prediction model is achieved in 1.866 RMSE. Without using any lag which turns the model to conventional network, test error results increased around to 4.2 and has doubled. Also, using 2 or 3 lags makes test error results increase around to 3.4 and become worse.
The minimum RMSE is obtained in a network model with less layer neurons and less past data. This shows that the relationship of the value to its first past value in the timeseries is high.