Modelling VM Migration in a Fog Computing Environment

1 Abstract —Fog Computing is created to efficiently store and access data without the limitations challenging Cloud Computing deployments, such as network latency or bandwidth constraints. This is achieved by performing most of the processing on servers located as close as possible to where data is being collected. When mobile devices are equipped with limited resources and small capabilities, it would be convenient to make their associated computing and network resources follow them as much as possible. In this paper, migration process is studied and an algorithmic model is designed, selecting a generic Fat Tree architecture as the underlying topology, which may be useful to get a list of all devices being traversed through each of the redundant paths available.


I. INTRODUCTION
Fog Computing paradigm is characterised by the allocation of computing resources at the edge of the network, thus bringing the cloud computing assets closer to the end user [1].In this context, special attention may be set on its use in Internet of Things (IoT) deployments and, particularly, in IoT moving environments [2].
Such moving IoT devices have special characteristics according to its limited computing capacity, limited battery resources, and limited bandwidth [3].So, a good solution for their implementation is to decouple their computing assets and move them to more resourceful facilities, powerful enough to take responsibility for a lot of IoT devices, but close enough to reduce latency and bandwidth usage [4].Therefore, those facilities are composed of a bunch of capable servers being able to assign a Virtual Machine (VM) to each user to cover the computing needs of each IoT device.
When talking about moving IoT devices, this outlook is crucial as those devices lack resources of all kinds [5], such as those related before, and the use of a VM to carry the computing assets of each device may help to cope with the issues regarding the resources [6].
However, a new problem arises with the moving IoT devices because, as they are moving around, those VMs might end up being too far away in the complex network Manuscript received 18 December, 2018; accepted 1 July, 2019.architecture from their associated devices.Hence, VMs should be moved as close as possible.This mechanism is called VM migration and must be taken into account in those environments [7].
Therefore, two types of movements can be distinguished: the movement of the device throughout the coverage area and the movement of the virtual machine associated to that device trying to get as close as possible to its owner.
The first sort of movement, the one regarding just the moving IoT device, is called mobility.Its study is all about trying to model the most usual movements as well as the not so usual ones.Regarding literature, there are some attempts to model general human movements in wireless environments, such as [8] and [9].Also, there are some simple mathematical models, such as the ones proposed in [10], [11], and other more complex models related to crowd interaction, such as [12], [13], and [14].
The second kind of movement, the one related to VMs associated to moving IoT devices trying to follow them around, brings about the issue of trying to migrate a VM from the server hosting it to another one being located nearer to the actual position of the moving IoT device in order to facilitate the interaction between the device and its computing power, as a consequence of reducing the latency and bandwidth of such communications.
Regarding literature, a conceptual live VM migration framework is proposed in [15], also for Cloud Computing in [16] and for Fog Computing in [17], whereas a comparison between live VM migration in both environments for multimedia services is presented in [18].
In this paper, we are going to concentrate on studying the VM migration happening in the situations mentioned above.Furthermore, we will focus on getting a general algorithm for modelling it in a generic Fat Tree architecture for to make an interesting framework being able to support the necessary infrastructure for allocating VMs within physical servers and facilitating VM migration throughout any pair of available servers.
The paper is organized as follows.In Section II, a general procedure for live VM migration is introduced.In Section III, a Clos network overview is shown.The description of the behaviour of a Fat Tree architecture from the modelling point of view is given in Section IV.Later, in Section V, a general algorithm aimed at modelling VM migration in a Fat Tree topology is proposed.A method for getting all devices on redundant paths is presented in Section VI, whereas final conclusions are drawn in Section VII.

II. LIVE VM MIGRATION PROCESS
Regarding VM migration, there are three main approaches to be taken into account [19], such as cold migration, where the VM is shut down before moving it, hot migration, where just its OS is suspended before the movement, and, finally live migration, thus allowing the services running on it to keep going in a seamless manner whilst the movement is performed.The latter one is the most interesting situation.
There are three key parameters to measure the performance of live migration [20], such as downtime, which represents the amount of time the VM is halted during the migration, total migration time carrying the amount of time elapsed for the whole process and the amount of dirty pages migrated referring to the data being changed during the process, and, therefore, having to be further sent over again.
Regarding the live VM migration process, a trade-off must be considered between the downtime and the total migration time.In order to achieve this, the memory transfer is the key player.However, connections to the local devices and network interfaces may also be taken into account.
Generally speaking, the memory transfers can be broken up into three stages [21]: 1. Push phase, where the source VM keeps running whilst the transfer process starts taking place; 2. Stop-and-copy phase, where the source VM is halted, pages are copied through, and destination VM is started; 3. Pull phase, where the new VM runs and, if a requested page has not yet been copied, it is retrieved from the source VM.Some techniques are proposed to undertake the live VM migration process in an efficient manner by just focusing on one or two of the stages described above, such as pure stopand-copy, pure demand-migration or post-copy live migration.However, it seems that the most efficient approach is the pre-copy migration composed by a combination of a bounded iterative push stage with a very short stop-and-copy stage, where a number of iterations take place until all dirty pages are transferred.
The pre-copy migration process between two hosts may be divided into six stages, where a VM transaction between the two hosts takes place according to a pre-established migration timeline: 1. Pre-migration, where a destination host with enough resources is preselected; 2. Reservation, where resources are allocated beforehand at that destination host; 3. Iterative pre-copy, where the whole RAM is sent in the first iteration and dirty pages are sent in the following iterations; 4. Stop-and-copy, where the source VM is halted, so as to copy its CPU state and remaining inconsistent pages to the destination VM; 5. Commitment, where the destination host acknowledges it has received a consistent VM copy and the source host acknowledges it back prior to discarding the original VM; 6. Activation, where the migrated VM gets activated and the device drivers are attached to the new VM.
To round it all up, Fig. 1 exhibits the timeline for the live VM iterative pre-copy migration process.

III. CLOS NETWORKS
Back in the fifties, Clos networks were designed in order to switch telephone calls in an efficient manner [22] by virtue of using crossbar switches.Basically, the point was the use of equipment with multiple stages of interconnection in order for the calls to be completed, hence providing alternative paths between sources and destinations allowing the phone call to be always connected and not blocked by any other call.
Later in the nineties, Ethernet switches came along and the concept of Clos networks was expanded for to achieve cost-effective, reduced operational complexity, and limited scalability [23].The point there was to create multistage topologies built with commodity switches, so cost-effective deployments might be attained.
Afterwards, with the arrival of the 21 st century, Data Centers and Cloud Computing facilities are still using those topologies with different proposals, such as two-stage designs [24], three-stage ones [25] or even alternative ones [26] each one having its own benefits and drawbacks, hence providing a full range of solutions in order to deal with different situations.
Those topologies may be well used regarding the underlying structure of the Fog Computing environments in order to host VMs and support the necessary live VM migrations, where two of the main proposals in literature are Leaf and Spine [27] and Fat Tree architectures [28].
Leaf and Spine is a 2-tier topology, where the lower one is composed of switches directly connecting with servers and the upper one is made of switches interconnecting the lower ones in a full mesh fashion.This design provides full redundancy, as there are always a number of redundant paths among any two given switches, which are equal to the number of switches being part of the upper layer.The main drawback of the design, as it is prone to scalability issues as the number of switches increases, is the number of redundant connections to be provided.Figure 2 depicts an example of such a topology with four switches in the Spine layer.This way, there are less redundant paths among any two given switches, but there are no scalability issues any more.Figure 3 exhibits an example of the Fat Tree topology with K = 4 and 1:1 oversubscription meaning that all theoretical links in the topology have been used.

IV. FAT TREE BEHAVIOUR FOR MODELLING
In order to obtain a modelling for the Fat Tree architecture, it is necessary to describe how each layer of the topology behaves.To start with taking the Fat Tree architecture depicted above as a ground for a formal model, devices are identified from left to right for each different layer, so as to design a model showing how this architecture allows the communication flow among servers.
Fat Tree architecture is composed of three layers of switches, where Edge layer is the lower one, Aggregation layer is the middle one, and Core layer is the upper one.Additionally, a bottom layer holding hosts or servers is also considered.
Regarding the name of the Fat Tree, the word "Tree" comes from the inverted tree-like structure of this architecture, where Core layer might be seen as the root layer and Hosts layer as the leaves layer.
Furthermore, following that analogy, the word "Fat" comes from the existence of more links on the root layer than in the leaves layer, such that there are (K/2) 2 links between upper and middle layers, (K/2) 1 links between middle and lower layers, and (K/2) 0 , i.e., 1 link between lower and hosts layers.
The Fat Tree structure may be considered as a K-ary tree, being K the main parameter of this structure, as there are K Pods each of those containing K switches divided into lower and middle layers and each switch also has K ports.
In Figure 3, K = 4 is used, although it may be extended to any natural even number.Therefore, it will be represented by the algorithmic model to be shown.
The model might be built up by using Fig. 3 as a source of information.As it may be seen over there, switches at the all layers are numbered from 0 onwards considering left to right direction.
As a matter of fact, there is a total of (K/2) hosts hanging out of each lower switch, which means that there are (K 2 /2) hosts hanging out of each Pod, which also means that there are (K 3 /4) hosts in the whole topology.In addition to that, there is a total of (K 2 /2) switches in both the lower layer and the middle layer, whereas there are (K 2 /4) in the upper one.
As per nomenclature of the ports of each item, it is said that the servers have only one port, which is labeled as 0, whereas on the switches the ports are labeled from 0 to (K-1) from left to right starting from the bottom and finishing on the top (Fig. 4).With all this in mind, the model is going to state the atomic actions for communicating messages involved in the live VM migration process, such as send or receive in a way that both will bear the item and the corresponding port taking part in such communication.Furthermore, the model is going to show the decision-making processes in order to guide a VM from a given source to the proper destination following the optimum paths that are one, two or three hops away.
The arguments for the send and receive actions are the source host (a), the destination host (b), and the proper VM to be migrated owned by a user identifier (u), i.e., VM(u).
All items within each layer are modelled in a generic manner, such as HOST Hh, EDGE Ei, AGGREGATION Aj, and CORE Cl, where each variable is bounded by the number of items for each layer exposed above.The model is expressed with some snippets coded in C-style for clarity purposes.It takes into account all considerations being made also.

V. VM MIGRATION MODEL  HOST Hh:
The server layer, also known as the Host layer, is the easiest to model as each server (h) may perform just two actions, such as receiving or sending a VM.Those actions are the key actions chosen herein for modelling the VM migration process.Further considerations, such as VM creation or VM termination are not borne in mind for the simplification purposes.Let us suppose there is enough room to allocate the VMs assigned to all the users within the system.
All the servers in the topology are awaiting to undertake any of those two actions of receiving and sending at any given time as it is not usually known beforehand when a VM migration is going to take place.As per the receiving part, it is performed when the VM associated to a particular user is not located into a given host, hence the host is ready to receive that VM in anytime.Otherwise, the sending part is carried out when the aforesaid VM is indeed located in that host and it is time to leave (see Algorithm 1).
Additionally, each server has only one port, namely eth0, so messages to and from the Edge layer move through that single port, whilst messages have the structure (a, b, and VM(u)).

 EDGE Ei:
The lower layer, also known as the Edge layer, is the one directly connected to the servers.Each switch located therein is continuously monitoring all its ports in order to receive a VM to be translated from a source host to a destination host.
It is to be remarked that any switch located in this layer has the lower half of its ports looking downwards, whereas the upper half are looking upwards.
On the one hand, in case a VM is received from a given host on any switch, it is going to come from any of its lower ports, and then two cases may be distinguished.If the destination is another host hanging on the same switch, it is going to be forwarded downwards straight to that other host.In this case, both hosts may be considered as being part of the same IntraNet, as they may be just one-hop away to each other.Otherwise, if that is not the case, it is going to be forwarded upwards.
On the other hand, in case a VM is received from any switch standing on the middle layer, it is going to come from any of its upper ports, and then it will be guided through the port directly connected to the destination host (see Algorithm 2).The middle layer, also known as the Aggregation layer, has the same port configuration as the lower layer.It has to be noted that there is a full mesh topology among switches staying on both layers within a single Pod.
As in the previous layer, all switches therein are monitoring their ports and all the time waiting for incoming VMs.Therefore, if a VM is received from any of the lower ports of any of these switches, two case scenarios may be distinguished.
If the destination is another host situated on the same Pod, that VM will be forwarded downwards through the lower switch, where the destination is hanging on as both source and destination hosts may be considered as being a part of the same Pod, also known as IntraPod, being two-hops away from each other.Otherwise, if this is not the case, it is going to be forwarded upwards.
Alternatively, if a VM is received from any upper ports of any of those switches, it is going to be headed for the lower layer switch, where the destination host is hanging on (see Algorithm 3).
Then, let us take an IntraPod scenario.The model considers both Hosts being 2-hops away with K/2 redundant paths in between as both Hosts share the same Pod.Each Aggregation switch within that single Pod defines a different path being reached from the same source Edge switch and being redirected to the same destination Edge switch.Thus, the second conditional sentence in the algorithm is going to hold and the components of each path are going to be identified.
Finally, let us take an InterPod scenario.The model considers both Hosts being 3-hops away with K 2 /4 redundant paths between both as both Hosts stand in different Pods.Hence, each Core switch defines a different path being reached from one of the Aggregation switches in the source Pod and being redirected to one of the Aggregation switches in the destination Pod.Therefore, the else sentence in the conditional sentence is going to hold and all the components of each path are going to be identified.
To round it all up, the use of this algorithm matches what is happening in a real Fat Tree architecture regarding all possible case scenarios.Therefore, the model may be considered as verified.

VII. CONCLUSIONS
In this paper, the VM migration process between a given source host and a given destination host, both being interconnected through a Fat Tree architecture, is studied.First of all, migration types are exposed noting that live VM migration is the most interesting one and, among all its variations, iterative pre-copy is the most efficient regarding the trade-off between the downtime and the total migration time.
Then, Clos networks are presented and a comparison between Leaf and Spine and Fat Tree architectures is introduced leading to the consideration of Fat Tree as being more scalable.That is why it is selected to build an algorithmic model regarding VM migration.
Eventually, each of the three layers of the Fat Tree and an additional layer for the hosts, where VMs are located, are modelled.The behaviour of each layer is expressed in terms of receiving a VM owned by a user coming from a source host and going to a destination host and, in turn, sending it to the optimal path to reach that destination with the minimal possible number of hops.If that number of hops is more than one, those are redundant paths, which are all optimal.So, the model gives them all.
As a sort of the proof of the concept, two algorithms are proposed.The first one lists the devices to be traversed through all redundant paths available from a given source host to a given destination host.The second one is an extension of the former quoting the ports involved in each link through each redundant path.
In conclusion, the algorithmic model proposed, which is based on arithmetic operations, succeeds in expressing the VM migration process from one host to another in a Fat Tree architecture.

Fig. 2 .
Fig. 2. Leaf and Spine topology.On the other hand, the Fat Tree is an alternative to the above mentioned topology because it allows better scalability.It is a 3-tier topology subdivided in Pods, where the full mesh interconnection among the switches located in the lower layer and the upper layer of each Pod is.It is quite similar to the first topology introduced, except having the extra top layer, which is in charge of interconnecting the different Pods taking part in the topology.This way, there are less redundant paths among any two given switches, but there are no scalability issues any more.Figure3exhibits an example of the Fat Tree topology with K = 4 and 1:1 oversubscription meaning that all theoretical links in the topology have been used.

Fig. 4 .
Fig. 4. Links on each layer of the Fat Tree architecture.