Modelling of Ambient Comfort Affect Reward Based Adaptive Laboratory Climate Controller

There are enlarged capabilities of the Ambient Comfort Affect Reward Based Laboratory Climate Controller (ACAR-Controller) by developing and integrating of the Heating/Ventilation/Air Conditioning (HVAC) and the Red- Green-Blue-Yellow (RGBY) Light Emitting Diode (LED) lighting sub-models of one room laboratory in the ACAR- Controller model. The model was validated by implementation and testing of the following elements of the laboratory prototype of ACAR-Controller: a) the sustainable electric power distribution subsystem; b) the intelligent RGBY LED lighting subsystem; c) the non-invasive measuring subsystem of human reaction to comfort conditions in the laboratory; d) the ATMEGA128RFA1-ZU transceivers based wireless communication subsystem; e) the software for the ACAR- Controller.


I. INTRODUCTION
The variable air volume (VAV) type of heating, ventilating, and/or air-conditioning (HVAC) systems help to creating of users thermal and lighting comfort [1]- [3] in the sustainable home.LED lighting can also increase comfort and save energy in the sustainable home.There is more evidence that blue light (400 -500 nm) can damage our eyes, with people who have had cataracts removed being particularly vulnerable [2], [4].In this paper, the HVAC and RGBY LED lighting sub-models and elements of prototype of one room sustainable laboratory are developed to enlarging capabilities of the Ambient Comfort Affect Reward Based Laboratory Climate Controller (ACAR-Controller) proposed in [5].

II. ENLARGEMENT OF THE ACAR-CONTROLLER MODEL
The enlargement of the ACAR-Controller model of Fig. 1 is based on modification of the Ambient Comfort Affect Reward, the ACAR index function [5], as follows ACAR = f{a(l ci ,t a ,a ci ,t b ,c,d),v(l ci ,t a a ci ,t b ,c,d)} = [-3,3], (1)   Using reinforcement learning for control with continuous state and action space, a function approximation must be used [6], [7].A feature state consists of N features, each having an activation factor in the interval [0, 1].Linear approximation calculate their function value with: Modelling of Ambient Comfort Affect Reward Based Adaptive Laboratory Climate Controller 2 2 ( ) exp 2 ( ) ( ) ( ) ( ) where φ(x) is the activation function and w i is the weight of the feature i, ) (i s φ is dependent only on the distance between the state, s, the center state of the feature, c i and it is relative to the feature's width, σ i , r(t) is the external reinforcement reward signal, 0 < γ < 1 denotes the discount factor that is used to determine the proportion of the delay to the future rewards.The TD error indicates the goodness of the actual action [7], and the weight vector θ of the policy function and the value function are updated as: ( ) The Model of ACAR-Controller consists of the following parts: the Environment Evaluation System, the Radial Basis Neural Network, and the Learning Algorithm.The Environment Evaluation System is used to evaluate the ambient human comfort by sensing the influence of the following environment parameters: the Temperature, the Lighting, and the Air Conditioning.

III. HVAC SUB-MODEL OF A SINGLE ROOM LABORATORY
The model of [8] was used for representing a single zone HVAC system with a single heater and humidifier described by the following state-space equation , , where the state variable matrix Nomenclature given in [8] for the equations above are as follows: T he -temperature of air when it leaves the heater ( 0 C); T hu -temperature of air when it leaves the humidifier( 0 C); T out -temperature of air when it leaves the duct( 0 C); T w1 -temperature of wall 1 and 3( 0 C); T w2temperature of the wall 2 and 4 ( 0 C); T r -temperature of the roof ( 0 C); T z -temperature of the zone( 0 C); T o -outside air temperature ( 0 C); T m -temperature of mixed air ( 0 C); W hespecific humidity of air when it leaves the heater (kg/kg); W hu -specific humidity of air when it leaves humidifier (kg/kg); W z -specific humidity of the zone(kg/kg); W ospecific humidity of outside air (kg/kg); W m -specific humidity of the mixed air (kg/kg); C ah -thermal capacity of the air handling unit (kJ/C ); C h -thermal capacity of the humidifier (kJ/C ); C w1 -thermal capacity of the wall 1 and 3 (kJ/C ); C w2 -thermal capacity of wall 2 and 4 (kJ/C ); C rthermal capacity of roof (kJ/C ); C z -thermal capacity of the zone (kJ/C ); C pw -specific heat of water (kJ/kg);specific heat of air (kJ/kg ); V ah -volume of air handling unit; V h -volume of humidifier; V z -volume of the zone; f sw -flow rate of hot water (m 2 /s); f sa -flow rate of air (m 2 /s); UA a -overall transmittance area factor of the humidifier (kJ/sC); a h -overall transmittance area factor of the air handling unit (kJ/sC); ρ a 2 -density of the air (kg/m 2 ); ρ wdensity of water (kg/m 2 ); h(t) -rate of moisture air produced in the humidifier (kg/m 2 ); h i -heat transfer coefficient inside the duct (W/m 2 C); h o = -heat transfer coefficient in the ambient (W/m 2 C); m a = 6.04 kg/m -mass of the duct (kg/m); U w1 , U w2 , and U r -overall heat transfer coefficient of the walls 1 and 3, 2 and 4, and of the roof respectively (W/m 2 C); A w1 , A w2 , and A r -area of the wall 1 and 3, wall 2 and 4, and of the roof respectively (m 2 ); To verify the Matlab simulation of the state-space equations (7), the following values of the parameters of the HVAC sub model of one room laboratory were used: C ah = 4.5, C h = 0.63; C w1 = 70; C w2 = 60; C r = 80; C z = 47.1;C pw = 4.168; C pa = 1.005;V ah = 2.88; V z = 36; V z = 36; f sa = 0.192; UA a = 0.04; ρ a = 1.2; a h = 0.0183; ρ w = 998; h i = 8.33; h o = 16.6;m a = 6.04;U w1 = 2; U w2 = 2; U r = 1; A w1 = 18; A w2 = 9; A r = 18.

IV. LIGHTING SUB-MODEL IN A SINGLE ROOM LABORATORY
Modeling the lighting inside the room, the illumination inside the environment can be classified into two categories: artificial lighting and natural illumination.The light radiation is assumed to be uniform while, for light sources the following assumptions are as follows: a) artificial sources are approximated to point light sources; b) natural sources are considered as extended light sources.Given the room/lights and room/window dimensions, the above assumptions can be considered by using the expression (8) proposed in [2] ( ) ( ) ( ) where E a (P) -environment illumination at the point of interest P(x,y,z) (Lux), E(ND gl ) j -natural diffuse illumination on windows (Lux), E(NR gl ) j -natural reflection illumination on glass (Lux), E(NAT gl ) j -natural direct illumination on glass (Lux), C mc /C mr -diffuse/reflection illumination at the point of interest P(x,y,z) (Lux), I L -artificial light source (cd/klm), Lumen -luminous flux (lm), γ -incidence angle of the light radiation in the relation to the point of interest P(x,y,z) ( o ), ddistance between point of interest and light source (m), σ weighted -reflection coefficient of the walls, sum area -total area of the reflective walls A w1 = 2*18 + A w2 =2* 9 + A r = 18 = 63 (m 2 ), η -efficiency of the light source, and Menvironmental maintenance factor.

V. PRACTICAL IMPLEMENTATION OF ELEMENTS OF ACAR-CONTROLLER FOR THE SINGLE ROOM LABORATORY
Figure 2, Fig. 3 and Fig. 5 represent the following wireless communication elements of prototype of the ACAR-Controller for the single room laboratory: the sustainable electric power distribution subsystem (SEPDS) for measuring, sustainable control and delivering power to electric heater and fans by using triac type AC power controllers (Fig. 2).The intelligent RGBY lighting subsystem with dimmable current stabilizers for each RGBY power LED (Fig. 3).The noninvasive measuring subsystem of human reaction to comfort conditions in the laboratory (Fig. 5); the ATMEGA128RFA1-ZU transceivers based wireless communication subsystem which uses the 802.15.4 ZigBee low-power, short-distance wireless ISM standard for the 2.4GHz license-free radio bands (Fig. 2(c)).The calculations of RMS values of current, 2 /n) = 230.0V, apparent power, S apparent = V RMS *I RMS = 649.9VA, real power, P = Σ(v i C i v v C v /nG) = 628.119W, power factor, PF = P/S apparent = 0.9665, C i = 0.1206, C v = 0.75, G = 1|10|200, and n = 3000 for the 0,7 kW heater are performed by system software every 0.7 sec as shown in Fig. 4. The constants C v and C i were calculated by using values of voltmeter, V (not shown in Fig. 2  The sample error can be: between the values of Err max = 512*(sin (0.03925) = 20,09 (3.92%) and Err min = 512* sin (3.14/2) -(3.14/2-0.03925))= 0.461 (0.09% ) which is acceptable.

VII. CONCLUSIONS
The HVAC and RGBY LED lighting sub-models and elements of prototype of one room sustainable laboratory are developed to enlarging capabilities of the Ambient Comfort Affect Reward Based Laboratory Climate Controller (ACAR-Controller).
The validation of the model of ACAR-Controller is performed by implementation and testing of the electric power distribution, RGBY lighting, the ATMEGA128RFA1-ZU transceivers based wireless communication subsystems, and the module of noninvasive measuring of human reaction to comfort conditions in the laboratory.
Digital measurements of instantaneous values of samples of pairs of the mains voltage and current every 0.8 ms as well as calculations of the active, apparent power and the power factor are performed in real time every 0.7 sec by Atmega32 microcontroller for the following electric power consumers: the 0.7 kW heater, the computer monitor, and the refregerator.
The maximum sample error was less then or equal to 3.92% which is acceptable.
where a and v are arousal and valence functions respectively dependent on the following ambient comfort and human physiological parameters: l ci -the RGBY LED lighting index defined by the lighting sub-model of a single room laboratory, T z = t a -ambient temperature of the zone, and Wz = a ci -air conditioning index of specific humidity in the zone defined by the HVAC sub-model of the single room laboratory, t b -human body temperature, c -ECG, electrocardiogram and d -EDA, electro-dermal activity.We use fuzzy logic to approximate (1) by defining two fuzzy inference systems: the Arousal-Valence System, and the Ambient Comfort Affect Reward (ACAR) System.The Radial Basis Neural Network is the main component of the model responsible for two roles -the policy structure, known as the Actor, used to select actions, and the estimated value function, known as the Critic that criticizes the actions made by the Actor.

Fig. 1 .
Fig. 1.Block diagram of the model of the ACAR-Controller.

Fig. 2 .
The sustainable electric power distribution subsystem (SEPDS) for measuring, sustainable control and delivering power to electric heater and fans by using triac type AC power controllers: a) block diagram; b) and c) sub modules; d) Mega128RFA1 transceiver board.

Fig. 4 .
Fig. 4. C program block for implementation of measurement of current and voltage characteristics on Mega32-P.

Figure 6
Figure 6 depicts digital measurement results of output signals of VOLTAGE (VACP block of Fig. 2(a)) and CURRENT (ACP block of Fig. 2(a)) measured by Mega32-P board of Fig. 2(b) respectively: for voltage a) and current b) of 0,7 Kw heater, for voltage c) and current d) of the computer monitor, and for voltage e) and current f) of an refrigerator.The calculations of RMS values of current,I RMS = SQRT((Σ(v i C i /G) 2 /n) = 2825.6mA, voltage, V RMS = SQRT((Σ(v v C v )2 /n) = 230.0V, apparent power, S apparent = V RMS *I RMS = 649.9VA, real power, P = Σ(v i C i v v C v /nG) = 628.119W, power factor, PF = P/S apparent = 0.9665, C i = 0.1206, C v = 0.75, G = 1|10|200, and n = 3000 for the 0,7 kW heater are performed by system software every 0.7 sec as shown in Fig.4.The constants C v and C i were calculated by using values of voltmeter, V (not shown in Fig.2(a)), ampermeter, A, and watmeter, W, of Fig.2a.The values of power factor of monitor and refrigerator are as follows: PF monitor = 0.3437, and PF refregarator = 0.2598.Digital measurements of instantaneous values of k sample of the mains voltage (v k + v k+2 )/2 and current i k+1 taken by the load from the mains have been performed every 0.8 ms at 25 samples per 20 ms period or 14.4 o .Instantaneous active power value p k = ((v k + v k+2 )/2 )* i k+1 was calculated after 3 first type differential convertions of every adjacent sample that takes 25 ADC clock cycles at a maximum speed of convertion of 200 KHz each, and 25/200.10 3 = 0.125 ms or 6.28*0.125/20=0.03925radians/2.25degrees.
Figure 6 depicts digital measurement results of output signals of VOLTAGE (VACP block of Fig. 2(a)) and CURRENT (ACP block of Fig. 2(a)) measured by Mega32-P board of Fig. 2(b) respectively: for voltage a) and current b) of 0,7 Kw heater, for voltage c) and current d) of the computer monitor, and for voltage e) and current f) of an refrigerator.The calculations of RMS values of current,I RMS = SQRT((Σ(v i C i /G) 2 /n) = 2825.6mA, voltage, V RMS = SQRT((Σ(v v C v )2 /n) = 230.0V, apparent power, S apparent = V RMS *I RMS = 649.9VA, real power, P = Σ(v i C i v v C v /nG) = 628.119W, power factor, PF = P/S apparent = 0.9665, C i = 0.1206, C v = 0.75, G = 1|10|200, and n = 3000 for the 0,7 kW heater are performed by system software every 0.7 sec as shown in Fig.4.The constants C v and C i were calculated by using values of voltmeter, V (not shown in Fig.2(a)), ampermeter, A, and watmeter, W, of Fig.2a.The values of power factor of monitor and refrigerator are as follows: PF monitor = 0.3437, and PF refregarator = 0.2598.Digital measurements of instantaneous values of k sample of the mains voltage (v k + v k+2 )/2 and current i k+1 taken by the load from the mains have been performed every 0.8 ms at 25 samples per 20 ms period or 14.4 o .Instantaneous active power value p k = ((v k + v k+2 )/2 )* i k+1 was calculated after 3 first type differential convertions of every adjacent sample that takes 25 ADC clock cycles at a maximum speed of convertion of 200 KHz each, and 25/200.10 3 = 0.125 ms or 6.28*0.125/20=0.03925radians/2.25degrees.

Fig. 6 .
Fig. 6.Instantaneous values of output signals of VOLTAGE (VACP block of Fig. 2(a)) and CURRENT (ACP block of Fig. 2(a)) measured by Mega32-P board of Fig. 2(b) respectively: for voltage a) and current b) of 0,7 kW heater; for voltage c) and current d) of computer monitor; for voltage e) and current f) of refrigerator.