Improvement of PMSM Sensorless Control Based on Synergetic and Sliding Mode Controllers Using a Reinforcement Learning Deep Deterministic Policy Gradient Agent +

ABSTRACT:

The field‐oriented control (FOC) strategy of a permanent magnet synchronous motor (PMSM) in a simplified form is based on PI‐type controllers. In addition to their low complexity (an advantage for real‐time implementation), these controllers also provide limited performance due to the nonlinear character of the description equations of the PMSM model under the usual conditions of a relatively wide variation in the load torque and the high dynamics of the PMSM speed reference. Moreover, a number of significant improvements in the performance of PMSM control systems, also based on the FOC control strategy, are obtained if the controller of the speed control loop uses sliding mode control (SMC), and if the controllers for the inner control loops of id and iq currents are of the synergetic type. Furthermore, using such a control structure, very good performance of the PMSM control system is also obtained under conditions of parametric uncertainties and significant variations in the combined rotor‐load moment of inertia and the load resistance. To improve the performance of the PMSM control system without using controllers having a more complicated mathematical description, the advantages provided by reinforcement learning (RL) for process control can also be used. This technique does not require the exact knowledge of the mathematical model of the controlled system or the type of uncertainties. The improvement in the performance of the PMSM control system based on the FOC‐type strategy, both when using simple PI‐type controllers or in the case of complex SMC or synergetic‐type controllers, is achieved using the RL based on the Deep Deterministic Policy Gradient (DDPG). This improvement is obtained by using the correction signals provided by a trained reinforcement learning agent, which is added to the control signals ud, uq, and iqref. A speed observer is also implemented for estimating the PMSM rotor speed. The PMSM control structures are presented using the FOC‐type strategy, both in the case of simple PI‐type controllers and complex SMC or synergetic‐type controllers, and numerical simulations performed in the MATLAB/Simulink environment show the improvements in the performance of the PMSM control system, even under conditions of parametric uncertainties, by using the RL‐DDPG.

KEYWORDS:

1.      Permanent magnet synchronous motor

2.      Sliding mode control

3.      Synergetic control

4.      Reinforcement learning

5.      Deep neural networks

SOFTWARE: MATLAB/SIMULINK

BLOCK DIAGRAM:

 

Figure 1. Block diagram for FOC‐type control of the PMSM based on PI‐type controllers using RL.

EXPECTED SIMULATION RESULTS:

 

 

Figure 2. Time evolution for the numerical simulation of the PMSM control system based on the FOC‐type strategy.

Figure 3. Time evolution for the numerical simulation of the PMSM control system based on the RL‐TD3 agent for the correction of iqref.

Figure 4. Time evolution for the numerical simulation of the PMSM control system based on the

RL‐TD3 agent for the correction of udref and uqref.

Figure 5. Time evolution for the numerical simulation of the PMSM control system based on the

RL‐TD3 agent for the correction of udref, uqref, and iqref.

Figure 6. Time evolution for the numerical simulation of the PMSM control system based on control

using SMC and synergetic controllers.

Figure 7. Time evolution for the numerical simulation of the PMSM control system based on control

using SMC and synergetic controllers using an RL‐TD3 agent for the correction of iqref.

 

CONCLUSION:

This paper presents the FOC‐type control structure of a PMSM, which is improved in terms of performance by using a RL technique. Thus, the comparative results are presented for the case where the RL‐TD3 agent is properly trained and provides correction signals that are added to the control signals ud, uq, and iqref. The FOC‐type control structure for the PMSM control based on an SMC speed controller and synergetic current controller is also presented. To improve the performance of the PMSM control system without using controllers having a more complicated mathematical description, the advantages provided by the RL on process control can also be used. This improvement is obtained using the correction signals provided by a trained RL‐TD3 agent, which is added to the control signals ud, uq, and iqref. A speed observer is also implemented for estimating the PMSM rotor speed. The parametric robustness of the proposed PMSM control system is proved by very good control performances achieved even when the uniformly distributed noise is added to the load torque TL, and under high variations in the load torque TL and the moment of inertia J. Numerical simulations are used to prove the superiority of the control system that uses the RL‐TD3 agent.

REFERENCES:

1. Eriksson, S. Design of Permanent‐Magnet Linear Generators with Constant‐Torque‐Angle Control for Wave Power. Energies 2019, 12, 1312.

2. Ouyang, P.R.; Tang, J.; Pano, V. Position domain nonlinear PD control for contour tracking of robotic manipulator. Robot. Comput. Integr. Manuf. 2018, 51, 14–24.

3. Baek, S.W.; Lee, S.W. Design Optimization and Experimental Verification of Permanent Magnet Synchronous Motor Used in Electric Compressors in Electric Vehicles. Appl. Sci. 2020, 10, 3235.

4. Amin, F.; Sulaiman, E.B.; Utomo, W.M.; Soomro, H.A.; Jenal, M.; Kumar, R. Modelling and Simulation of Field Oriented Control based Permanent Magnet Synchronous Motor Drive System. Indones. J. Electr. Eng. Comput. Sci. 2017, 6, 387.

5. Mohd Zaihidee, F.; Mekhilef, S.; Mubin, M. Robust Speed Control of PMSM Using Sliding Mode Control (SMC)—A Review. Energies 2019, 12, 1669.

Improvement of PMSM Control Using Reinforcement Learning Deep Deterministic Policy Gradient Agent

ABSTRACT:

Based on the advantage of using the reinforcement learning on process control, provided by the fact that it is not necessary to know the exact mathematical model and the structure of its uncertainties, this article approaches the possibility of improving the performances of the Permanent Magnet Synchronous Motor (PMSM) control system based on the Field Oriented Control (FOC) type control strategy, by using the correction signals provided by a trained reinforcement learning agent, which will be added to the control signals ud, uq, and iqref . The type of reinforcement learning used is the Deep Deterministic Policy Gradient (DDPG). The combination possibilities of these control structures are presented, and their superiority over the FOC type control strategy is validated by numerical simulations.

KEYWORDS:

1.      Permanent magnet motors

2.      Field oriented control

3.      Reinforcement learning

4.      Intelligent agent

5.      Deep neural networks

SOFTWARE: MATLAB/SIMULINK

BLOCK DIAGRAM:

 

Fig. 1. Block diagram for FOC-type control of the PMSM based on reinforcement learning.

EXPECTED SIMULATION RESULTS:

Fig. 2. Time evolution for the numerical simulation of the PMSM control system based on the FOC-type strategy.

 

Fig. 3. Time evolution for the numerical simulation of the PMSM control system based on TD3 agent for the correction of udref and uqref .

 

Fig. 4. Time evolution for the numerical simulation of the PMSM control system based on TD3 agent for the correction of iqref.

 

Fig. 5. Time evolution for the numerical simulation of the PMSM control system based on TD3 agent for the correction of udref, uqref and iqref.

 

CONCLUSION:

This article presents the FOC-type control structure of a PMSM, which is improved in terms of performance by using a reinforcement learning technique. Thus, the comparative results are presented for the case where the reinforcement learning agent is properly trained and provides correction signals that will be added to the control signals ud, uq, and iqref. Numerical simulations are used to demonstrate the superiority of the control system that uses the reinforcement learning, and the following papers will study the possibilities for optimization in terms of the implementation of the reinforcement learning in the PMSM control.           

REFERENCES:

[1] B. Wu and M. Narimani, Control of Synchronous Motor Drives, in High-Power Converters and AC Drives , Wiley-IEEE Press, 2017, pp.353-391.

[2] B. K. Bose, Modern power electronics and AC drives, Prentice Hall, Knoxville, Tennessee, USA, 2002.

[3] H. Wang and J. Leng, “Summary on development of permanent magnet synchronous motor,” Chinese Control And Decision Conference (CCDC), Shenyang, China, 2018, pp. 689-693.

[4] Z. Liu, Y. Li, and Z. Zheng, “A review of drive techniques for multiphase machines,” in CES Transactions on Electrical Machines and Systems, vol. 2, pp. 243-251, June 2018.

[5] S. Sakunthala, R. Kiranmayi, and P. N. Mandadi, “A Review on Speed Control of Permanent Magnet Synchronous Motor Drive Using Different Control Techniques,”International Conference on Power, Energy, Control and Transmission Systems (ICPECTS), Chennai, China , 2018, pp. 97-102.