Abstract—This paper presents a novel online learning algorithm, in an actor-critic structure, to find state-feedback optimal controllers for partially unknown nonlinear systems. The algorithm converges online to the optimal solution under the condition of initial stabilizing controller. It is derived from integral reinforcement learning (IRL) technique, and makes use of semi-parametric regression model (SPRM) to approximate the optimal controller and the optimal cost function of a control dynamic system. The convergence to the optimal controller is proven, and the stability of the closed-loop nonlinear system is also guaranteed. The feasibility of the proposed learning algorithm is demonstrated in simulation on two example systems.
Index Terms—Policy iteration, optimal control, actor-critic structure, SPRM, nonlinear systems.
The authors are with the Automation Department, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016 China (e-mail: sunjingliangac@163.com. liuchsh@nuaa.edu.cn, liunian@nuaa.edu.cn).
Cite: Jingliang Sun, Chunsheng Liu, and Nian Liu, "Policy Iteration Based Optimal Control for Partially Unknown Nonlinear Systems via Semi-parametric Regression Model," International Journal of Machine Learning and Computing vol. 6, no. 3, pp. 172-178, 2016.