Guilherme B. Sousa^1*, Janes V. R.Lima¹, Patrícia H. M.Rêgo², Alain G.Souza³ and João V. FonsecaNeto⁴, ¹State University of Maranhão -UEMA, Brazil, ²State University of Maranhão -UEMA, Brazil, ³São José dos Campos, SP, Brazil and ⁴Federal University of Maranhão - UFMA, Brazil

This paper presents the design and investigation of performance of a 3-DOF Quanser helicopter system using a learning optimal control approach that is grounded on approximate dynamic programming paradigms, specifically action-dependent heuristic dynamic programming (ADHDP). This approach results in an algorithm that is embedded in the actor-critic reinforcement learning architecture, that characterizes this design as a model-free structure. The developed methodology aims at implementing an optimal controller that acts in real-time in the plant control, using only the input and output signals and states measured along the system trajectories. The feedback control design technique is capable of an online tuning of the controller parameters according to the plant dynamics, which is subject to the model uncertainties and external disturbances. The experimental results demonstrate the desired performance of the proposed controller implemented on the 3-DOF Quanser helicopter.

Action-Dependent Heuristic Dynamic Programming, Actor-Critic Reinforcement Learning, Real-Time Control, 3-DOF Helicopter