Open-loop Stackelberg learning solution for hierarchical control problems

摘要

This work presents a novel framework based on adaptive learning techniques to solve the continuous-time open-loop Stackelberg games. The method yields real-time approximations of the game value and convergence of the policies to the open-loop Stackelberg-equilibrium solution, while also guaranteeing asymptotic stability of the equilibrium point of the closed-loop system. It is implemented as a separate actor/critic parametric network approximator structure for every player and involves simultaneous continuous-time adaptation. To introduce and implement the hierarchical structure to the coupled optimization problem, we adjoin to the leader the controller dynamics of the follower. A persistence of excitation condition guarantees convergence of both critics to the actual game values that eventually solve the hierarchical optimization problem. A simulation example shows the efficacy of the proposed approach.