A branching particle system approximation for solving partially observed stochastic optimal control problems via stochastic maximum principle

Wan, Hexiang1; Wang, Guangchen1; Xiong, Jie2,3

2023-03-01

10.1007/s40072-023-00294-w

STOCHASTICS AND PARTIAL DIFFERENTIAL EQUATIONS-ANALYSIS AND COMPUTATIONS   Impact Factor & Quartile

2194-0401

2194-041X

This paper develops an efficient numerical algorithm for solving a class of partially observed stochastic optimal control problems with correlated noises. The main contribution of this paper is threefold: first, we introduce a relaxed system and assume the Roxin condition (convexity requirement) on coefficients. Then, an optimal relaxed system provides an optimal admissible control in a broader sense, and a relaxed control turns out to be a usual admissible control. Second, we transform the optimal control problem into an optimization problem for a convex functional by employing a projection operator. A stochastic gradient descent approach is then proposed and its convergence properties are demonstrated. Last but not least, we present a branching particle system (branching particle filter) to approximate the optimal filter. Due to the random nature of the coefficients in the Zakai equation, neither the dual approach nor the mild solution approach can be used. We devise a novel method for establishing the convergence of the branching particle system approximation, as well as its rate of convergence. This branching-type particle filter algorithm allows us to tackle non-Markovian environments. The major body of this paper concludes with a numerical case study.

Zakai equation Branching particle system Forward-backward stochastic differential equation Optimal control of partially observed system Maximum principle stochastic gradient decent

SCI

English

Corresponding

NSFC["11831010","2022YFA1006103","2022YFA1006102"] ; NSF of Shandong Province["61925306","61821004"] ; null[ZR2019ZD42] ; null[ZR2020ZD24]

Mathematics

Mathematics, Applied ; Statistics & Probability

WOS:000955853200001

SPRINGER

Web of Science

1.Shandong Univ, Sch Control Sci & Engn, Jinan 250061, Peoples R China
2.Southern Univ Sci & Technol, Dept Math, Shenzhen 518055, Peoples R China
3.Southern Univ Sci & Technol, SUSTech Int Ctr Math, Shenzhen 518055, Peoples R China

Wan, Hexiang,Wang, Guangchen,Xiong, Jie. A branching particle system approximation for solving partially observed stochastic optimal control problems via stochastic maximum principle[J]. STOCHASTICS AND PARTIAL DIFFERENTIAL EQUATIONS-ANALYSIS AND COMPUTATIONS,2023.

Wan, Hexiang,Wang, Guangchen,&Xiong, Jie.(2023).A branching particle system approximation for solving partially observed stochastic optimal control problems via stochastic maximum principle.STOCHASTICS AND PARTIAL DIFFERENTIAL EQUATIONS-ANALYSIS AND COMPUTATIONS.

Wan, Hexiang,et al."A branching particle system approximation for solving partially observed stochastic optimal control problems via stochastic maximum principle".STOCHASTICS AND PARTIAL DIFFERENTIAL EQUATIONS-ANALYSIS AND COMPUTATIONS (2023).