Sparse Local Patch Transformer for Robust Face Alignment and Landmarks Inherent Relation Learning

Jiahao Xia1; Weiwei Qu2; Wenjian Huang2; Jianguo Zhang2; Xi Wang3; Min Xu1

10.1109/CVPR52688.2022.00402

2022

Conference on Computer Vision and Pattern Recognition (CVPR)

1063-6919

978-1-6654-6947-0

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

2022-June

4042-4051

18-24 June 2022

New Orleans, LA, USA

10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA

IEEE COMPUTER SOC

Heatmap regression methods have dominated face alignment area in recent years while they ignore the inherent relation between different landmarks. In this paper, we propose a Sparse Local Patch Transformer (SLPT) for learning the inherent relation. The SLPT generates the representation of each single landmark from a local patch and aggregates them by an adaptive inherent relation based on the attention mechanism. The subpixel coordinate of each landmark is predicted independently based on the aggregated feature. Moreover, a coarse-to-fine framework is further introduced to incorporate with the SLPT, which enables the initial landmarks to gradually converge to the target facial landmarks using fine-grained features from dynamically resized local patches. Extensive experiments carried out on three popular benchmarks, including WFLW, 300W and COFW, demonstrate that the proposed method works at the state-of-the-art level with much less computational complexity by learning the inherent relation between facial landmarks. The code is available at the project website(1).

Face And Gestures Deep Learning Architectures And Techniques

Corresponding

English

EI ; CPCI-S

program of China Scholarships Council[202006130004]

Computer Science ; Imaging Science & Photographic Technology

Computer Science, Artificial Intelligence ; Imaging Science & Photographic Technology

WOS:000867754204030

20224613120153

IEEE

1.Faculty of Engineering and IT, University of Technology Sydney
2.Dept. of Comp. Sci. and Eng., Southern University of Science and Technology
3.CaimCar

Jiahao Xia,Weiwei Qu,Wenjian Huang,et al. Sparse Local Patch Transformer for Robust Face Alignment and Landmarks Inherent Relation Learning[C]. 10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA:IEEE COMPUTER SOC,2022:4042-4051.