Bayesian Optimization over Mixed Type Inputs with Encoding Methods

Liu，Zhihao; Ou，Weiming; Wang，Songhao

10.1007/978-3-031-33377-4_16

2023

0302-9743

1611-3349

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

13936 LNCS

203-215

Traditional Bayesian Optimization (BO) algorithms assume that the objective function is defined over numeric input space. To generalize BO for mixed numeric and categorical inputs, existing approaches mainly model or optimize them separately and thus cannot fully capture the relationship among different types of inputs. The complexity incurred by additional operations for the categorical inputs in these approaches can also reduce the efficiency of BO, especially when facing high-cardinality inputs. In this paper, we revisit the encoding approaches, which transfer categorical inputs to numerical ones to form a concise and easy-to-use BO framework. Specifically, we propose the target mean encoding BO (TmBO) and aggregate encoding BO (AggBO), where TmBO transfers each value of a categorical input based on the outputs corresponding to this value, and AggBO encodes multiple choices of a categorical input through several distinct ranks. Different from the prominent one-hot encoding, both approaches transfer each categorical input into exactly one numerical input and thus avoid severely increasing the dimension of the input space. We demonstrate that TmBO and AggBO are more efficient than existing approaches on several synthetic and real-world optimization tasks.

First ; Corresponding

English

2-s2.0-85163311181

Scopus

SUSTech Business School,Southern University of Science and Technology,Shenzhen,518055,China

Liu，Zhihao,Ou，Weiming,Wang，Songhao. Bayesian Optimization over Mixed Type Inputs with Encoding Methods[C],2023:203-215.