Automatic relevance determination of categorical variables for mixed variables Gaussian process regression.

Categorical variables combined with continuous variables are gaining interest in multiple applications of Gaussian processes. Different covariance functions can be used to handle categorical variables, but we found in a previous study that one of the most versatile was initially proposed in an optimization method called COCABO. Choosing an adequate prior when using a Gaussian process regression can drastically improve the regression performances. We propose in this paper a modification to the COCABO covariance function to allow the Gaussian processes to automatically find the relevance of each categorical variable. This mechanism, is inspired by Automatic Relevance Determination (ARD) for the continuous variables, can effectively be used with covariance functions that use Hamming distance to treat categorical variables. While we use the Categorical-ARD (CATARD) mechanism with the CoCaBO covariance function, it can be generalized to every covariance function that does not relax categorical variables to calculate the covariance. We used both synthetic benchmarks and real world data in order to establish the performances of the CATARD covariance function.