Multinomial Dirichlet Gaussian Process Model for Classification of Multidimensional Data

We present probabilistic multinomial Dirichlet classification model for multidimensional data and Gaussian process priors. Here, we have considered efficient computational method that can be used to obtain the approximate posteriors for latent variables and parameters needed to define the multiclass Gaussian process classification model. We first investigated the process of inducing a posterior distribution for various parameters and latent function by using the variational Bayesian approximations and important sampling method, and next we derived a predictive distribution of latent function needed to classify new samples. The proposed model is applied to classify the synthetic multivariate dataset in order to verify the performance of our model. Experiment result shows that our model is more accurate than the other approximation methods.

Authors:

• Wanhyun Cho
• Soonja Kang
• Sangkyoon Kim
• Soonyoung Park

Keywords:

• Multinomial dirichlet classification model
• Gaussian process priors
• variational Bayesian approximation
• Importance sampling
• approximate posterior distribution
• Marginal likelihood evidence.

References:

[1] C. E. Rasmussen, and C. K. I. Williams, Gaussian Processes for Machine
Learning, MIT Press, 2006.
[2] H. Nicklisch, and C. E. Rasmussen, “Approximation for Binary Gaussian
process Classification”, Journal of Machine Learning Research, vol 9, pp
2035-75, 2008.
[3] C. K. I. Williams, and D. Barber, “Bayesian Classification with Gaussian
Processes”, IEEE Tran. On PAMI, vol 12, pp 1342-1351, 1998.
[4] T. P. Minka, “Expectation Propagation for Approximate Bayesian
Inference”, In UAI, Morgan Kaufmann, pp 362-369, 2001.
[5] M. Opper, and O. Winther, “Gaussian Processes for Classification: Mean
Field Algorithms”, Neural Computation, vol 12 pp 2655-2204, 2000.
[6] L. Csato, E. Fokoue, M. Opper, and B. Schottky, “Efficient Approaches
to Gaussian Process Classification”, In Neural Information Processing
Systems, vol 12, pp 251-257, 2000.
[7] M. Girolami and S. Rogers, “Variational Bayesian Multinomial Probit
Regression with Gaussian Process Priors”, Neural Computation, vol 18
pp 1790-1817, 2006.