GPC-MA builds on top of the popular GPML Matlab toolkit for Gaussian processes by giving it the support to handle data from multiple annotators and Crowds, thereby allowing the estimation of the reliability of the different annotators as well as finding better estimates of the (unobserved) ground truth labels when compared to standard GP classification or majority-voting-based approaches. See the original paper for further details:

Rodrigues, F. and Pereira, F.C. and Ribeiro, B., Gaussian Process Classification and Active Learning with Multiple Annotators, in proceedings of the International Conference on Machine Learning (ICML), 2014.

The tar.gz with the source code can be obtained here.

The datasets (from Amazon’s Mechanical Turk) used in the paper are also available here for download.

DOWNLOAD:

Source code // Datasets

CONTACT:

Please send questions and comments to fmpr [at] dei.uc.pt