Set noise prior automatically about gpax HOT 3 CLOSED

This looks good, specially in the multi-fidelity case, choice of noise prior seemed more critical. However, few questions-

1. is it possible to make the value 0.2 somewhat learnable param in future or say doing parameter optimization?
2. How do you think it can handle if any outlier is present in training data? It will unnecessarily put a high variance which will impact the prediction. If we have a big range of y, we wont know whether the training sample is the outlier or a sample from region of interest (if we maximize), unless we run few BO iterations. Will it be good idea to always provide normalized y then in training? Let me know your thoughts

from gpax.

@arpanbiswas52 - good points.

is it possible to make the value 0.2 somewhat learnable param in future or say doing parameter optimization?

Note that 0.2 is used (together with measured data) to set variance in the prior distribution for noise and the noise itself is a learnable parameter. So we can think of 0.2 as an 'initial guess.' But in principle, yes, it can be learned by placing a (hyper)prior on it.

How do you think it can handle if any outlier is present in training data? It will unnecessarily put a high variance which will impact the prediction. If we have a big range of y, we wont know whether the training sample is the outlier or a sample from region of interest (if we maximize), unless we run few BO iterations. Will it be good idea to always provide normalized y then in training? Let me know your thoughts

Yes, the assumption (for all models) is that data went through basic preprocessing, with large or unphysical outliers removed. That said, using the suggested half-normal distribution can give us advantage since the noise level will be "pushed" towards zero unless the data strongly suggests otherwise.

from gpax.

We're going to keep the default log-normal priors for now

from gpax.