Comments (9)
Hello Paul,
Thank you for your answer.
I totally understand your point about dimension reduction, but I was more interested in observing the contribution of the derivatives.
I chose a relatively small size of the training sample on purpose: I wanted to measure the contribution of the derivatives when the data is scarce. I hoped that the derivatives could compensate the data scarcity to some extent.
I tried to double the size of the training sample for the 2-dimensional Rosenbrock function: KPLS becomes more accurate than GEKPLS.
It's counterintuitive to me that the derivatives worsen the accuracy. (Maybe it comes from ill-conditioning.)
Thank you for the 5-dimensional example. I observe that we gain only 5% of accuracy at the cost of computing 50 gradients.
I take note that GEKPLS requires an effective dimension reduction.
Cheers,
Benoît
from smt.
Hi Benoit. Maybe rosenbrock is not the best function to make GEK shine. The surface changes smoothly and plain kriging fits it basically pretty well and derivatives does not bring much more information (at least it does not worsen the prediction 😅 ). I would say that benefits should be seen with a function with more strong changes between training points.
from smt.
Hi Rémi,
Thank you for your answer.
On the figure attached to my previous message we can see that the median of the relative L2-errors of both KPLS and GEKPLS is about 50%. So neither KPLS nor GEKPLS seem to fit the Rosenbrock function pretty well.
You will find below a graph of the KPLS model that achieves median accuracy and the GEKPLS model trained at the same inputs (in orange, they look almost the same) and the Rosenbrock function (in blue).
As you can see the KPLS model does not fit the Rosenbrock function very well, there is clearly room for improvement. Adding the derivatives to the training data with GEKPLS does not really improve the accuracy.
Isn't it surprising? I would expect the derivatives to bring a lot of information, especially for such a smooth function.
Thank you for your time,
Benoît
from smt.
We have detected some problems in GEKPLS in SMT GitHub master, not present in SMT 1.3. Do you use SMT 1.3?
from smt.
Yes, I use SMT 1.3.
from smt.
Hello Benoit,
-
KPLS and GEKPS are meant to reduce the dimension of the problem. Here, you chose n_comp=2 but Rosenbrock(ndim=2). Therefore, the PLS matrices are just the identity matrix with numerical errors for both KPLS and GEKPLS.
-
You chose to build a model with only 10 points (sample_size=10). But the Rosenbrock function grows quick on the sides. Therefore, if you have no points in that zones, the model just can't predict it.
=> What you are comparing is therefore a prediction error related to the sample size and some noise from an ill-conditioned PLS matrix.
If you chose a reduction of the model size from Rosenbrock(ndim=5) to n_comp=2 and if you chose a sample size of 50 points you would obtain the result below.
PredictionAccuracy(Rosenbrock(ndim=5), validation_size=1000).plot(
sample_size=50,
number_of_trainings=20,
n_comp=2,
n_start=20,
So GEKPLS helps but need a certain amount of points and an effective reduction of the dimension for the model.
from smt.
Hi,
I totally understand your point. What you want to compare is Kriging vs Gradient-Enhanced Kriging (GEK). In this case, we should obtain better performances with GEK, you are totally right !
Unfortunately, GEK is not implemented in SMT (it has a lot of limitations for a large number of points or an high-dimensional problem).
You are also right for the bad accuracy, as we add more badly approximate direction with GEKPLS, the numerical errors should add up and the accuracy decrease.
In fact, GEKPLS is based on the PLS-reduced principal components derivatives. In this case, not only the PLS directions are ill-conditioned but it do not corresponds to GEK as GEK do not use dimension reduction at the price of being more expensive.
https://arxiv.org/pdf/1708.02663.pdf
from smt.
Thank you Paul for the explanation.
Cheers,
Benoît
from smt.
Thanks Paul!
from smt.
Related Issues (20)
- data-driven Multi-Fidelity Kriging HOT 1
- ValueError by passing an int instead of a float HOT 3
- SMT 2.5 is not available on conda-forge HOT 2
- Error from SMT --> ValueError: setting an array element with a sequence. HOT 4
- Question on new syntax to train GENN (JENN) models HOT 6
- Meta: add "Discussions" to GitHub repo? HOT 1
- Saving models without using pickle? HOT 1
- Number of parameters can be handled by KPLSK HOT 3
- Deprecation warning HOT 1
- Document mapping between function arguments and equations HOT 2
- Hessian computation/Compatibility with dual numbers HOT 4
- SIGBUS (Misaligned Address Error) HOT 1
- negative axis 1 index
- Training two samples take ~480 seconds HOT 3
- OOB access in RMTB HOT 1
- SGP Gradients unsupported HOT 1
- Relevance of the warning `R is too ill conditioned...` HOT 3
- ConfigSpace vs Numpy 2.0 compatibility HOT 1
- Document parameters to improve accuracy of SGP
- Recommendation for surrogate optimization
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from smt.