Comments (5)
Add the below type of functionality into an xgboost wrapper:
http://www.bigdatarepublic.nl/regression-prediction-intervals-with-xgboost/
https://www.snip2code.com/Snippet/1651850/Customized-loss-function-for-quantile-re
http://codegists.com/code/xgboost-regression/
e.g.
import numpy as np
def xgb_quantile_eval(preds, dmatrix, quantile=0.2):
"""
Customized evaluation function that equals
to quantile regression loss.
Quantile regression is regression that
estimates a specified quantile of target's
distribution conditional on given features.
@type preds: numpy.ndarray
@type dmatrix: xgboost.DMatrix
@type quantile: float
@return: float
"""
labels = dmatrix.get_label()
return ('q{}_loss'.format(quantile),
np.nanmean((preds >= labels) * (1 - quantile) * (preds - labels) +
(preds < labels) * quantile * (labels - preds)))
def xgb_quantile_loss(preds, dmatrix, quantile=0.2):
"""
Computes first-order derivative of quantile
regression loss and a non-degenerate
placeholder for second-order derivative.
Placeholder is returned instead of zeros,
because XGBoost requires non-zero
second-order derivatives. See this page:
https://github.com/dmlc/xgboost/issues/1825
to see why it is possible to use this trick.
However, be sure that hyperparameter named
`max_delta_step` is small enough to satisfy:
```0.5 * max_delta_step <=
min(quantile, 1 - quantile)```.
@type preds: numpy.ndarray
@type dmatrix: xgboost.DMatrix
@type quantile: float
@return: (numpy.ndarray, numpy.ndarray)
"""
try:
assert 0 <= quantile <= 1
except AssertionError:
raise ValueError("Quantile value must be float between 0 and 1.")
labels = dmatrix.get_label()
errors = preds - labels
left_mask = errors < 0
right_mask = errors > 0
grad = -quantile * left_mask + (1 - quantile) * right_mask
hess = np.ones_like(preds)
return grad, hess
# Example of usage:
# bst = xgb.train(hyperparams, train, num_rounds,
# obj=xgb_quantile_loss, feval=xgb_quantile_eval)
How done in scikit:
http://scikit-learn.org/dev/auto_examples/ensemble/plot_gradient_boosting_quantile.html#example-ensemble-plot-gradient-boosting-quantile-py
scikit-learn/scikit-learn#9978
How h2o-3 does it:
http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/algo-params/distribution.html
How lightgbm does it:
microsoft/LightGBM#1036 (comment)
Example use on kaggle:
https://www.kaggle.com/c/allstate-claims-severity/discussion/26440
from h2o4gpu.
Can be used in xgb glm
from h2o4gpu.
I don't see why this was closed.
from h2o4gpu.
We still manage xgboost and just because xgboost related doesn't mean we don't have work to do.
from h2o4gpu.
Sure, sounds good. Sorry for the premature closing of the issue.
from h2o4gpu.
Related Issues (20)
- R random_forest_classifier problem with high dimensions HOT 2
- Support setuptools > 39.1.0
- KMeans on 4 gpus HOT 1
- If i run the below command i am getting attached error. HOT 2
- dependencies incompatibilities between h2o4gpu and cudf HOT 7
- Lightgbm hangs sometimes HOT 4
- take of dask or cupy arrays as input for logistic regression, elastic_net, ridge and linear_regression
- Low GPU utilization during Random Forest predict
- Posted to wrong project
- LogisticRegression default solver is outdated
- LogististicRegression default multi_class parameter is outdated
- get_gpu_info_c not detecting any GPU (compatibility issue with CUDA 11?) HOT 2
- AttributeError: 'NoneType' object has no attribute 'make_ptr_double' HOT 4
- Logistic Regressions predict_proba returns predictions, not probabilities
- install_github failing
- Elastic Net Regression in R HOT 1
- AMD gpu HOT 1
- Installation of h2o4gpu on Kaggle using reticulate package HOT 1
- NameError: name 'output_margin' is not defined
- H2o4GPU build Failed
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from h2o4gpu.