LinearForestRegressor may give biased coefficients for base estimator about linear-tree HOT 3 CLOSED

Hi,
Did you try using LinearForestRegressor inside a pipeline with at the top a StandardScaler like:

from sklearn.pipeline import make_pipeline
model = make_pipeline(StandardScaler(), LinearForestRegressor(...))

from linear-tree.

Hi, Did you try using LinearForestRegressor inside a pipeline with at the top a StandardScaler like:

from sklearn.pipeline import make_pipeline
model = make_pipeline(StandardScaler(), LinearForestRegressor(...))

Thank you for your reply! I have tried the pipeline and it worked!

I've seen the lase case on the repository homepage that linear forest can be used to resolve the extrapolation issues of random forest. After I used the pipeline, I found that the R-squared of linear forest on the test set (~0.65) is lower than random forest (0.70). Is this commonly seen in the regression problems? How can I improve the fitness of linear forest or does it mean that there is a trade-off between the fitness and the extropolation ability?

Thank you!

from linear-tree.

Finding the trade-off between the predictive and the extrapolation ability is one of the hardest tasks in the ML ecosystem. Some models are good for maximizing accuracy, others to extract explicative insights. There is no silver bullet for this kind of problem. You should make the proper choices according to your data and needs.
All the best

from linear-tree.