Feature request: Allow the training set to be passed to custom error metrics about skforecast HOT 2 OPEN

Hi @JavierEscobarOrtiz !

I wonder if you will be able to do this using a ForecasterEquivalentDate + a backtesting with a custom metric 🤔 (But it is true that this is more labor-intensive for the user)

I agree that this would be very laborious for a user for what is a relatively common error metric.

I also want to add that it would be nice to compute error metrics which are pooled over multiple time series. For example the normalised deviation and normalised RMSE. See the definitions here:

https://arxiv.org/pdf/1704.04110.pdf

These metrics are recommended in this review paper: https://link.springer.com/article/10.1007/s10618-022-00894-5

So it would be nice to be able to compute them.

Thank you!
Kishan

from skforecast.

Hello @KishManani

Thanks for opening the issue! It is a great point to give the users more flexibility.

I wonder if you will be able to do this using a ForecasterEquivalentDate + a backtesting with a custom metric 🤔 (But it is true that this is more labor-intensive for the user)

What do you think @JoaquinAmatRodrigo ?

https://skforecast.org/latest/user_guides/forecasting-baseline

https://skforecast.org/latest/user_guides/backtesting#backtesting-with-custom-metric

from skforecast.