cknd / pyesn Goto Github PK

I've been using pyESN for a few weeks now, and I've found it to be very helpful. Thank you!

I've been using it in conjunction with a Keras network, and I was thinking that it could be really handy to turn this into a Keras layer. Keras encourages making your own layers, so I don't think it would be too technically challenging. I don't have a lot of experience with object-oriented Python, but I'd be willing to take a shot at it.

My initial thinking is to create a Reservoir layer that wraps the current pyESN class and then use Keras's existing Dense layers for input and output weights.

Would this be a good addition to this project? Is there anything else I should consider?

In the second cell of freqgen-mine.ipynb , rescaling periods is wrong.

periods = frequency_control * (max_period - min_period) + max_period

should be changed as follows:

periods = frequency_control * (max_period - min_period) + min_period

Is there anywhere in the code the leaking parameter (a) in order to calculate the states of the network?

Thank you.

How to configure the network to perform classification?

Hi,
I just don't understand why we use the date produced by np.ones(future) to predict future ? Intuitively, there is no connection between np.ones(future) data and future data .

Hi Clemens I thought your code was pretty cool. I used it here to create a crawler that is predicting a three body problem with measurement error noise here and a few other time series. It appears on the overall leaderboard and other leaderboards with the nom de plume "Exactable Fox".
-Peter

I can't dump a trained ESN model to disk. Could you please help me?

I got this error:
PicklingError: Can't pickle <function ESN.<lambda> at 0x1a749bb70>: it's not found as ml.train.pyESN.ESN.<lambda>

I tried to use for multivariate forecast but got ValueError: arg must have length 3 since I inserted the n_inputs=3 to train the model

Hello Clemens, thank you for your ESN implementation, which really helps me a lot in my research. Youuse np.linalg.pinv in your evaluation of W_out at line 192, pyESN.py, which can make it unstable. When I tried to transplant it from Numpy (default float type double) to PyTorch (default float type float32), the network cannot run correctly . It may be better to use self.W_out = np.linalg.lstsq(extended_states[transient:, :], self.inverse_out_activation(teachers_scaled[transient:, :])).T or self.W_out = np.linalg.solve(extended_states[transient:, :], self.inverse_out_activation(teachers_scaled[transient:, :])).T. Thanks!

Hi,

at first, I'd like to thank you for the nice code.

I think I found a small error at line 188:

transient = min(int(inputs.shape[1] / 10), 100)

this is always 0 (for 1-dimensional inputs) as the time the dimension in the inputs is the 0-th one. So I think it should be 0 instead of 1.

The funny thing is that the performance gets worse if you introduce a transient. This is weird.

Hello,

Thanks for the work I've been looking for an echo state network that is easy to use for awhile now. Anyway, whenever I use the predict function 0. is the last prediction value. You can see it in your Mackey glass example if you print prediction[-1]. It seems like it was a possible prediction value in that time series, but whenever I use another time series the last prediction continues to be 0.

Am I wrong here? Can you replicate this problem? Thanks again, your echo state networks are giving me good results and are very easy to use.

Theoretically, ESN model could be trained incrementally online. However, It seems that pyESN model does not behavior as expected when I tried to fit in two bathes as compared to fit in a whole batch. Could you please tell me if pyESN model support fitting incrementally or not? Thanks for your excellent work.

Hi,

I'm a little confused about how the ESN actually trains and predicts. In the Mackey-Glass example training is done in the following way:
pred_training = esn.fit(np.ones(trainlen),data[:trainlen])

This seems like I only give the inputs?
I guess the np.ones gives the bias term, but shouldn't I give the inputs and the desired outputs? I mean something like esn.fit(data[:trainlen], data[1:trainlen+1]) for 1 step prediction. ( I think something like this is done in freqgen.ipynb, but correct me if I'm wrong.)

In the Mackey-Glass example for prediction you only give np.ones(future) as input. Does this mean the network starts with the last training sample and then starts to generate the timeseries by feeding the output back as input?

When n_reservoir == trainlen, there is a singularity in the mean squared error. I'm not sure if this is a "feature" of echo state networks or if this is a true bug. In the attached figure, n_reservoir == trainlen == 1000.