julesbelveze / time-series-autoencoder Goto Github PK

where is the dataset？I would appreciate it if you can provide it for me and My mail is [email protected].

How do I get y_hist when I try to run this model in real time

Hello, I am interested in using your code to remove noise from a set of time series (reconstruction). I would like to confirm the following: the series are divided (in temporal order) into the first 80% for training and the remaining 20% for testing, correct? So, if I wish to remove noise from one (or several) complete time series, how should I input the data? Regarding this, I have another question: I see that the reconstruction example applies training and evaluation in the training, how would it be to train it and then only test it (I don't see an example where only the evaluation is applied)? One last question: does the denoising process take into account the interactions between the different series, or is it carried out independently for each one? Thank you very much in advance!

Thank you for this great repo!

I am going to use this for classification and 3d representation of my numerical and time-series data. Could you please give me a few tips to achieve this goal?

I have seen it is relatively easy to use tensorboard for PCA/tSNE representations. Do you have any suggestions about that? I should add tensorboard to this repo for such visualization? or you have better suggestions...

Hi, Jules. Here is a little problem when I was trying to test the examples in this repo. I found it can be a little bit difficult to get the dataset(due to some annoying reason...maybe the network, or the expired url, I dont know). So, if it's possible could you please add the correspond dataset in this repo? I would be very appreciate if you are ok with that~. Thanks any way.

i can't understand that AttnDecoder return the 'self.fc_out(torch.cat((h_t[0], context.to(device)), dim=1))' . i can‘t find the correlation between the 'h_t' and 'context',plz tell me. Thank you!!!!!!!!!!!!!!!!

I get error when the output_size isn't equal to 1 in forecasting code.
for example when the output_size is 10. I get this error :

return F.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (16x65 and 74x10)

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

How is this code licensed? As it stands, the original author holds the copyright. Anyone using this repository without explicit permission is on shakey legal grounds.

When I evaluate the model during training, it get a decent reconstruction result and loss value finally. But when I use the same data set to test the trained model, the results are quite different from what I got during the training. What is the right way to do the evaluation after finishing training with the saved model? Right now, I just add the ckpt path to the YAML file, and it starts to evaluate without training. Where else should I adjust, or what else should I do?

Downloaded NFLX data from 2014 - 2021. Trained the model using default parameters. And the forecasting is wrongly scaled.

The NFLX stock value is around ~$515 on July 22, 2021, but the prediction value is around ~$20. Attached is the below graph for test and pred.

Below is the values for test and pred.

Pred: tensor([20.5919, 20.5882, 20.6023, 20.5925, 20.5604, 20.5682, 20.5184, 20.4771,
20.4323, 20.3718, 20.3013, 20.2565, 20.2335, 20.1562, 20.0954, 20.0564])

Test: tensor([492.4100, 498.3400, 500.7700, 497.0000, 508.8200, 512.7400, 518.0600,
527.0700, 533.0300, 533.5000, 528.2100, 533.5400, 533.9800, 541.6400,
535.9600, 530.7600])

ValueError: Index Date_Time invalid

“feature, y_hist, target = batch” in the "train.py" raised error "ValueError: not enough values to unpack (expected 3, got 2)",because "batch" only has two values,so what does "y_hist" mean? Sorry I just know a little about Attention ,I just want to run first.

Thanks for sharing the code, i learned a lot from it.
I see that in eval.py evaluation is performed on the target scaled with StandardScaler
I think that evaluation will decrease the actual mse.
I'm new to time series forecasting and don't know if it's reasonable to evaluate it on standardized data.

• Find a toy dataset for forecasting in order to provide an example
• Do the same for reconstruction purposes.

Hi,

As I understand well you create y_hist in order to perform training using teacher forcing by providing the true target value to the Decoder. As an argument to the forward Decoder function, you only pass y_hist. I cannot understand why you do not pass also the Encoded Vector as the final hidden state produced from the encoder part of the model.

Thanks

Traceback (most recent call last): File "main.py", line 101, in run(vars(args)) File "main.py", line 96, in run train(train_iter, test_iter, model, criterion, optimizer, config) File "/home/jupyter/time-series-autoencoder/train.py", line 44, in train output = model(feature.to(config["device"]), y_hist.to(config["device"])) File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/home/jupyter/time-series-autoencoder/model.py", line 254, in forward outputs = self.decoder(encoder_output, y_hist.float()) File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/home/jupyter/time-series-autoencoder/model.py", line 222, in forward dim=1)) # (batch_size, out_size) RuntimeError: Tensors must have same number of dimensions: got 2 and 1

What could be the reason of the following error

Here are my args
python main.py --batch-size 2 --output-size 2 --label-col T --input-att true --temporal-att true --seq-len 5 --hidden-size-encoder 5 --hidden-size-decoder 5 --reg-factor1 0.1 --reg-factor2 0.2 --reg1 true --reg2 true --denoising true --do-train true --do-eval true --data-path data.csv --output-dir output_dir

Hello, I have a question. Can this code be used to perform prediction with 8 features and 3 outputs, where each output depends on the 8 features

AssertionError if prediction window > 1.

torch==1.4.0

Traceback (most recent call last):

  File "<ipython-input-96-df6f6f907e9b>", line 107, in <module>
    run(vars(args))

  File "<ipython-input-96-df6f6f907e9b>", line 90, in run
    train_iter, test_iter, nb_features = ts.get_loaders(batch_size=config["batch_size"])

  File "/Users/rmehta/fin/AI4Fin/time-series-autoencoder/tsa/dataset.py", line 84, in get_loaders
    train_dataset = self.frame_series(X_train, y_train)

  File "/Users/rmehta/fin/AI4Fin/time-series-autoencoder/tsa/dataset.py", line 70, in frame_series
    return TensorDataset(features_var, y_hist_var, target_var)

  File "/opt/anaconda3/envs/rlf/lib/python3.7/site-packages/torch/utils/data/dataset.py", line 158, in __init__
    assert all(tensors[0].size(0) == tensor.size(0) for tensor in tensors)

AssertionError

Thanks @JulesBelveze for your intuitive work on LSTM Autoencoders.
I had one doubt.
IN your model.py file, you dont mention batch_first=True in your LSTM initialization in Encoder and Decoder class both.
But in your forward function, you iterate like inp[:, t, :] for t in range of (seqlen)
According to documentation in Pytorch, the dimensions to the input of LSTM should be (seqlen, batchsize, inputdims) if batch_first flag is False(Which is the default case)

So aren't you iterating in batch dims rather than doing it in seqlen dims? Please correct me if I am wrong.

Shouldn't it be inp[t, :, :] for t in range(seqlen) ?

Or you are transposing the input matrix before passing?

Thanks

The model worked fine for Netflix ticker but when I run for some other ticker, the model doesn't seem to learn. Or it learns with large errors.

Can you try to predict for the following attached dataset? Also, please attach the config as well.

https://drive.google.com/file/d/18lJk5JouTRomHV2wli99uUL3qWG1N8Vy/view?usp=sharing

My configs:

label_col=["close"],  # name of target column
index_col="date",

output_size=1,  # for forecasting

num_epochs=200,
batch_size=64,
lr=3e-4,
reg1=True,
reg2=False,
reg_factor1=1e-4,
reg_factor2=1e-4,
seq_len=20,  # previous timestamps to use
prediction_window=1,  # number of timestamps to forecast
hidden_size_encoder=128,
hidden_size_decoder=128,
input_att=True,
temporal_att=True,
denoising=True,
directions=1,

max_grad_norm=0.1,
gradient_accumulation_steps=1,
logging_steps=100,
lrs_step_size=5000,

I followed the example for forecasting. I kept the prediction window as 2 to forecast 2 timesteps. It is impossible to calculate the MSE in forecasting as you won't have the values for the future timesteps. How is the forecasting done here? Do we drop the last n timestep values and use the previous data to forecast n timesteps?

If yes, how do we not remove the previous values and forecast the data using all observations for 'n' timesteps?

Hi, nice project.
Thanks for sharing.
I am looking around and testing.
It seems get_loaders function does not fit run_reconstruction.
In particular the following line:

X_train, X_test, y_train, y_test = self.preprocess_data()

it is not correct for reconstruction. For reconstruction we do not need y values.