z331565360 / state-frequency-memory-stock-prediction Goto Github PK

great work and great architecture,but I think for ( financial ) time-series ,DFT or FFT is not enough,because such kind of transform does not provide time-frequency information at the same time . freqs can be obtained but for a certain length time-series ,DFT or FFT do not provide any infornation when are the main freqs happening or when vanished

so I have been thinking about using traditional way of signal processing ,wavelet-transform or EMD-HHT,which provide clear information of time and instantaneous freqs.but it's hard to combine signal processing procedures into a RNN ,may be 2-D LSTM will work better,1st-D for time information and 2nd-D for freqs .

second ,since CNN has really good achievement for classification ,so I guess add some CNNs as autoencoders might improve result

btw ,according to the experiments of the article , SFM have learned the capability of predicting time-series somehow 20 steps away ,but there are no information about how many steps been sent into SFM ,so can you give some hints?

there is something wrong with training, why is not have the size of windon hyperparameter?

the time_step of LSTM is only one?

"Exception: When using TensorFlow, you should define explicitly the number of timesteps of your sequences. If your first layer is an Embedding, make sure to pass it an "input_length" argument. Otherwise, make sure the first layer has an "input_shape" or "batch_input_shape" argument, including the time axis. Found input shape at layer itosfm_1: (None, None, 1)"
Anyone got idea about this?

I set up the environment with the right versions of keras and theano and after fixing some minor issues I am getting the below error when running the command "python test.py --step=1" and I have no idea how to fix it ..

File "test.py", line 50, in
model = build.build_model([1, hidden_dim, 1], freq, 0.01)
File "..\test\build.py", line 55, in build_model
return_sequences=True))
File "C:\Anaconda3\envs\MyEnv\lib\site-packages\keras\models.py", line 107, in add
layer.create_input_layer(batch_input_shape, input_dtype)
File "C:\Anaconda3\envs\MyEnv\lib\site-packages\keras\engine\topology.py", line 341, in create_input_layer
self(x)
File "C:\Anaconda3\envs\MyEnv\lib\site-packages\keras\engine\topology.py", line 485, in call
self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
File "C:\Anaconda3\envs\MyEnv\lib\site-packages\keras\engine\topology.py", line 543, in add_inbound_node
Node.create_node(self, inbound_layers, node_indices, tensor_indices)
File "C:\Anaconda3\envs\MyEnv\lib\site-packages\keras\engine\topology.py", line 148, in create_node
output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
File "C:\Anaconda3\envs\MyEnv\lib\site-packages\keras\layers\recurrent.py", line 213, in call
': ' + str(input_shape))
Exception: When using TensorFlow, you should define explicitly the number of timesteps of your sequences.
If your first layer is an Embedding, make sure to pass it an "input_length" argument. Otherwise, make sure the first layer has an "input_shape" or "batch_input_shape" argument, including the time axis. Found input shape at layer itosfm_1: (None, None, 1)

Has anybody come across this error please? and how did you manage to fix it?

Any help is appreciate because I cannot progress any further..

Thanks

In equation 10 of the paper, A_t is taken to the be square root of square(S_re)+square(S_im). However, in line 158 of train/itosfm.py, A_t is taken to be square(S_re)+square(S_im), without the square root. Is this a typo or my understanding is wrong? Thank you very much!

why do states have the index of 0-7 instead of 0-4?

Thanks for your realizations from the paper. I am just curious about the data structure from the part of "Test with pretrained model".
Also, I checked out the Keras document and refered to section of "Writing your own Keras layers" and still have the problems as following:

TypeError: ('Keyword argument notunderstood:', 'input_dim')

I tried different ways to fix this but doesn't work, can you help revise it , thx!!
PS. I adjusted initializations to initializers (keras 2.1.5)

max_data min_data那里是对全数据集取的最大最小值？这岂不是用到了未来数据？

when i run python test.py --step=1, i got the following errors:

Using TensorFlow backend.

Loading data...
<type 'numpy.ndarray'>
Traceback (most recent call last):
File "test.py", line 35, in
X_train, y_train, X_val, y_val, X_test, y_test, gt_test, max_data, min_data = build.load_data(data_file, step)
File "/home/canl/nick/State-Frequency-Memory-stock-prediction/test/build.py", line 30, in load_data
x_train = data[:,:train_split]
TypeError: slice indices must be integers or None or have an index method

bc my pandas or numpy is wrong version??
Thanks