Hi, interesting examples and presentation.
I would like to give a try.
So I have just clone the git in my Ubuntu 16.04 and installed the most recent version of TensorFlow with pip.
But I get the following error:

root@ubumate:/opt/tensorflow/TensorFlow-Tutorials-for-Time-Series# python3 lstm-for-epf.py WARNING:tensorflow:From lstm-for-epf.py:34 in <module>.: calling BaseEstimator.evaluate (from tensorflow.contrib.learn.python.learn.estimators.estimator) with batch_size is deprecated and will be removed after 2016-12-01. Instructions for updating: Estimator is decoupled from Scikit Learn interface by moving into separate class SKCompat. Arguments x, y and batch_size are only available in the SKCompat class, Estimator will only accept input_fn. Example conversion: est = Estimator(...) -> est = SKCompat(Estimator(...))

Do you have an idea how I should change the following line:

regressor = learn.TensorFlowEstimator(model_fn=lstm_model(TIMESTEPS, RNN_LAYERS, DENSE_LAYERS), n_classes=0, verbose=1, steps=TRAINING_STEPS, optimizer='Adagrad', learning_rate=0.03, batch_size=BATCH_SIZE))

in order to make it working?
Thanks.
Regards.

When I tried your reference for giving multiple inputs (3 variables), and one output for prediction. I get an issue of lhs not matching rhs. Somewhere deep inside the abstraction.

I modified the code in load_predictor.py. for loading my csv file. I have attached the screen log in the attachments.
log.txt

LSTM-Time-Series-Analysis-using-Tensorflow.zip

I took @pusj latest updated code. I have attached the log below. Can someone help me with this?

in the lstm_predictor.py file , i change to the tf.unpack. But still error pops out: Expected int for argument 'axis' not <tf.Tensor 'input:0' shape=(?, 10, 1) dtype=float64>.

def lstm_model(X, y):
stacked_lstm = tf.nn.rnn_cell.MultiRNNCell(lstm_cells(rnn_layers), state_is_tuple=True)
**x = tf.unpack(1, time_steps, X)**

full version of error report:

File "", line 1, in
runfile('/home/sherlock/Downloads/TensorFlow-Tutorials-for-Time-Series-master/lstm-for-epf.py', wdir='/home/sherlock/Downloads/TensorFlow-Tutorials-for-Time-Series-master')

File "/usr/local/lib/python2.7/dist-packages/spyder/utils/site/sitecustomize.py", line 866, in runfile
execfile(filename, namespace)

File "/usr/local/lib/python2.7/dist-packages/spyder/utils/site/sitecustomize.py", line 94, in execfile
builtins.execfile(filename, *where)

File "/home/sherlock/Downloads/TensorFlow-Tutorials-for-Time-Series-master/lstm-for-epf.py", line 42, in
regressor.fit(X['train'], y['train'], monitors=[validation_monitor], logdir=LOG_DIR)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/base.py", line 257, in fit
monitors=monitors)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 662, in _train_model
train_op, loss_op = self._get_train_ops(features, targets)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 963, in _get_train_ops
_, loss, train_op = self._call_model_fn(features, targets, ModeKeys.TRAIN)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 946, in _call_model_fn
return self._model_fn(features, targets, mode=mode)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/base.py", line 493, in _model_fn
predictions, loss = model_fn(features, targets)

File "lstm_predictor.py", line 116, in lstm_model
x = tf.unpack(1, time_steps, X)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 799, in unpack
return gen_array_ops._unpack(value, num=num, axis=axis, name=name)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 3023, in _unpack
name=name)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 666, in apply_op
attr_value.i = _MakeInt(value, key)

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 155, in _MakeInt
(arg_name, repr(v)))

TypeError: Expected int for argument 'axis' not <tf.Tensor 'input:0' shape=(?, 10, 1) dtype=float64>.

Hello, I can't figure out why has no attribute.
Using TensorFlow0.12

Hi,

I m trying to run the script lstm-for-epf.py with tensorflow 0.10

the execution abort due to the following error

File ".../tensorflow/TensorFlow-Tutorials-for-Time-Series-master/lstm_predictor.py", line 117, in lstm_model
output, layers = tf.nn.rnn(stacked_lstm, x, dtype=dtypes.float32)
File "/Library/Python/2.7/site-packages/tensorflow/python/ops/rnn.py", line 219, in rnn
(output, state) = call_cell()
File "/Library/Python/2.7/site-packages/tensorflow/python/ops/rnn.py", line 206, in
call_cell = lambda: cell(input_, state)
File "/Library/Python/2.7/site-packages/tensorflow/python/ops/rnn_cell.py", line 812, in call
cur_inp, new_state = cell(cur_inp, cur_state)
File "/Library/Python/2.7/site-packages/tensorflow/python/ops/rnn_cell.py", line 310, in call
concat = _linear([inputs, h], 4 * self._num_units, True)
File "/Library/Python/2.7/site-packages/tensorflow/python/ops/rnn_cell.py", line 906, in _linear
res = math_ops.matmul(array_ops.concat(1, args), matrix)
File "/Library/Python/2.7/site-packages/tensorflow/python/ops/array_ops.py", line 697, in concat
name=name)
File "/Library/Python/2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 414, in _concat
values=values, name=name)
File "/Library/Python/2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 437, in apply_op
raise TypeError("%s that don't all match." % prefix)
TypeError: Tensors in list passed to 'values' of 'Concat' Op have types [float64, float32] that don't all match.

there is a float64/32 conversion issue.

adding this in _lstm_model fix the issue.

X = tf.cast(X,tf.float32)

there is probably another better fix, I just started playing with the code :)

Trying to run any of the LSTM examples, results in this error:

InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [10,1] rhs shape= [10,6]

[[Node: save/Assign_17 = Assign[T=DT_FLOAT, _class=["loc:@linear_regression/weights"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/cpu:0"](linear_regression/weights, save/restore_slice_17)]]

Seems to be related to lstm_predictor.py.

while training my own datasets i got following errors

(tensorflow) pi@pi-HP-280-G1-MT:~/Desktop/models/research/object_detection$ python3 train.py --logtostderr --train_dir=train --pipeline_config_path=train/ssd_mobilenet_v1_coco.config
WARNING:tensorflow:From /home/pi/Desktop/models/research/object_detection/object_detection/trainer.py:176: create_global_step (from tensorflow.contrib.framework.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.create_global_step
Traceback (most recent call last):
File "/home/pi/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 455, in _apply_op_helper
as_ref=input_arg.is_ref)
File "/home/pi/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1169, in internal_convert_n_to_tensor
ctx=ctx))
File "/home/pi/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1104, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/home/pi/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1034, in _autopacking_conversion_function
return _autopacking_helper(v, inferred_dtype, name or "packed")
File "/home/pi/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 976, in _autopacking_helper
elem))
TypeError: Cannot convert a list containing a tensor of dtype <dtype: 'int32'> to <dtype: 'float32'> (Tensor is: <tf.Tensor 'Preprocessor/stack_1:0' shape=(1, 3) dtype=int32>)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "train.py", line 198, in
tf.app.run()
File "/home/pi/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 126, in run
_sys.exit(main(argv))
File "train.py", line 194, in main
worker_job_name, is_chief, FLAGS.train_dir)
File "/home/pi/Desktop/models/research/object_detection/object_detection/trainer.py", line 192, in train
clones = model_deploy.create_clones(deploy_config, model_fn, [input_queue])
File "/home/pi/Desktop/models/research/slim/deployment/model_deploy.py", line 193, in create_clones
outputs = model_fn(*args, **kwargs)
File "/home/pi/Desktop/models/research/object_detection/object_detection/trainer.py", line 124, in _create_losses
images = tf.concat(images, 0)
File "/home/pi/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1189, in concat
return gen_array_ops.concat_v2(values=values, axis=axis, name=name)
File "/home/pi/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 953, in concat_v2
"ConcatV2", values=values, axis=axis, name=name)
File "/home/pi/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 483, in _apply_op_helper
raise TypeError("%s that don't all match." % prefix)
TypeError: Tensors in list passed to 'values' of 'ConcatV2' Op have types [, , , , , , , , , ] that don't all match.

pls help me out

def model(X, W, B, lstm_size):
outputs, _states = tf.nn.rnn(lstm, X_split, dtype=tf.float32)

Where is tf.nn.rnn in tf 1.0

Thanks

I made it working for ver 1.0 with this code for model:

lstm = tf.contrib.rnn.BasicLSTMCell(lstm_size, forget_bias=1.0, state_is_tuple=True)
outputs, states = tf.contrib.rnn.static_rnn(lstm, X_split, dtype=tf.float32)
return tf.matmul(outputs[-1], W) + B, lstm.state_size # State size to initialize the stat

Nice tutorial, thanks

Hi,
I am upgrading the example to the latest TF.
In your comments you mention that split.squeeze should be replaced by tf.unpak, however there is no tf.unpack in the latest TF. Can you please clarify what you had in mind?

Thanks,

WARNING:tensorflow:From C:\Users\JOSEPH\Anaconda3\envs\tensorflow-cpu\lib\site-packages\tensorflow\contrib\learn\python\learn\monitors.py:269: BaseMonitor.init (from tensorflow.contrib.learn.python.learn.monitors) is deprecated and will be removed after 2016-12-05.
Instructions for updating:
Monitors are deprecated. Please use tf.train.SessionRunHook.

WARNING:tensorflow:From C:\Users\JOSEPH\Anaconda3\envs\tensorflow-cpu\lib\site-packages\tensorflow\contrib\learn\python\learn\monitors.py:269: BaseMonitor.init (from tensorflow.contrib.learn.python.learn.monitors) is deprecated and will be removed after 2016-12-05.
Instructions for updating:
Monitors are deprecated. Please use tf.train.SessionRunHook.

TypeError Traceback (most recent call last)
in ()
55 early_stopping_rounds=1000)
56
---> 57 regressor.fit(X['train'], y['train'],monitors=[validation_monitor],logdir=LOG_DIR)
58
59 # self.assertTrue(hook.loss_collection)

TypeError: fit() got an unexpected keyword argument 'monitors'

Hi I am trying to make this code compatible with TensorFlow 1.5. I am also using this transition to 1.1.: https://github.com/pusj/LSTM-Time-Series-Analysis-using-Tensorflow/blob/master/lstm_sin_rev1.py for help.

In the present tutorial, the following code is working:

stacked_lstm = tf.nn.rnn_cell.MultiRNNCell(lstm_cells(rnn_layers), state_is_tuple=True)
x_ = learn.ops.split_squeeze(1, time_steps, X)
output, layers = tf.nn.rnn(stacked_lstm, x_, dtype=dtypes.float32)

For it to work in tensorflow 1.5.0 I changed it to:

stacked_lstm = tf.nn.rnn_cell.MultiRNNCell(lstm_cells(rnn_layers), state_is_tuple=True)
x_ = tf.unstack(X, time_steps, 1)
output, layers = tf.nn.static_rnn(stacked_lstm, x_,dtype=dtypes.float32)

Replacing split_squeeze with unstack works well. However, when using tf.nn.static_rnn() instead of tf.nn.rnn(), I get the following error:

ValueError: An initializer for variable rnn/multi_rnn_cell/cell_0/basic_lstm_cell/kernel of <dtype: 'string'> is required

In the documentation for tf.nn.static_rnn, it is said that a Value Error is raised if "If inputs is None or an empty list, or if the input depth (column size) cannot be inferred from inputs via shape inference." I have checked and the inputs are not an empty list. Plus, each input is a tensor of 3068 rows and 2 columns, so it seems that their depth can be inferred via shape inference.

Does anyone know why this is happening and know how to fix it?
Thank you very much for any help.

I am quite new to Tensorflow and this is my first issue on github so please let me know if you need any additional information.

I'm not sure if this is done intentionally - but you have a variable mse (intuitively mean_squared_error) but it calculates mean_absolute_error() :
if it is intentional - it would be better if the variable is name "mae" so as to prevent confusion.
cool tutorial btw

Does the prediction fit for time series with an obvious linear trend plus some periodic term? Such as y=x+sin(x/50)+noise; It seems the Least Square (LS) can do much better than the TFTS in this case. I am not sure whether I use the TFTS correctly.