Giter Club home page Giter Club logo

Comments (5)

ageron avatar ageron commented on May 3, 2024 1

Ok, I fixed the notebook to support TensorFlow 1.1+.
Thanks again for bringing this issue to my attention. 👍

Cheers,
Aurélien

from handson-ml.

StrangeTcy avatar StrangeTcy commented on May 3, 2024 1

Ok, thanks a bunch!

from handson-ml.

jingw222 avatar jingw222 commented on May 3, 2024

I have similar issue with executing this cell of code, and I got this error:

ValueError: Trying to share variable rnn/multi_rnn_cell/cell_0/basic_rnn_cell/kernel, but specified shape (200, 100) and found shape (102, 100).

Why does it expect shape (200, 100) and get shape (102, 100)? Since under the assumption of your previous tutorial explaining things under the hood, It should expect shape (100, 100) instead. Previously we had this:

Y0 = tf.tanh(tf.matmul(X0, Wx) + b) 
Y1 = tf.tanh(tf.matmul(Y0, Wy) + tf.matmul(X1, Wx) + b)

So there was a sum op between the hidden state and a new input. However, the error indicating shape of either (200, 100) or (102, 100) was generally telling me It was a concat op instead, and the new input of dimension 2 went into the 'concat' op without multiplying its weight ((100 + 2 = 102) as opposed to (100 + 100 = 200)). Don't know if I get myself across.

from handson-ml.

ageron avatar ageron commented on May 3, 2024

Hi David,
It's a very busy week for me, so I can't investigate right now, but I'll have some time next week. In the meantime, could you please give me more details on your installation? What version of TensorFlow are you using? Can you try installing tensorflow 1.1? There might be some breaking changes in TensorFlow 1.2 regarding RNNs.

from handson-ml.

ageron avatar ageron commented on May 3, 2024

Ok, I've look into this error, it seems to be due to a breaking change in TensorFlow 1.1 in how stacked RNNs are handled.

With TensorFlow 1.0, this code works fine:

>>> import tensorflow as tf
>>> cell = tf.contrib.rnn.BasicRNNCell(num_units=100)
>>> multi_layer_cell = tf.contrib.rnn.MultiRNNCell([cell]*3)
>>> X = tf.placeholder(tf.float32, [None, 20, 10])
>>> outputs, states = tf.nn.dynamic_rnn(multi_layer_cell, X, dtype=tf.float32)
>>> 

Then I upgrade to TensorFlow 1.1:

$ pip install --upgrade tensorflow
[...]
Successfully installed tensorflow-1.1.0

And now the same exact code causes the problem:

>>> import tensorflow as tf
>>> cell = tf.contrib.rnn.BasicRNNCell(num_units=100)
>>> multi_layer_cell = tf.contrib.rnn.MultiRNNCell([cell]*3)
>>> X = tf.placeholder(tf.float32, [None, 20, 10])
>>> outputs, states = tf.nn.dynamic_rnn(multi_layer_cell, X, dtype=tf.float32)
Traceback (most recent call last):
[...]
  File "/home/ageron/dev/py/envs/ml/lib/python3.5/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py", line 77, in _checked_scope
    type(cell).__name__))
ValueError: Attempt to reuse RNNCell <tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.BasicRNNCell object at 0x7f5887f9cc18> with a different variable scope than its first use.  First use of cell was with scope 'rnn/multi_rnn_cell/cell_0/basic_rnn_cell', this attempt is with scope 'rnn/multi_rnn_cell/cell_1/basic_rnn_cell'.  Please create a new instance of the cell if you would like it to use a different set of weights.  If before you were using: MultiRNNCell([BasicRNNCell(...)] * num_layers), change to: MultiRNNCell([BasicRNNCell(...) for _ in range(num_layers)]).  If before you were using the same cell instance as both the forward and reverse cell of a bidirectional RNN, simply create two instances (one for forward, one for reverse).  In May 2017, we will start transitioning this cell's behavior to use existing stored weights, if any, when it is called with scope=None (which can lead to silent model degradation, so this error will remain until then.)
>>> 

So now the proper way to create a MultiRNNCell looks like this:

>>> import tensorflow as tf
>>> cells = [tf.contrib.rnn.BasicRNNCell(num_units=100) for layer in range(3)]
>>> multi_layer_cell = tf.contrib.rnn.MultiRNNCell(cells)
>>> X = tf.placeholder(tf.float32, [None, 20, 10])
>>> outputs, states = tf.nn.dynamic_rnn(multi_layer_cell, X, dtype=tf.float32)
>>>

I'm fixing the notebook for chapter 14, I'll push it by tomorrow.

Thanks for your feedback! :)

from handson-ml.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.