ValueError: Attempt to reuse RNNCell with a different variable scope than its first use

2024/10/9 22:21:01

The following code fragment

import tensorflow as tf
from tensorflow.contrib import rnnhidden_size = 100
batch_size  = 100
num_steps   = 100
num_layers  = 100
is_training = True
keep_prob   = 0.4input_data = tf.placeholder(tf.float32, [batch_size, num_steps])
lstm_cell = rnn.BasicLSTMCell(hidden_size, forget_bias=0.0, state_is_tuple=True)if is_training and keep_prob < 1:lstm_cell = rnn.DropoutWrapper(lstm_cell)
cell = rnn.MultiRNNCell([lstm_cell for _ in range(num_layers)], state_is_tuple=True)_initial_state = cell.zero_state(batch_size, tf.float32)iw = tf.get_variable("input_w", [1, hidden_size])
ib = tf.get_variable("input_b", [hidden_size])
inputs = [tf.nn.xw_plus_b(i_, iw, ib) for i_ in tf.split(input_data, num_steps, 1)]if is_training and keep_prob < 1:inputs = [tf.nn.dropout(input_, keep_prob) for input_ in inputs]outputs, states = rnn.static_rnn(cell, inputs, initial_state=_initial_state)

produces the following error:

ValueError: Attempt to reuse RNNCell <tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.BasicLSTMCell object at 0x10210d5c0> with a different variable scope than its first use. First use of cell was with scope 'rnn/multi_rnn_cell/cell_0/basic_lstm_cell', this attempt is with scope `'rnn/multi_rnn_cell/cell_1/basic_lstm_cell'``.

Please create a new instance of the cell if you would like it to use a different set of weights.

If before you were using: MultiRNNCell([BasicLSTMCell(...)] * num_layers), change to: MultiRNNCell([BasicLSTMCell(...) for _ in range(num_layers)]).

If before you were using the same cell instance as both the forward and reverse cell of a bidirectional RNN, simply create two instances (one for forward, one for reverse).

In May 2017, we will start transitioning this cell's behavior to use existing stored weights, if any, when it is called with scope=None (which can lead to silent model degradation, so this error will remain until then.)

How to solve this problem?

My version of Tensorflow is 1.0.

Answer

As suggested in the comments my solution is:
changing this

cell = tf.contrib.rnn.LSTMCell(state_size, state_is_tuple=True)
cell = tf.contrib.rnn.DropoutWrapper(cell, output_keep_prob=0.8)
rnn_cells = tf.contrib.rnn.MultiRNNCell([cell for _ in range(num_layers)], state_is_tuple = True)
outputs, current_state = tf.nn.dynamic_rnn(rnn_cells, x, initial_state=rnn_tuple_state, scope = "layer")

into:

def lstm_cell():cell = tf.contrib.rnn.LSTMCell(state_size, reuse=tf.get_variable_scope().reuse)return tf.contrib.rnn.DropoutWrapper(cell, output_keep_prob=0.8)rnn_cells = tf.contrib.rnn.MultiRNNCell([lstm_cell() for _ in range(num_layers)], state_is_tuple = True)
outputs, current_state = tf.nn.dynamic_rnn(rnn_cells, x, initial_state=rnn_tuple_state)

which seems to solve the reusability problem. I don't fundamentally understand the underlying problem, but this solved the issue for me on TF 1.1rc2
cheers!

https://en.xdnf.cn/q/69964.html

Related Q&A

Convex Hull and SciPy

Im trying to use scipy (0.10.1) for a quick hack to visualize the convex hull.I can get the convex hull using the following code:vecs = [[-0.094218, 51.478927], [-0.09348, 51.479364], [-0.094218, 51.4…

Flask Confirm Action

Im creating a site using the Flask framework, and am implementing a confirmation page for (mainly administrative) actions; i.e. deleting a user.My current method (detailed below) works, but feels quite…

Regex for accent insensitive replacement in python

In Python 3, Id like to be able to use re.sub() in an "accent-insensitive" way, as we can do with the re.I flag for case-insensitive substitution.Could be something like a re.IGNOREACCENTS fl…

Python + Flask REST API, how to convert data keys between camelcase and snakecase?

I am learning Python, and coding simple REST API using Flask micro-framework.I am using SQLAlchemy for Object-relational-mapping and Marshmallow for Object-serialization/deserialization.I am using snak…

pytest reports too much on assert failures

Is there a way for pytest to only output a single line assert errors?This problem arises when you have modules with asserts, If those asserts fails, it dumps the entire function that failed the assert…

pulp.solvers.PulpSolverError: PuLP: cannot execute glpsol.exe

I am a newbie with python and optimization. I am getting some error, please help me resolve it. I tried running the below mentioned code in PyCharm where I am running Anaconda 3from pulp import * x = L…

Django urldecode in template file

is there any way do the urldecode in Django template file? Just opposite to urlencode or escapeI want to convert app%20llc to app llc

Structure accessible by attribute name or index options

I am very new to Python, and trying to figure out how to create an object that has values that are accessible either by attribute name, or by index. For example, the way os.stat() returns a stat_resul…

Loading data from Yahoo! Finance with pandas

I am working my way through Wes McKinneys book Python For Data Analysis and on page 139 under Correlation and Covariance, I am getting an error when I try to run his code to obtain data from Yahoo! Fin…

Run Multiple Instances of ChromeDriver

Using selenium and python I have several tests that need to run in parallel. To avoid using the same browser I added the parameter of using a specific profile directory and user data (see below). The p…