Background:

I have some complex reinforcement learning algorithm that I want to run in multiple threads.

Problem

When trying to call sess.run in a thread I get the following error message:

RuntimeError: The Session graph is empty. Add operations to the graph before calling run().

Code reproducing the error:

import tensorflow as tfimport threadingdef thread_function(sess, i):inn = [1.3, 4.5]A = tf.placeholder(dtype=float, shape=(None), name="input")P = tf.Print(A, [A])Q = tf.add(A, P)sess.run(Q, feed_dict={A: inn})def main(sess):thread_list = []for i in range(0, 4):t = threading.Thread(target=thread_function, args=(sess, i))thread_list.append(t)t.start()for t in thread_list:t.join()if __name__ == '__main__':sess = tf.Session()main(sess)

If I run the same code outside a thread it works properly.

Can someone give some insight on how to use Tensorflow sessions properly with python threads?

Not only can the Session be the current thread default, but also the graph. While you pass in the session and call run on it, the default graph will be a different one.

You can ammend your thread_function like this to make it work:

def thread_function(sess, i):with sess.graph.as_default():inn = [1.3, 4.5]A = tf.placeholder(dtype=float, shape=(None), name="input")P = tf.Print(A, [A])Q = tf.add(A, P)sess.run(Q, feed_dict={A: inn})

However, I wouldn't hope for any significant speedup. Python threading isn't what it means in some other languages, only certain operations, like io, would run in parallel. For CPU heavy operations it's not very useful. Multiprocessing can run code truely in parallel, but you wouldn't share the same session.