I'm using tf.data.Dataset in data processing and I want to do apply some python code with tf.py_func.

BTW, I found that in tf.py_func, I cannot return a dictionary. Is there any way to do it or workaround?

I have code which looks like below

def map_func(images, labels):"""mapping python function"""# do something# cannot be expressed as a tensor graphreturn {'images': images,'labels': labels,'new_key': new_value}
def tf_py_func(images, labels):return tf.py_func(map_func, [images, labels], [tf.uint8, tf.string], name='blah')return dataset.map(tf_py_func)

===========================================================================

It's been a while and I forgot I asked this question. I solved it other way around and it was so easy that I felt I was almost a stupid. The problem was:

1. tf.py_func cannot return dictionary.
2. dataset.map can return dictionary.

And the answer is: map twice.

def map_func(images, labels):"""mapping python function"""# do something# cannot be expressed as a tensor graphreturn processed_images, processed_labelsdef tf_py_func(images, labels):return tf.py_func(map_func, [images, labels], [tf.uint8, tf.string], name='blah')def _to_dict(images, labels):return { 'images': images, 'labels': labels }return dataset.map(tf_py_func).map(_to_dict)

You could turn the dictionary into a string which you return and then split into a dictionary.

This could look something like this:

return (images + " " + labels + " " + new value)

and then in your other function:

l = map_func(image, label).split(" ")
d['images'] = l[0]
d[
...