multiprocessing numpy not defined error

2024/9/8 10:51:04

I am using the following test code:

from pathos.multiprocessing import ProcessingPool as Pool
import numpydef foo(obj1, obj2):a = obj1**2b = numpy.asarray(range(1,5))return obj1, bif __name__ == '__main__':p = Pool(5)res = p.map(foo, [1,2,3], [4,5,6])

It gives error:

File "C:\Python27\lib\site-packages\multiprocess\pool.py", line 567, in getraise self._value
NameError: global name 'numpy' is not defined

What am I doing wrong in the code?

Edit: Why was this question voted down twice?

I have numpy installed and my interpreter has been using it correctly until I try to do it for multiprocessing. I have been coding with same install for a while.

Answer

It seems like imports are not shared between processes. Therefore you need to import numpy in all your processes seperatly.

In your case this means adding the import numpy in your foo function. Processes are not light-weight so the import won't slow you down (at least not significantly).

The other alternative would be to pass the module to the functions (not recommended and I'm not sure if that will work):

if __name__ == '__main__':p = Pool(5)res = p.map(foo, numpy, [1,2,3], [4,5,6])def foo(np, obj1, obj2):a = obj1**2b = np.asarray(range(1,5))return obj1, b
https://en.xdnf.cn/q/73118.html

Related Q&A

Convert mp4 sound to text in python

I want to convert a sound recording from Facebook Messenger to text. Here is an example of an .mp4 file send using Facebooks API: https://cdn.fbsbx.com/v/t59.3654-21/15720510_10211855778255994_5430581…

pandas map one column to the combination of two columns

I am working with a DataFrame which looks like thisList Numb Name 1 1 one 1 2 two 2 3 three 4 4 four 3 5 fiveand I am trying to compute…

Pyspark command not recognised

I have anaconda installed and also I have downloaded Spark 1.6.2. I am using the following instructions from this answer to configure spark for Jupyter enter link description hereI have downloaded and …

Sliding window in Python for GLCM calculation

I am trying to do texture analysis in a satellite imagery using GLCM algorithm. The scikit-image documentation is very helpful on that but for GLCM calculation we need a window size looping over the im…

Keras multi-label image classification with F1-score

I am working on a multi-label image classification problem with the evaluation being conducted in terms of F1-score between system predicted and ground truth labels.Given that, should I use loss="…

Thread safe locale techniques

Were currently writing a web application based on a threaded python web server framework (cherrypy) and would like to simultaneously support users from multiple locales.The locale module doesnt appear …

How to draw image from raw bytes using ReportLab?

All the examples I encounter in the internet is loading the image from url (either locally or in the web). What I want is to draw the image directly to the pdf from raw bytes.UPDATE:@georgexsh Here is …

How to speed up numpy code

I have the following code. In principle it takes 2^6 * 1000 = 64000 iterations which is quite a small number. However it takes 9s on my computer and I would like to run it for n = 15 at least.from __f…

MyPy gives error Missing return statement even when all cases are tested

I am getting a MyPy error "Missing return statement", even when I check for all possible cases inside a function.For example, in the following code, MyPy is still giving me an error "9: …

Python Json with returns AttributeError: __enter__

Why does this return AttributeError: __enter__Sorting method is just a string created based on how the list is sorted, and current time uses stfttimecurrent_time = strftime("%Y-%m-%d %H-%M-%S"…