Based on this question I assumed that creating new process should be almost as fast as creating new thread in Linux. However, little test showed very different result. Here's my code:
from multiprocessing import Process, Pool
from threading import Threadtimes = 1000def inc(a):b = 1return a + bdef processes():for i in xrange(times):p = Process(target=inc, args=(i, ))p.start()p.join()def threads():for i in xrange(times):t = Thread(target=inc, args=(i, ))t.start()t.join()
Tests:
>>> timeit processes()
1 loops, best of 3: 3.8 s per loop>>> timeit threads()
10 loops, best of 3: 98.6 ms per loop
So, processes are almost 40 times slower to create! Why does it happen? Is it specific to Python or these libraries? Or did I just misinterpreted the answer above?
UPD 1. To make it more clear. I understand that this piece of code doesn't actually introduce any concurrency. The goal here is to test the time needed to create a process and a thread. To use real concurrency with Python one can use something like this:
def pools():pool = Pool(10)pool.map(inc, xrange(times))
which really runs much faster than threaded version.
UPD 2. I have added version with os.fork()
:
for i in xrange(times):child_pid = os.fork()if child_pid:os.waitpid(child_pid, 0)else:exit(-1)
Results are:
$ time python test_fork.py real 0m3.919s
user 0m0.040s
sys 0m0.208s$ time python test_multiprocessing.py real 0m1.088s
user 0m0.128s
sys 0m0.292s$ time python test_threadings.pyreal 0m0.134s
user 0m0.112s
sys 0m0.048s