concurrent.futures.ThreadPoolExecutor doesnt print errors

2024/9/19 10:03:37

I am trying to use concurrent.futures.ThreadPoolExecutor module to run a class method in parallel, the simplified version of my code is pretty much the following:

class TestClass:def __init__(self, secondsToSleepFor):self.secondsToSleepFor = secondsToSleepFordef testMethodToExecInParallel(self):print("ThreadName: " + threading.currentThread().getName())print(threading.currentThread().getName() + " is sleeping for " + str(self.secondsToSleepFor) + " seconds")time.sleep(self.secondsToSleepFor)print(threading.currentThread().getName() + " has finished!!")with concurrent.futures.ThreadPoolExecutor(max_workers = 2) as executor:futuresList = []print("before try")try:testClass = TestClass(3)        future = executor.submit(testClass.testMethodToExecInParallel)futuresList.append(future)except Exception as exc:print('Exception generated: %s' % exc)

If I execute this code it seems to behave like it is intended to. But if I make a mistake like specifying a wrong number of parameters in "testMethodToExecInParallel" like:

 def testMethodToExecInParallel(self, secondsToSleepFor):

and then still submitting the function as:

future = executor.submit(testClass.testMethodToExecInParallel)

or trying to concatenate a string object with an integer object (without using str(.) ) inside a print statement in "testMethodToExecInParallel" method:

def testMethodToExecInParallel(self):print("ThreadName: " + threading.currentThread().getName())print("self.secondsToSleepFor: " + self.secondsToSleepFor) <-- Should report an Error here

the program doesn't return any error; just prints "before try" and ends execution...

Is trivial to understand that this makes the program nearly undebuggable... Could someone explain me why such behaviour happens?

(for the first case of mistake) concurrent.futures.ThreadPoolExecutor doesn't check for a function with the specified signature to submit and, eventually, throw some sort of "noSuchFunction" exception?

Maybe there is some sort of problem in submitting to ThreadPoolExecutor class methods instead of simple standalone functions and, so, such behaviour could be expected?

Or maybe the error is thrown inside the thread and for some reason I can't read it?

-- EDIT --

Akshay.N suggestion of inserting future.result() after submitting functions to ThreadPoolExecutor makes the program behave as expected: goes nice if the code is correct, prints the error if something in the code is wrong.

I think users must be warned about this very strange behaviour of ThreadPoolExecutor: if you only submit functions to ThreadPoolExecutor WITHOUT THEN CALLING future.result():

  • if the code is correct, the program goes on and behaves as expected
  • if something in the code is wrong seems the program doesn't call the submitted function, whatever it does: it doesn't report the errors in the code
Answer

As far as my knowledge goes which is "not so far", you have to call "e.results()" after "executor.submit(testClass.testMethodToExecInParallel)" in order to execute the threadpool . I have tried what you said and it is giving me error, below is the code

>>> import concurrent.futures as cf
>>> executor = cf.ThreadPoolExecutor(1)
>>> def a(x,y):
...     print(x+y)
...
>>> future = executor.submit(a, 2, 35, 45)
>>> future.result()
Traceback (most recent call last):File "<stdin>", line 1, in <module>File "C:\Users\username 
\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\_base.py", line 
425, in resultreturn self.__get_result()File "C:\Users\username
\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\_base.py", line 
384, in __get_resultraise self._exceptionFile "C:\Users\username
\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\thread.py", line 
57, in runresult = self.fn(*self.args, **self.kwargs)
TypeError: a() takes 2 positional arguments but 3 were given

Let me know if it still doesn't work

https://en.xdnf.cn/q/72401.html

Related Q&A

How to write a Dictionary to Excel in Python

I have the following dictionary in python that represents a From - To Distance Matrix.graph = {A:{A:0,B:6,C:INF,D:6,E:7},B:{A:INF,B:0,C:5,D:INF,E:INF},C:{A:INF,B:INF,C:0,D:9,E:3},D:{A:INF,B:INF,C:9,D:0…

How can I check pooled connections in SQLAlchemy before handing them off to my application code?

We have a slightly unreliable database server, for various reasons, and as a consequence sometimes the database connections used by my application vanish out from under it. The connections are SQLAlch…

pandas list of dictionary to separate columns

I have a data set like below:name status number message matt active 12345 [job: , money: none, wife: none] james active 23456 [group: band, wife: yes, money: 10000] adam in…

Where is console input history stored on Python for Windows?

Good afternoon,The QuestionIs there a particular spot that the entries are stored, or is it just a local set of stored variables, for the windows version of Python?The ContextI am curious about where …

Matplotlib Animation: how to dynamically extend x limits?

I have a simple animation plot like so: import numpy as np from matplotlib import pyplot as plt from matplotlib import animation# First set up the figure, the axis, and the plot element we want to anim…

How to get round the HTTP Error 403: Forbidden with urllib.request using Python 3

Hi not every time but sometimes when trying to gain access to the LSE code I am thrown the every annoying HTTP Error 403: Forbidden message.Anyone know how I can overcome this issue only using standard…

Installing lxml in virtualenv for windows

Ive recently started using virtualenv, and would like to install lxml in this isolated environment.Normally I would use the windows binary installer, but I want to use lxml in this virtualenv (not glob…

Saving a model in Django gives me Warning: Field id doesnt have a default value

I have a very basic model in Django:class Case(models.Model):name = models.CharField(max_length=255)created_at = models.DateTimeField(default=datetime.now)updated_at = models.DateTimeField(default=date…

Authorization architecture in microservice cluster

I have a project with microservice architecture (on Docker and Kubernetes), and 2 main apps are written in Python using AIOHTTP and Django (also there are and Ingress proxy, static files server, a coup…

fastest way to load images in python for processing

I want to load more than 10000 images in my 8gb ram in the form of numpy arrays.So far I have tried cv2.imread,keras.preprocessing.image.load_image,pil,imageio,scipy.I want to do it the fastest way pos…