Multiple subprocesses with timeouts

2024/10/3 10:37:45

I'm using a recipe that relies on SIGALRM to set alarm interrupt -- Using module 'subprocess' with timeout

The problem is that I have more than one Python script using signal.ALARM process to set time-outs, and only the latest alarm gets called. What is a good way to improve this multiple Python functions setting time-outs?

Answer

Except for simple, quick hacks, avoid SIGALRM. It's a very old, limited mechanism, not suited to anything more complex: you can only set a single alarm, and it interrupts any system call at the time rather than just the one you intend to interrupt.

It's much cleaner to use a timeout thread to kill the process, for example:

import subprocess, signal, os, threading, errno
from contextlib import contextmanagerclass TimeoutThread(object):def __init__(self, seconds):self.seconds = secondsself.cond = threading.Condition()self.cancelled = Falseself.thread = threading.Thread(target=self._wait)def run(self):"""Begin the timeout."""self.thread.start()def _wait(self):with self.cond:self.cond.wait(self.seconds)if not self.cancelled:self.timed_out()def cancel(self):"""Cancel the timeout, if it hasn't yet occured."""with self.cond:self.cancelled = Trueself.cond.notify()self.thread.join()def timed_out(self):"""The timeout has expired."""raise NotImplementedErrorclass KillProcessThread(TimeoutThread):def __init__(self, seconds, pid):super(KillProcessThread, self).__init__(seconds)self.pid = piddef timed_out(self):try:os.kill(self.pid, signal.SIGKILL)except OSError as e:# If the process is already gone, ignore the error.if e.errno not in (errno.EPERM, errno. ESRCH):raise e@contextmanager
def processTimeout(seconds, pid):timeout = KillProcessThread(seconds, pid)timeout.run()try:yieldfinally:timeout.cancel()def example():proc = subprocess.Popen(["sleep", "5"], stdin=subprocess.PIPE, stdout=subprocess.PIPE)with processTimeout(1, proc.pid):print proc.communicate()resultcode = proc.wait()if resultcode < 0:print "error: %i" % resultcodeif __name__ == '__main__':example()

Depending on what you're timing out, you may want to use a lighter signal than SIGKILL to allow the timing-out process to clean up after itself.

https://en.xdnf.cn/q/70735.html

Related Q&A

what is the difference between tfidf vectorizer and tfidf transformer

I know that the formula for tfidf vectorizer is Count of word/Total count * log(Number of documents / no.of documents where word is present)I saw theres tfidf transformer in the scikit learn and I just…

Use Pandas string method contains on a Series containing lists of strings

Given a simple Pandas Series that contains some strings which can consist of more than one sentence:In: import pandas as pd s = pd.Series([This is a long text. It has multiple sentences.,Do you see? M…

Is this the correct way of whitening an image in python?

I am trying to zero-center and whiten CIFAR10 dataset, but the result I get looks like random noise! Cifar10 dataset contains 60,000 color images of size 32x32. The training set contains 50,000 and tes…

Python zlib output, how to recover out of mysql utf-8 table?

In python, I compressed a string using zlib, and then inserted it into a mysql column that is of type blob, using the utf-8 encoding. The string comes back as utf-8, but its not clear how to get it bac…

Incorrect user for supervisord celeryd

I have some periodic tasks that I run with celery (daemonized by supervisord), but after trying to create a directory in the home dir for the user i setup for the supervisord process I got a "perm…

Pandas drop rows where column contains *

Im trying to drop all rows from this df where column DB Serial contains the character *:DB Serial 0 13058 1 13069 2 *13070 3 13070 4 13044 5 13042I am using:df = df[~df[DB Serial…

How to stop scrapy spider after certain number of requests?

I am developing an simple scraper to get 9 gag posts and its images but due to some technical difficulties iam unable to stop the scraper and it keeps on scraping which i dont want.I want to increase t…

What is the difference between single and double bracket Numpy array?

import numpy as np a=np.random.randn(1, 2) b=np.zeros((1,2)) print("Data type of A: ",type(a)) print("Data type of A: ",type(b))Output:Data type of A: <class numpy.ndarray> D…

How to make tkinter button widget take up full width of grid

Ive tried this but it didnt help. Im making a calculator program. Ive made this so far: from tkinter import * window = Tk()disp = Entry(window, state=readonly, readonlybackground="white") dis…

Python strip() unicode string?

How can you use string methods like strip() on a unicode string? and cant you access characters of a unicode string like with oridnary strings? (ex: mystring[0:4] )