is there any pool for ThreadingMixIn and ForkingMixIn for SocketServer?

2024/10/3 0:30:57

I was trying to make an http proxy using BaseHttpServer which is based on SocketServer which got 2 asynchronous Mixins (ThreadingMixIn and ForkingMixIn)

the problem with those two that they work on each request (allocate a new thread or fork a new subprocess for each request)

is there a Mixin that utilize a pool of let's say 4 subprocesses and 40 threads in each so requests get handled by those already created threads ?

because this would be a big performance gain and I guess it would save some resources.

Answer

You could use a pool from concurrent.futures (in stdlib since Python 3.2):

from BaseHTTPServer   import HTTPServer, test
from SimpleHTTPServer import SimpleHTTPRequestHandler
from SocketServer     import ThreadingMixInfrom concurrent.futures import ThreadPoolExecutor # pip install futuresclass PoolMixIn(ThreadingMixIn):def process_request(self, request, client_address):self.pool.submit(self.process_request_thread, request, client_address)def main():class PoolHTTPServer(PoolMixIn, HTTPServer):pool = ThreadPoolExecutor(max_workers=40)test(HandlerClass=SimpleHTTPRequestHandler, ServerClass=PoolHTTPServer)if __name__=="__main__":main()

As you can see the implementation for a threading case is rather trivial.

If you save it to server.py then you could run it as:

$ python -mserver

This command uses upto 40 threads to serve requests on http://your_host:8000/.

The main use case of HTTPServer is for testing purposes.

https://en.xdnf.cn/q/70793.html

Related Q&A

Python - Read data from netCDF file with time as seconds since beginning of measurement

I need to extract values from a netCDf file. I am pretty new to python and even newer this file format. I need to extract time series data at a specific location (lat, lon). I have found that there is …

PyQt Multiline Text Input Box

I am working with PyQt and am attempting to build a multiline text input box for users. However, when I run the code below, I get a box that only allows for a single line of text to be entered. How to …

Calculate the sum of model properties in Django

I have a model Order which has a property that calculates an order_total based on OrderItems linked by foreign key.I would like to calculate the sum of a number of Order instances order_total propertie…

Set Host-header when using Python and urllib2

Im using my own resolver and would like to use urllib2 to just connect to the IP (no resolving in urllib2) and I would like set the HTTP Host-header myself. But urllib2 is just ignoring my Host-header:…

Full-featured date and time library

Im wondering if anyone knows of a good date and time library that has correctly-implemented features like the following:Microsecond resolution Daylight savings Example: it knows that 2:30am did not exi…

Mean of a correlation matrix - pandas data fram

I have a large correlation matrix in a pandas python DataFrame: df (342, 342).How do I take the mean, sd, etc. of all of the numbers in the upper triangle not including the 1s along the diagonal?Thank…

How to set imshow scale

Im fed up with matplotlib in that its so hard to plot images in specified size.Ive two images in 32*32, 20*20 sizes. I just want to plot them in its original size, or in proportion to its original size…

Python distutils gcc path

Im trying to cross-compile the pycrypto package, and Im getting closer and closer however, Ive hit an issue I just cant figure out.I want distutils to use the cross-compile specific gcc- so I set the C…

TypeError: builtin_function_or_method object has no attribute __getitem__

Ive got simple python functions.def readMainTemplate(templateFile):template = open(templateFile, r)data = template.read()index1 = data.index[[] #originally I passed it into data[]index2 = data.index[]]…

Extract currency amount from string in Python

Im making a program that takes currency from a string and converts it in to other currencies. For example, if the string was the car cost me $13,250 I would need to get $ and 13250. I have this regex a…