How to detect write failure in asyncio?

2024/9/8 7:23:49

As a simple example, consider the network equivalent of /dev/zero, below. (Or more realistically, just a web server sending a large file.)

If a client disconnects early, you get a barrage of log messages:

WARNING:asyncio:socket.send() raised exception.

But I'm not finding any way to catch said exception. The hypothetical server continues reading gigabytes from disk and sending them to a dead socket, with no effort on the client's part, and you've got yourself a DoS attack.

The only thing I've found from the docs is to yield from a read, with an empty string indicating closure. But that's no good here because a normal client isn't going to send anything, blocking the write loop.

What's the right way to detect failed writes, or be notified that the TCP connection has been closed, with the streams API or otherwise?

Code:

from asyncio import *
import logging@coroutine
def client_handler(reader, writer):while True:writer.write(bytes(1))yield from writer.drain()logging.basicConfig(level=logging.INFO)
loop = get_event_loop()
coro = start_server(client_handler, '', 12345)
server = loop.run_until_complete(coro)
loop.run_forever()
Answer

I did some digging into the asyncio source to expand on dano's answer on why the exceptions aren't being raised without explicitly passing control to the event loop. Here's what I've found.

Calling yield from wirter.drain() gives the control over to the StreamWriter.drain coroutine. This coroutine checks for and raises any exceptions that that the StreamReaderProtocol set on the StreamReader. But since we passed control over to drain, the protocol hasn't had the chance to set the exception yet. drain then gives control over to the FlowControlMixin._drain_helper coroutine. This coroutine the returns immediately because some more flags haven't been set yet, and the control ends up back with the coroutine that called yield from wirter.drain().

And so we have gone full circle without giving control to the event loop to allow it handle other coroutines and bubble up the exceptions to writer.drain().

yielding before a drain() gives the transport/protocol a chance to set the appropriate flags and exceptions.

Here's a mock up of what's going on, with all the nested calls collapsed:

import asyncio as aiodef set_exception(ctx, exc):ctx["exc"] = exc@aio.coroutine
def drain(ctx):if ctx["exc"] is not None:raise ctx["exc"]return@aio.coroutine
def client_handler(ctx):i = 0while True:i += 1print("write", i)# yield # Uncommenting this allows the loop.call_later call to be scheduled.yield from drain(ctx)CTX = {"exc": None}loop = aio.get_event_loop()
# Set the exception in 5 seconds
loop.call_later(5, set_exception, CTX, Exception("connection lost"))
loop.run_until_complete(client_handler(CTX))
loop.close()

This should probably fixed upstream in the Streams API by the asyncio developers.

https://en.xdnf.cn/q/72811.html

Related Q&A

how to get place details from place id in google places api for python

I am using the Google Places API with Python to build a collective intelligence app for food. e.g. what restaurants are around, what ratings they have, what are their timings, etc.I am doing the follow…

pandas.algos._return_false causes PicklingError with dill.dump_session on CentOS

I have a code framework which involves dumping sessions with dill. This used to work just fine, until I started to use pandas. The following code raises a PicklingError on CentOS release 6.5:import pan…

How to send an image directly from flask server to html?

I am new to flask and am trying to make an app such an image is taken by the html and js from the webcam and then it is sent to the server with ajax request. I got this part. Then some processing is do…

Alternatives to nested numpy.where for multiconditional pandas operations?

I have a Pandas DataFrame with conditional column A and numeric column B. A B 1 foo 1.2 2 bar 1.3 3 foo 2.2I also have a Python dictionary that defines ranges of B which denote "success" g…

OpenCV findContours() just returning one external contour

Im trying to isolate letters in a captcha, I managed to filter a captcha and that result in this black and white image:But when I tried to separate the letters with findContours method of OpenCV it jus…

Selenium/ChromeDriver Unknown policy Errors

I am currently using Python (v3.5.1), Selenium (v3.7), and Chromedriver (v2.33).When I run the following command:from selenium import webdriver driver = webdriver.Chrome(C:\Program Files\ChromeWebdrive…

Mako escaping issue within Pyramid

I need to put javascript function to mako template. The first argument of this function is string, so I write in my *.mako file (dict(field_name=geom)):init_map(${field_name} );But when I see my html p…

All arguments should have the same length plotly

I try to do a bar graph using plotly.express but I find this problemAll arguments should have the same length. The length of argument y is 51, whereas the length of previously-processed arguments [x] …

Python curves intersection with fsolve() and function arguments using numpy

I am trying to use fsolve as quoted here : http://glowingpython.blogspot.gr/2011/05/hot-to-find-intersection-of-two.html,On order to find the intersection between two curves. Both curves basically are …

What is the Python freeze process?

The Python Doc states:Frozen modules are modules written in Python whose compiled byte-codeobject is incorporated into a custom-built Python interpreter byPython’s freeze utility. See Tools/freeze/ fo…