Does python logging.FileHandler use block buffering by default?

2024/10/7 20:35:08

The logging handler classes have a flush() method.

And looking at the code, logging.FileHandler does not pass a specific buffering mode when calling open(). Therefore when you write to a log file, it will be buffered using a default block size.

Is that correct?


It surprises me, because when I manage my own system, I am used to watching log files as a live (or near-live) view on the system. For this use case, line-buffering is desired. Also, traditional syslog() to a logging daemon does not buffer messages.


I am interested in Python versions 2.7 and 3.7.

Answer

Not really. It will flush each individual message, which is what you want.

FileHandler inherits from StreamHandler. StreamHandler calls self.flush() after every write() to the stream.

The flush() method starts to make more sense if you look at logging.MemoryHandler. For programs which want to add buffering, MemoryHandler allows wrapping another handler, and buffering a set number of messages. It will also flush immediately on messages above a set severity level. logging does not include a handler which automatically flushes every second or so, but you could always write one yourself.

The flush calls in StreamHandler also mean that it does what you want, if your program is run as a systemd service and you log to stderr. Python 3 requires flushes in this case. Python 3 currently uses block buffering for stderr when it is not a TTY. See discussion on Python issue 13597

Possible reason for my mistake

I think I was confused by the StreamHandler code. If the user never needed to call the flush() method, why would StreamHandler define a non-empty, publicly documented implementation?

I think I was assuming too much, and I did not allow for how inheritance (argh) is used here. E.g. the base Handler class has an empty flush() method, but StreamHandler does not want to inherit that because it has a weird docstring "This version does nothing and is intended to be implemented by subclasses".

https://en.xdnf.cn/q/70197.html

Related Q&A

Non brute force solution to Project Euler problem 25

Project Euler problem 25:The Fibonacci sequence is defined by the recurrence relation: Fn = Fn−1 + Fn−2, where F1 = 1 and F2 = 1. Hence the first 12 terms will be F1 = 1, F2 = 1, F3 = 2, F4 = 3, F5 =…

python:class attribute/variable inheritance with polymorphism?

In my endeavours as a python-apprentice i got recently stuck at some odd (from my point of view) behaviour if i tried to work with class attributes. Im not complaining, but would appreciate some helpfu…

Unable to load firefox in selenium webdriver in python

I have installed Python 3.6.2, Selenium 3.5.0 with GeckoDriver 0.18.0 and the firefox version is 54.0.1version on windows 7. I am trying to run a selenium script which is loading a firefox where i get …

Plot hyperplane Linear SVM python

I am trying to plot the hyperplane for the model I trained with LinearSVC and sklearn. Note that I am working with natural languages; before fitting the model I extracted features with CountVectorizer …

Calculating plugin dependencies

I have the need to create a plugin system that will have dependency support and Im not sure the best way to account for dependencies. The plugins will all be subclassed from a base class, each with i…

Vectorization: Not a valid collection

I wanna vectorize a txt file containing my training corpus for the OneClassSVM classifier. For that Im using CountVectorizer from the scikit-learn library. Heres below my code: def file_to_corpse(file…

Solve a simple packing combination with dependencies

This is not a homework question, but something that came up from a project I am working on. The picture above is a packing configuration of a set of boxes, where A,B,C,D is on the first layer and E,F,G…

ImportError: cannot import name FFProbe

I cant get the ffprobe package to work in Python 3.6. I installed it using pip, but when I type import ffprobe it saysTraceback (most recent call last): File "<stdin>", line 1, in <m…

Generate larger synthetic dataset based on a smaller dataset in Python

I have a dataset with 21000 rows (data samples) and 102 columns (features). I would like to have a larger synthetic dataset generated based on the current dataset, say with 100000 rows, so I can use it…

Executing python script in android terminal emulator

I installed python 2.7 in my Android device and I tried executing a python script by typing the command in terminal emulator. The problem is that although I use the full path for python the following e…