How can I configure gunicorn to use a consistent error log format?

2024/10/4 3:32:47

I am using Gunicorn in front of a Python Flask app. I am able to configure the access log format using the --access-log-format command line parameter when I run gunicorn. But I can't figure out how to configure the error logs.

I would be fine with the default format, except it's not consistent. It looks like Gunicorn status messages have one format, but application exceptions have a different format. This is making it difficult to use log aggregation.

For example, here a few messages from Gunicorn's error log. The first few lines have a different format than the exception line. Event the datetime format is different.

[2017-07-13 16:33:24 +0000] [15] [INFO] Booting worker with pid: 15
[2017-07-13 16:33:24 +0000] [16] [INFO] Booting worker with pid: 16
[2017-07-13 16:33:24 +0000] [17] [INFO] Booting worker with pid: 17
[2017-07-13 16:33:24 +0000] [18] [INFO] Booting worker with pid: 18
[2017-07-13 18:31:11,580] ERROR in app: Exception on /api/users [POST]
Traceback (most recent call last):File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1982, in wsgi_appresponse = self.full_dispatch_request()File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1614, in full_dispatch_requestrv = self.handle_user_exception(e)
...

What's the best way to configure Gunicorn to use consistent formatting for its error logs?

Answer

Using this logging config file, I was able to change the error log format

[loggers]
keys=root, gunicorn.error[handlers]
keys=error_console[formatters]
keys=generic[logger_root]
level=INFO
handlers=error_console[logger_gunicorn.error]
level=INFO
handlers=error_console
propagate=0
qualname=gunicorn.error[handler_error_console]
class=StreamHandler
formatter=generic
args=(sys.stderr, )[formatter_generic]
format=%(asctime)s %(levelname)-5s [%(module)s] ~ %(message)s
datefmt=%Y-%m-%d %H:%M:%S %Z
class=logging.Formatter

The key is to overwrite the gunicorn.error logger config, and the snipped above does exactly that.

Note the propagate=0 field, it is important otherwise your log messages will be printed twice (gunicorn always keeps the default logging config).

https://en.xdnf.cn/q/70655.html

Related Q&A

Implementing seq2seq with beam search

Im now implementing seq2seq model based on the example code that tensorflow provides. And I want to get a top-5 decoder outputs to do a reinforcement learning.However, they implemented translation mode…

Pandas Random Weighted Choice

I would like to randomly select a value in consideration of weightings using Pandas.df:0 1 2 3 4 5 0 40 5 20 10 35 25 1 24 3 12 6 21 15 2 72 9 36 18 63 45 3 8 1 4 2 7 5 4 16 2 8 4…

Matplotlib TypeError: NoneType object is not callable

Ive run this code many times but now its failing. Matplotlib wont work for any example, even the most trivial. This is the error Im getting, but Im not sure what to make of it. I know this is vague and…

Resize image faster in OpenCV Python

I have a lot of image files in a folder (5M+). These images are of different sizes. I want to resize these images to 128x128. I used the following function in a loop to resize in Python using OpenCVdef…

How to install Yandex CatBoost on Anaconda x64?

Iv successfully installed CatBoost via pip install catboostBut Iv got errors, when I tried sample python script in Jupiter Notebookimport numpy as np from catboost import CatBoostClassifierImportError:…

pyspark returns a no module named error for a custom module

I would like to import a .py file that contains some modules. I have saved the files init.py and util_func.py under this folder:/usr/local/lib/python3.4/site-packages/myutilThe util_func.py contains al…

Perform a conditional operation on a pandas column

I know that this should be simple, but I want to take a column from a pandas dataframe, and for only the entries which meet some condition (say less than 1), multiply by a scalar (say 2).For example, i…

How to programmatically get SVN revision number?

Like this question, but without the need to actually query the SVN server. This is a web-based project, so I figure Ill just use the repository as the public view (unless someone can advise me why this…

Convert fractional years to a real date in Python

How do I convert fractional years to a real date by using Python? E. g. I have an array [2012.343, 2012.444, 2012.509] containing fractional years and I would like to get "yyyy-mm-dd hh:mm".

Django template: Translate include with variable

I have a template in which you can pass a text variable. I want to include this template into another one but with a translated text as its variable. How can you achieve this?I would like something li…