Airflow - Disable heartbeat logs

2024/10/15 10:15:27

My logs are getting completely flooded with useless messages for every heartbeat.

[2019-11-27 21:32:47,890] {{logging_mixin.py:112}} INFO - [2019-11-27 21:32:47,889] {local_task_job.py:124} WARNING - Time since last heartbeat(0.02 s) < heartrate(5.0 s), sleeping for 4.983326 s
[2019-11-27 21:32:52,921] {{logging_mixin.py:112}} INFO - [2019-11-27 21:32:52,921] {local_task_job.py:124} WARNING - Time since last heartbeat(0.02 s) < heartrate(5.0 s), sleeping for 4.984673 s
[2019-11-27 21:32:57,949] {{logging_mixin.py:112}} INFO - [2019-11-27 21:32:57,949] {local_task_job.py:124} WARNING - Time since last heartbeat(0.01 s) < heartrate(5.0 s), sleeping for 4.985953 s
[2019-11-27 21:33:02,980] {{logging_mixin.py:112}} INFO - [2019-11-27 21:33:02,980] {local_task_job.py:124} WARNING - Time since last heartbeat(0.02 s) < heartrate(5.0 s), sleeping for 4.984688 s
[2019-11-27 21:33:08,008] {{logging_mixin.py:112}} INFO - [2019-11-27 21:33:08,008] {local_task_job.py:124} WARNING - Time since last heartbeat(0.01 s) < heartrate(5.0 s), sleeping for 4.987069 s
[2019-11-27 21:33:13,041] {{logging_mixin.py:112}} INFO - [2019-11-27 21:33:13,041] {local_task_job.py:124} WARNING - Time since last heartbeat(0.02 s) < heartrate(5.0 s), sleeping for 4.984275 s
[2019-11-27 21:33:18,072] {{logging_mixin.py:112}} INFO - [2019-11-27 21:33:18,071] {local_task_job.py:124} WARNING - Time since last heartbeat(0.01 s) < heartrate(5.0 s), sleeping for 4.986545 s
[2019-11-27 21:33:23,101] {{logging_mixin.py:112}} INFO - [2019-11-27 21:33:23,100] {local_task_job.py:124} WARNING - Time since last heartbeat(0.02 s) < heartrate(5.0 s), sleeping for 4.984889 s
[2019-11-27 21:33:28,130] {{logging_mixin.py:112}} INFO - [2019-11-27 21:33:28,130] {local_task_job.py:124} WARNING - Time since last heartbeat(0.01 s) < heartrate(5.0 s), sleeping for 4.986234 s
[2019-11-27 21:33:33,160] {{logging_mixin.py:112}} INFO - [2019-11-27 21:33:33,160] {local_task_job.py:124} WARNING - Time since last heartbeat(0.01 s) < heartrate(5.0 s), sleeping for 4.986669 s

Our logs are now too noisy to be useful. How can we disable these? Changing the log level to something like logging.WARN is not an option because we have lots of actually useful information at the INFO level.

EDIT: I'm using Airflow 1.10.6

Answer

Currently, it is not possible to disable it. I have created a PR that changes the log level to DEBUG. Unfortunately, this won't be available until the next release (1.10.7).

The short-time solution would to manually apply the change in the PR to your Airflow installation.

https://en.xdnf.cn/q/69296.html

Related Q&A

different validation in drf serializer per request method

Lets say i have a model like so:class MyModel(models.Model):first_field = models.CharField()second_field = models.CharField()and an API view like so:class MyModelDetailAPI(GenericAPIView):serializer_cl…
READ MORE...

How to import r-packages in Python

Im a bit troubled with a simple thing. I was trying to install a package called hunspell, but I discovered it is originally an R package. I installed this version: https://anaconda.org/conda-forge/r-hu…
READ MORE...

XPath predicate with sub-paths with lxml?

Im trying to understand and XPath that was sent to me for use with ACORD XML forms (common format in insurance). The XPath they sent me is (truncated for brevity):./PersApplicationInfo/InsuredOrPrinci…
READ MORE...

Best way to access and close a postgres database using python dataset

import dataset from sqlalchemy.pool import NullPooldb = dataset.connect(path_database, engine_kwargs={poolclass: NullPool})table_f1 = db[name_table] # Do operations on table_f1db.commit() db.execut…
READ MORE...

Using different binds in the same class in Flask-SQLAlchemy

I currently have multiple databases with identical Tables and Columns (but different data inside). So clearly I need to use binds to access all of them, but its apparently not as simple as doing this:c…
READ MORE...

Correctly parse date string with timezone information

Im receiving a formatted date string like this via the pivotal tracker API: "2012/06/05 17:42:29 CEST"I want to convert this string to a UTC datetime object, it looks like python-dateutil doe…
READ MORE...

Can I add a sequence of markers on a Folium map?

Suppose I had a list, or pandas series, or latitude longitude pairs. With Folium, I can plot markers for a single pair of coordinates using coords = [46.8354, -121.7325] map_4 = folium.Map(location=[4…
READ MORE...

Tkinter in Python 3.4 on Windows dont post internal clipboard data to the Windows clipboard on exit

I use the following code to place result of my small scripts in clipboard.from tkinter import Tk r = Tk() r.withdraw() r.clipboard_clear() r.clipboard_append("Result")It works fine on Python …
READ MORE...

How do I group date by month using pd.Grouper?

Ive searched stackoverflow to find out how to group DateTime by month and for some reason I keep receiving this error, even after I pass the dataframe through pd.to.datetimeTypeError: Only valid with D…
READ MORE...

Python Too many indices for array

I am reading a file in python using pandas and then saving it in a numpy array. The file has the dimension of 11303402 rows x 10 columns. I need to split the data for cross validation and for that I …
READ MORE...
Latest Q&A

Copyright @ 2022~2023