celery: Substantial drift from

2024/9/20 5:26:58

I have quite a problem with celery on my distribted system. I have couple of machines among different localizations and I've got a lot of warnings in my log files like:

"Substantial drift from celery@host [...]"

I was able to set date to return the same values (even that the machines are in different countries) but python print(utcoffset()) returns different results on main server and nodes.

How to fix that issue? I was unable to find any good solution except that utcoffset() should return the same value - but how?

Thanks for help.

Answer

I encountered a similar issue while starting flower:

[W 161208 08:42:00 state:74] Substantial drift from [email protected] may mean clocks are out of sync.  Current drift is 10800 seconds.  [orig: 2016-12-08 08:42:00.722560 recv: 2016-12-08 05:42:00.719938]

versions used:

celery==3.1.23 (and 3.1.25)
flower==0.9.1
rabbitmq 3.5.6

with settings:

CELERY_TIMEZONE='US/Pacific'
TIME_ZONE = 'US/Pacific'
USE_TZ = True

Looked through all available issue reports and suggestions on stackoverflow and git, but all indicated fixes that I should already have in place with > 3.1.23

This is how I solved it

My system and subsequent bash environment was running EST for timezone. As you can see above, the django app runs 'US/Pacific' (PST). This is is a 3 hour difference, or 10800 seconds.

By setting the bash timezone to PST as well, to match the django app, the drift error went away:

export TZ="US/Pacific"
https://en.xdnf.cn/q/72379.html

Related Q&A

jinja2 link to static files

I am trying to understand how to create a link to static files in jinja2.Everything I look up relates to Flask whereas I am using just webapp2 at this stage.My main.py file looks as follows:import os i…

How to upgrade tensorflow with GPU on google colaboratory

Currently google colaboratory uses tensorflow 1.4.1. I want to upgrade it to 1.5.0 version. Each time when i executed !pip install --upgrade tensorflow command, notebook instance succesfully upgrades t…

How to print warnings and errors when using setuptools (pip)

I am using setuptools to package code such that it can be easily installed using cd project_name && pip install .During the setup process, I want to warn the user about pre-existing config file…

TypeError: not supported between instances of State and State PYTHON 3

I am trying to utilize a PriorityQueue from the queue class. However, im having issues putting custom objects into my PQ. I have implemented the __cmp__ function below:def __cmp__(self, other):return (…

Threading and information passing -- how to

To reframe from confusion i have edited the question:one.pyimport threading count = 5 dev = threading.Thread(name=dev, target=dev,args=(workQueue,count,)) dev.setDaemon(True) dev.start() workQueue = Qu…

How to rename a node with Python LXML?

How do I rename a node using LXML? Specifically, how to rename a parent node i.e. a <body> tag while preserving all the underlying structure? I am parsing using the lxml.html module but suppos…

In python, selenium, how do I set the error messages for a wait?

I have the following code:WebDriverWait(self.driver, 20).until(expected_conditions.element_to_be_clickable(click))Now this sometimes fails and I know why it fails. But the error gives me TimeoutExcept…

Python: the mechanism behind list comprehension

When using list comprehension or the in keyword in a for loop context, i.e:for o in X:do_something_with(o)orl=[o for o in X]How does the mechanism behind in works? Which functions\methods within X do…

PRAW: Comment Submitters Username

Im developing a reddit bot that needs to know which user submitted a comment. According to the PRAW API wrapper docs, theres no specific way to get the username of a Comment objects author. Ideally I c…

Paramiko, exec_command get the output stream continuously [duplicate]

This question already has answers here:Get output from a Paramiko SSH exec_command continuously(6 answers)Closed 2 years ago.I dry on a Python script. I create a python script for a given IP will conne…