Execute Python (selenium) script in crontab

2024/10/4 11:28:11

I have read most of the python/cron here in stackoverflow and yet couldn't make my script run. I understood that I need to run my script through shell (using zsh & ipython by the way), but really I have no idea what to do :/

My simple code:

In the crontab-

*/1 * * * * ipython /home/usr/Data/progs/cron_test.py

My python script-

import picklefrom selenium import webdriverdriver = webdriver.Firefox()
driver.get('http://www.google.com')t=driver.current_url
pickle.dump(t,open('noreal','wb'))

I have tried some things already but to no avail:

#!python ../python etc
SHELL = /usr/bin/zsh
PATH =/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

Simply because I don't know what the problem is, I assume something with running python script via interpreter, but I don't know what I am doing really :)

A working solution would be great, and I would really appreciate an explanation to accompany any solution so I could understand the why and how and not just be like 'It works! Thanks! Bye!'

Thanks for any help!

Update So far I have narrowed down the problem, and now python runs with the following settings:

*/3 * * * * /usr/local/bin/ipython /home/user/Data/progs/RF/cron_test.py

and I got a traceback:

[1;31m---------------------------------------------------------------------------[0m
[1;31mWebDriverException[0m                        Traceback (most recent call last)
[1;32m/usr/local/lib/python2.7/dist-packages/IPython/utils/py3compat.pyc[0m in [0;36mexecfile[1;34m(fname, *where)[0m
[0;32m    176[0m             [1;32melse[0m[1;33m:[0m[1;33m[0m[0m
[0;32m    177[0m                 [0mfilename[0m [1;33m=[0m [0mfname[0m[1;33m[0m[0m
[1;32m--> 178[1;33m             [0m__builtin__[0m[1;33m.[0m[0mexecfile[0m[1;33m([0m[0mfilename[0m[1;33m,[0m [1;33m*[0m[0mwhere[0m[1;33m)[0m[1;33m[0m[0m
[0m
[1;32m/home/user/Data/progs/FB/cron_test.py[0m in [0;36m<module>[1;34m()[0m
[0;32m      9[0m [1;33m[0m[0m
[0;32m     10[0m [1;33m[0m[0m
[1;32m---> 11[1;33m [0mdriver[0m [1;33m=[0m [0mwebdriver[0m[1;33m.[0m[0mFirefox[0m[1;33m([0m[1;33m)[0m[1;33m[0m[0m
[0m[0;32m     12[0m [0mdriver[0m[1;33m.[0m[0mget[0m[1;33m([0m[1;34m'http://www.google.com'[0m[1;33m)[0m[1;33m[0m[0m
[0;32m     13[0m [1;33m[0m[0m[1;32m/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.pyc[0m in [0;36m__init__[1;34m(self, firefox_profile, firefox_binary, timeout, capabilities, proxy)[0m
[0;32m     58[0m         RemoteWebDriver.__init__(self,
[0;32m     59[0m             command_executor=ExtensionConnection("127.0.0.1", self.profile,
[1;32m---> 60[1;33m             self.binary, timeout),
[0m[0;32m     61[0m             desired_capabilities=capabilities)
[0;32m     62[0m         [0mself[0m[1;33m.[0m[0m_is_remote[0m [1;33m=[0m [0mFalse[0m[1;33m[0m[0m[1;32m/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/extension_connection.pyc[0m in [0;36m__init__[1;34m(self, host, firefox_profile, firefox_binary, timeout)[0m
[0;32m     45[0m         [0mself[0m[1;33m.[0m[0mprofile[0m[1;33m.[0m[0madd_extension[0m[1;33m([0m[1;33m)[0m[1;33m[0m[0m
[0;32m     46[0m [1;33m[0m[0m
[1;32m---> 47[1;33m         [0mself[0m[1;33m.[0m[0mbinary[0m[1;33m.[0m[0mlaunch_browser[0m[1;33m([0m[0mself[0m[1;33m.[0m[0mprofile[0m[1;33m)[0m[1;33m[0m[0m
[0m[0;32m     48[0m         [0m_URL[0m [1;33m=[0m [1;34m"http://%s:%d/hub"[0m [1;33m%[0m [1;33m([0m[0mHOST[0m[1;33m,[0m [0mPORT[0m[1;33m)[0m[1;33m[0m[0m
[0;32m     49[0m         RemoteConnection.__init__([1;32m/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/firefox_binary.pyc[0m in [0;36mlaunch_browser[1;34m(self, profile)[0m
[0;32m     59[0m [1;33m[0m[0m
[0;32m     60[0m         [0mself[0m[1;33m.[0m[0m_start_from_profile_path[0m[1;33m([0m[0mself[0m[1;33m.[0m[0mprofile[0m[1;33m.[0m[0mpath[0m[1;33m)[0m[1;33m[0m[0m
[1;32m---> 61[1;33m         [0mself[0m[1;33m.[0m[0m_wait_until_connectable[0m[1;33m([0m[1;33m)[0m[1;33m[0m[0m
[0m[0;32m     62[0m [1;33m[0m[0m
[0;32m     63[0m     [1;32mdef[0m [0mkill[0m[1;33m([0m[0mself[0m[1;33m)[0m[1;33m:[0m[1;33m[0m[0m[1;32m/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/firefox_binary.pyc[0m in [0;36m_wait_until_connectable[1;34m(self)[0m
[0;32m     98[0m                 raise WebDriverException("The browser appears to have exited "
[0;32m     99[0m                       [1;34m"before we could connect. The output was: %s"[0m [1;33m%[0m[1;33m[0m[0m
[1;32m--> 100[1;33m                       self._get_firefox_output())
[0m[0;32m    101[0m             [1;32mif[0m [0mcount[0m [1;33m==[0m [1;36m30[0m[1;33m:[0m[1;33m[0m[0m
[0;32m    102[0m                 [0mself[0m[1;33m.[0m[0mkill[0m[1;33m([0m[1;33m)[0m[1;33m[0m[0m[1;31mWebDriverException[0m: Message: 'The browser appears to have exited before we could connect. The output was: Error: no display specified\n' 
Answer

As seen here, you need to give cron a display:

30 5 * * * export DISPLAY=:0; /home/me/good_morning.sh
https://en.xdnf.cn/q/70611.html

Related Q&A

Get post data from ajax post request in python file

Im trying to post some data with an ajax post request and execute a python file, retrieving the data in the python file, and return a result.I have the following ajax code$(function () {$("#upload…

How to implement maclaurin series in keras?

I am trying to implement expandable CNN by using maclaurin series. The basic idea is the first input node can be decomposed into multiple nodes with different orders and coefficients. Decomposing singl…

Rowwise min() and max() fails for column with NaNs

I am trying to take the rowwise max (and min) of two columns containing datesfrom datetime import date import pandas as pd import numpy as np df = pd.DataFrame({date_a : [date(2015, 1, 1), date(2012…

Convert column suffixes from pandas join into a MultiIndex

I have two pandas DataFrames with (not necessarily) identical index and column names. >>> df_L = pd.DataFrame({X: [1, 3], Y: [5, 7]})>>> df_R = pd.DataFrame({X: [2, 4], Y: [6, 8]})I c…

sys-package-mgr*: cant create package cache dir when run python script with Jython

I want to run Python script with Jython. the result show correctly, but at the same time there is an warning message, "sys-package-mgr*: cant create package cache dir"How could I solve this p…

Python WWW macro

i need something like iMacros for Python. It would be great to have something like that:browse_to(www.google.com) type_in_input(search, query) click_button(search) list = get_all(<p>)Do you know …

Django custom context_processors in render_to_string method

Im building a function to send email and I need to use a context_processor variable inside the HTML template of the email, but this dont work.Example:def send_email(plain_body_template_name, html_body_…

Using string as variable name

Is there any way for me to use a string to call a method of a class? Heres an example that will hopefully explain better (using the way I think it should be):class helloworld():def world(self):print &…

How to sum all amounts by date in pandas dataframe?

I have dataframe with fields last_payout and amount. I need to sum all amount for each month and plot the output. df[[last_payout,amount]].dtypeslast_payout datetime64[ns] amount float64 d…

Unable to import decimal in Python 2.7 or Python 3.3 [duplicate]

This question already has answers here:Importing a library from (or near) a script with the same name raises "AttributeError: module has no attribute" or an ImportError or NameError(4 answers…