run a crontab job using an anaconda env

2024/11/19 11:32:07

I want to have a cron job execute a python script using an already existing anaconda python environment called my_env. The only thing I can think to do is have the cron job run a script called my_script.bash which in turn activates the env and then runs the python script.

#!/bin/bash
source activate my_env
python ~/my_project/main.py

Trying to execute this script from the command lines doesn't work:

$ sh scripts/my_script.bash
scripts/my_script.bash: 9: scripts/my_script.bash: source: not found

What do I need to do to make sure the proper environment is activated. Its ok to explain it to me like I'm 5.

Answer

I recently switched from canopy to Anaconda precisely to get away from having to activate an env in cron jobs. Anaconda makes this very simple, based on the PATH enviornment variable. (I'm using miniconda not the full Anaconds install but I believe anaconda should work the same way)

There are two different approaches, I've tested;

  • Add a shebang in your python script, main.py

    #!/home/users/user_name/miniconda2/envs/my_env/bin/python

  • Add PATH to the top of your crontab

    PATH=/home/users/user_name/miniconda2/envs/my_env/bin

Update:

Jérôme's answer and cbarrick's comments are correct. I just got burned using the above approach in a Conda env which needed pynco, which needs the full conda environment to find proper the nco commands, such as ncks, ncrcat. Solved by running a bash script from cron which calls conda activate first.

https://en.xdnf.cn/q/26446.html

Related Q&A

pandas - Merging on string columns not working (bug?)

Im trying to do a simple merge between two dataframes. These come from two different SQL tables, where the joining keys are strings:>>> df1.col1.dtype dtype(O) >>> df2.col2.dtype dtyp…

Making a chart bigger in size

Im trying to get a bigger chart. However, the figure method from matplotlib does not seem to be working properly. I get a message, which is not an error: <matplotlib.figure.Figure at 0xa25f7f0>My…

index of non NaN values in Pandas

From Pandas data frame, how to get index of non "NaN" values?My data frame isA b c 0 1 q1 1 1 2 NaN 3 2 3 q2 3 3 4 q1 NaN 4 5 q2 7And I want the…

How can I type-check variables in Python? [duplicate]

This question already has answers here:Whats the canonical way to check for type in Python?(16 answers)Closed 3 months ago.I have a Python function that takes a numeric argument that must be an intege…

Py_INCREF/DECREF: When

Is one correct in stating the following:If a Python object is created in a C function, but the function doesnt return it, no INCREF is needed, but a DECREF is. [false]If the function does return it, yo…

pop/remove items out of a python tuple

I am not sure if I can make myself clear but will try.I have a tuple in python which I go through as follows (see code below). While going through it, I maintain a counter (lets call it n) and pop item…

Difference between frompyfunc and vectorize in numpy

What is the difference between vectorize and frompyfunc in numpy?Both seem very similar. What is a typical use case for each of them?Edit: As JoshAdel indicates, the class vectorize seems to be built…

Jupyter notebook command does not work on Mac

I installed jupyter using pip on my macbook air. Upon trying to execute the command jupyter notebook, I get an error jupyter: notebook is not a Jupyter commandI used the --h option to get a listing of …

Recursively compare two directories to ensure they have the same files and subdirectories

From what I observe filecmp.dircmp is recursive, but inadequate for my needs, at least in py2. I want to compare two directories and all their contained files. Does this exist, or do I need to build …

Specific reasons to favor pip vs. conda when installing Python packages

I use miniconda as my default python installation. What is the current (2019) wisdom regarding when to install something with conda vs. pip?My usual behavior is to install everything with pip, and onl…