Running qsub with anaconda environment

2024/10/9 18:22:28

I have a program that usually runs inside a conda environmet in Linux, because I use it to manage my libraries, with this instructions:

source activate my_environment
python hello_world.py

How can I run hello_world.py in a high computer that works with PBS. Instructions explains to run adapting the code script.sh, shown below, and calling with the instruction qsub.

# script.sh
#!/bin/sh
#PBS -S /bin/sh
#PBS -N job_example
#PBS -l select=24
#PBS -j oe
cd $PBS_O_WORKDIR
mpiexec ./programa_mpi

How do I run hello_world.py with qsub using my anaconda environment?

Answer

Unless it is in the environment by default, one also needs to "load" conda as well, inside the SGE(qsub) script (similar thing would hold for a slurm script too I assume). For example I had installed conda to the directory seen below in the SGE script, so I export the path (if conda is installed as a module on an HPCC, then simply load that instead):

#!/bin/bash
#$ -q compute
#$ -l compute
#$ -cwd
#$ -N name
#$ -j yesexport PATH=$HOME/miniconda3/bin:$PATHsource activate my_environmentenvironment function code...
https://en.xdnf.cn/q/69988.html

Related Q&A

Flask Application was not able to create a URL adapter for request [duplicate]

This question already has answers here:Flask.url_for() error: Attempted to generate a URL without the application context being pushed(3 answers)Closed 10 months ago.I have this code which used to work…

Python typing deprecation

The latest typing docs has a lot of deprecation notices like the following: class typing.Deque(deque, MutableSequence[T]) A generic version of collections.deque.New in version 3.5.4.New in version 3.6.…

Cant install tensorflow with pip or anaconda

Does anyone know how to properly install tensorflow on Windows?Im currently using Python 3.7 (also tried with 3.6) and every time I get the same "Could not find a version that satisfies the requi…

send xml file to http using python

how can i send an xml file on my system to an http server using python standard library??

Why python Wnck window.activate(int(time.time()))

This to me is VERY strange. Could someone please explain why the activate() function should want a timestamp? Wouldnt 99.9% of the time be NOW or ASAP or "At your earliest convenience"? And…

Regex subsequence matching

Im using python but code in any language will do as well for this question.Suppose I have 2 strings. sequence =abcd string = axyzbdclkdIn the above example sequence is a subsequence of stringHow can I …

Read a Bytes image from Amazon Kinesis output in python

I used imageio.get_reader(BytesIO(a), ffmpeg) to load a bytes image and save it as normal image.But the below error throws when I read the image using imageio.get_reader(BytesIO(a), ffmpeg)Traceback …

How to compute optical flow using tvl1 opencv function

Im trying to find python example for computing optical flow with tvl1 opencv function createOptFlow_DualTVL1 but it seems that there isnt enough documentation for it.Could anyone please let me do that?…

Python 3.3.4: python-daemon-3K ; How to use runner

Struggling to try and get a python daemon to work using Python 3.3.4. Im using the latest version of the python-daemon-3K from PyPi i.e. 1.5.8Starting point is the following code found How do you creat…

How to select specific data variables from xarray dataset

BACKGROUND I am trying to download GFS weather data netcdf4 files via xarray & OPeNDAP. Big thanks to Vorticity0123 for their prior post, which allowed me to get the bones of the python script sort…