I have a python code in which I am calling a shell command. The part of the code where I did the shell command is:

try:def parse(text_list):text = '\n'.join(text_list)cwd = os.getcwd()os.chdir("/var/www/html/alenza/hdfs/user/alenza/sree_account/sree_project/src/core/data_analysis/syntaxnet/models/syntaxnet")synnet_output = subprocess.check_output(["echo '%s' | syntaxnet/demo.sh 2>/dev/null"%text], shell = True)os.chdir(cwd)return synnet_output
except Exception as e:sys.stdout.write(str(e))

Now, when i run this code on a local file with some sample input (I did cat /home/sree/example.json | python parse.py) it works fine and I get the required output. But I am trying to run the code with an input on my HDFS (the same cat command but input file path is from HDFS) which contains exactly the same type of json entries and it fails with an error:

/bin/sh: line 62: to: command not found
list index out of range

I read similar questions on Stack Overflow and the solution was to include a Shebang line for the shell script that is being called. I do have the shebang line #!/usr/bin/bash in demo.sh script.

Also, which bash gives /usr/bin/bash.

Someone please elaborate.

You rarely, if ever, want to combine passing a list argument with shell=True. Just pass the string:

synnet_output = subprocess.check_output("echo '%s' | syntaxnet/demo.sh 2>/dev/null"%(text,), shell=True)

However, you don't really need a shell pipeline here.

from subprocess import check_output
from StringIO import StringIO  # from io import StringIO in Python 3
synnet_output = check_output(["syntaxnet/demo.sh"],stdin=StringIO(text),stderr=os.devnull)