My objective is to run Python 3 code on the AWS Lambda Service, which currently only supports Python 2.7. These are the steps I have done.

• Since I work on a Mac, setup a docker image similar to the AWS Lambda Linux instance.

• Build Python3 from source on the docker image.

• In the docker image create a virtual environment and copy it to my project.

• AWS Lambda requires you to create a zip of the code and upload it to their service. For this prototype, I have a zip with three artifacts at the root

  1. handler.py: This is a Python 2.7 file. The handler function in this file will be executed by the AWS Lambda Service when an event occurs (e.g. When a new file is created in a S3 bucket).

     def handler(event, context):execution_uuid = uuid.uuid4()commands = '''source venv/bin/activate && venv/bin/python3.6 ./handler_python3.py --execution_uuid {ex_uuid}'''.format(ex_uuid=str(execution_uuid))p = Popen('/bin/bash', shell=True, stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)stdout, stderr = p.communicate(commands)pprint(stdout)pprint(stderr)
     

  2. handler_python3.py. This is the Python3 file called by the earlier handler.py file. Note the execution_uuid being read. I have taken out the code that uses it for brevity but I do need it and I am using argparse to extract it.

     def read_execution_uuid():import argparseparser = argparse.ArgumentParser()parser.add_argument("--execution_uuid", required=True)args = parser.parse_args()return args.execution_uuiddef handler(event, context):import sysprint(sys.path)if __name__ == '__main__':execution_uuid = read_execution_uuid()handler(event, context)
     

     3. venv folder. This is the virtual environment folder that I copied from the docker image.

When I run the AWS Lambda Service, I get the following error

Traceback (most recent call last):File "./handler_python3.py", line 38, in <module>execution_uuid = read_execution_uuid()File "./handler_python3.py", line 7, in read_execution_uuidimport argparse
ModuleNotFoundError: No module named \'argparse\'

Notes:

• If I remove the argparse code and the handler function in handler_python3.py executes, it shows the following values for sys.path

  ['/var/task', '/var/runtime', '/var/task/venv/lib/python36.zip', '/var/task/venv/lib/python3.6', '/var/task/venv/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6', '/var/task/venv/lib/python3.6/site-packages']
  

Notes:

• I can install argparse explicitly. But I'd like not to.
• Note the source venv/bin/activate command in the python 2.7 file handler.py. That doesn't work on the Lambda instance though it works locally.

Creating a virtual environment does not copy all the modules from the /usr/local/lib/python3.6 directory. I had to copy all the files there.