How can you bundle all your python code into a single zip file?

2024/11/19 3:20:49

It would be convenient when distributing applications to combine all of the eggs into a single zip file so that all you need to distribute is a single zip file and an executable (some custom binary that simply starts, loads the zip file's main function and kicks python off or similar).

I've seen some talk of doing this online, but no examples of how to actually do it.

I'm aware that you can (if it is zip safe) convert eggs into zip files.

What I'm not sure about is:

Can you somehow combine all your eggs into a single zip file? If so, how?

How would you load and run code from a specific egg?

How would you ensure that the code in that egg could access all the dependencies (ie. other eggs in the zip file)?

People ask this sort of stuff a lot and get answers like; use py2exe. Yes, I get it, that's one solution. It's not the question I'm asking here though...

Answer

You can automate most of the work with regular python tools. Let's start with clean virtualenv.

[zart@feena ~]$ mkdir ziplib-demo
[zart@feena ~]$ cd ziplib-demo
[zart@feena ziplib-demo]$ virtualenv .
New python executable in ./bin/python
Installing setuptools.............done.
Installing pip...............done.

Now let's install set of packages that will go into zipped library. The trick is to force installing them into specific directory.

(Note: don't use --egg option either on command-line or in pip.conf/pip.ini because it will break file layout making it non-importable in zip)

[zart@feena ziplib-demo]$ bin/pip install --install-option --install-lib=$PWD/unpacked waitress
Downloading/unpacking waitressDownloading waitress-0.8.5.tar.gz (112kB): 112kB downloadedRunning setup.py egg_info for package waitressRequirement already satisfied (use --upgrade to upgrade): setuptools in ./lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg (from waitress)
Installing collected packages: waitressRunning setup.py install for waitressInstalling waitress-serve script to /home/zart/ziplib-demo/bin
Successfully installed waitress
Cleaning up...

Update: pip now has -t <path> switch, that does the same thing as --install-option --install-lib=.

Now let's pack all of them into one zip

[zart@feena ziplib-demo]$ cd unpacked
[zart@feena unpacked]$ ls
waitress  waitress-0.8.5-py2.7.egg-info
[zart@feena unpacked]$ zip -r9 ../library.zip *adding: waitress/ (stored 0%)adding: waitress/receiver.py (deflated 71%)adding: waitress/server.pyc (deflated 64%)adding: waitress/utilities.py (deflated 62%)adding: waitress/trigger.pyc (deflated 63%)adding: waitress/trigger.py (deflated 61%)adding: waitress/receiver.pyc (deflated 60%)adding: waitress/adjustments.pyc (deflated 51%)adding: waitress/compat.pyc (deflated 56%)adding: waitress/adjustments.py (deflated 60%)adding: waitress/server.py (deflated 68%)adding: waitress/channel.py (deflated 72%)adding: waitress/task.pyc (deflated 57%)adding: waitress/tests/ (stored 0%)adding: waitress/tests/test_regression.py (deflated 63%)adding: waitress/tests/test_functional.py (deflated 88%)adding: waitress/tests/test_parser.pyc (deflated 76%)adding: waitress/tests/test_trigger.pyc (deflated 73%)adding: waitress/tests/test_init.py (deflated 72%)adding: waitress/tests/test_utilities.pyc (deflated 78%)adding: waitress/tests/test_buffers.pyc (deflated 79%)adding: waitress/tests/test_trigger.py (deflated 82%)adding: waitress/tests/test_buffers.py (deflated 86%)adding: waitress/tests/test_runner.py (deflated 75%)adding: waitress/tests/test_init.pyc (deflated 69%)adding: waitress/tests/__init__.pyc (deflated 21%)adding: waitress/tests/support.pyc (deflated 48%)adding: waitress/tests/test_utilities.py (deflated 73%)adding: waitress/tests/test_channel.py (deflated 87%)adding: waitress/tests/test_task.py (deflated 87%)adding: waitress/tests/test_functional.pyc (deflated 82%)adding: waitress/tests/__init__.py (deflated 5%)adding: waitress/tests/test_compat.pyc (deflated 53%)adding: waitress/tests/test_receiver.pyc (deflated 79%)adding: waitress/tests/test_adjustments.py (deflated 78%)adding: waitress/tests/test_adjustments.pyc (deflated 74%)adding: waitress/tests/test_server.pyc (deflated 73%)adding: waitress/tests/fixtureapps/ (stored 0%)adding: waitress/tests/fixtureapps/filewrapper.pyc (deflated 59%)adding: waitress/tests/fixtureapps/getline.py (deflated 37%)adding: waitress/tests/fixtureapps/nocl.py (deflated 47%)adding: waitress/tests/fixtureapps/sleepy.pyc (deflated 44%)adding: waitress/tests/fixtureapps/echo.py (deflated 40%)adding: waitress/tests/fixtureapps/error.py (deflated 52%)adding: waitress/tests/fixtureapps/nocl.pyc (deflated 48%)adding: waitress/tests/fixtureapps/getline.pyc (deflated 32%)adding: waitress/tests/fixtureapps/writecb.pyc (deflated 42%)adding: waitress/tests/fixtureapps/toolarge.py (deflated 37%)adding: waitress/tests/fixtureapps/__init__.pyc (deflated 20%)adding: waitress/tests/fixtureapps/writecb.py (deflated 50%)adding: waitress/tests/fixtureapps/badcl.pyc (deflated 44%)adding: waitress/tests/fixtureapps/runner.pyc (deflated 58%)adding: waitress/tests/fixtureapps/__init__.py (stored 0%)adding: waitress/tests/fixtureapps/filewrapper.py (deflated 74%)adding: waitress/tests/fixtureapps/runner.py (deflated 41%)adding: waitress/tests/fixtureapps/echo.pyc (deflated 42%)adding: waitress/tests/fixtureapps/groundhog1.jpg (deflated 24%)adding: waitress/tests/fixtureapps/error.pyc (deflated 48%)adding: waitress/tests/fixtureapps/sleepy.py (deflated 42%)adding: waitress/tests/fixtureapps/toolarge.pyc (deflated 43%)adding: waitress/tests/fixtureapps/badcl.py (deflated 45%)adding: waitress/tests/support.py (deflated 52%)adding: waitress/tests/test_task.pyc (deflated 78%)adding: waitress/tests/test_channel.pyc (deflated 78%)adding: waitress/tests/test_regression.pyc (deflated 68%)adding: waitress/tests/test_parser.py (deflated 80%)adding: waitress/tests/test_server.py (deflated 78%)adding: waitress/tests/test_receiver.py (deflated 87%)adding: waitress/tests/test_compat.py (deflated 51%)adding: waitress/tests/test_runner.pyc (deflated 72%)adding: waitress/__init__.pyc (deflated 50%)adding: waitress/channel.pyc (deflated 58%)adding: waitress/runner.pyc (deflated 54%)adding: waitress/buffers.py (deflated 74%)adding: waitress/__init__.py (deflated 61%)adding: waitress/runner.py (deflated 58%)adding: waitress/parser.py (deflated 69%)adding: waitress/compat.py (deflated 69%)adding: waitress/buffers.pyc (deflated 69%)adding: waitress/utilities.pyc (deflated 60%)adding: waitress/parser.pyc (deflated 53%)adding: waitress/task.py (deflated 72%)adding: waitress-0.8.5-py2.7.egg-info/ (stored 0%)adding: waitress-0.8.5-py2.7.egg-info/dependency_links.txt (stored 0%)adding: waitress-0.8.5-py2.7.egg-info/installed-files.txt (deflated 83%)adding: waitress-0.8.5-py2.7.egg-info/top_level.txt (stored 0%)adding: waitress-0.8.5-py2.7.egg-info/PKG-INFO (deflated 65%)adding: waitress-0.8.5-py2.7.egg-info/not-zip-safe (stored 0%)adding: waitress-0.8.5-py2.7.egg-info/SOURCES.txt (deflated 71%)adding: waitress-0.8.5-py2.7.egg-info/entry_points.txt (deflated 33%)adding: waitress-0.8.5-py2.7.egg-info/requires.txt (deflated 5%)
[zart@feena unpacked]$ cd ..

Note that those files should be at top of zip, you can't just zip -r9 library.zip unpacked

Checking the result:

[zart@feena ziplib-demo]$ PYTHONPATH=library.zip python
Python 2.7.1 (r271:86832, Apr 12 2011, 16:15:16)
[GCC 4.6.0 20110331 (Red Hat 4.6.0-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import waitress
>>> waitress
<module 'waitress' from '/home/zart/ziplib-demo/library.zip/waitress/__init__.pyc'>
>>>
>>> from wsgiref.simple_server import demo_app
>>> waitress.serve(demo_app)
serving on http://0.0.0.0:8080
^C>>>

Update: since python 3.5 there is also zipapp module which can help with bundling the whole package into .pyz file. For more complex needs pyinstaller, py2exe or py2app might better fit the bill.

https://en.xdnf.cn/q/26485.html

Related Q&A

Pass Variable On Import

Lets say you have some time-consuming work to do when a module/class is first imported. This functionality is dependent on a passed in variable. It only needs to be done when the module/class is load…

Write a file to a directory that doesnt exist [duplicate]

This question already has answers here:How do I create a directory, and any missing parent directories?(27 answers)Closed 9 months ago.This post was edited and submitted for review 8 months ago and fa…

How to convert a string to an image?

I started to learn python a week ago and want to write a small program that converts a email to a image (.png) so that it can be shared on forums without risking to get lots of spam mails. It seems lik…

Numpy list of 1D Arrays to 2D Array

I have a large list files that contain 2D numpy arrays pickled through numpy.save. I am trying to read the first column of each file and create a new 2D array.I currently read each column using numpy.…

What is metrics in Keras?

It is not yet clear for me what metrics are (as given in the code below). What exactly are they evaluating? Why do we need to define them in the model? Why we can have multiple metrics in one model?…

ObjectNotExecutableError when executing any SQL query using AsyncEngine

Im using async_engine. When I try to execute anything: async with self.async_engine.connect() as con:query = "SELECT id, name FROM item LIMIT 50;"result = await con.execute(f"{query}&quo…

Store the cache to a file functools.lru_cache in Python = 3.2

Im using @functools.lru_cache in Python 3.3. I would like to save the cache to a file, in order to restore it when the program will be restarted. How could I do?Edit 1 Possible solution: We need to pi…

Moon / Lunar Phase Algorithm

Does anyone know an algorithm to either calculate the moon phase or age on a given date or find the dates for new/full moons in a given year?Googling tells me the answer is in some Astronomy book, but…

Flask-RESTful API: multiple and complex endpoints

In my Flask-RESTful API, imagine I have two objects, users and cities. It is a 1-to-many relationship. Now when I create my API and add resources to it, all I can seem to do is map very easy and genera…

Setting initial Django form field value in the __init__ method

Django 1.6I have a working block of code in a Django form class as shown below. The data set from which Im building the form field list can include an initial value for any of the fields, and Im having…