Suppose this wheel:
M Filemode Length Date Time File
- ---------- -------- ----------- -------- --------------------------------------------rw-rw-r-- 1358 26-Sep-2018 21:08:40 azure/common/__init__.py-rw-rw-r-- 327 26-Sep-2018 21:08:40 azure/common/_version.py-rw-rw-r-- 8737 26-Sep-2018 21:08:40 azure/common/client_factory.py-rw-rw-r-- 755 26-Sep-2018 21:08:40 azure/common/cloud.py-rw-rw-r-- 2479 26-Sep-2018 21:08:40 azure/common/credentials.py-rw-rw-r-- 805 26-Sep-2018 21:08:40 azure/common/exceptions.py-rw-rw-r-- 6079 26-Sep-2018 21:08:40 azure/profiles/__init__.py-rw-rw-r-- 3943 26-Sep-2018 21:08:40 azure/profiles/multiapiclient.py-rw-rw-r-- 6 26-Sep-2018 21:21:54 azure_common-1.1.16.dist-info/top_level.txt-rw-rw-r-- 110 26-Sep-2018 21:21:54 azure_common-1.1.16.dist-info/WHEEL-rw-rw-r-- 3805 26-Sep-2018 21:21:54 azure_common-1.1.16.dist-info/METADATA-rw-rw-r-- 997 26-Sep-2018 21:21:54 azure_common-1.1.16.dist-info/RECORD
- ---------- -------- ----------- -------- -------------------------------------------29401 12 files
It has three different packages in it:
- azure.common
- azure.profiles
- azure_common
All great names, and great layout. Also, a lot of greatness of mind that unmistakably went into engineering this miracle of modern software engineering.
This wheel is distributed by the name azure-common
. So, when you depend on in in setup.py
like this:
setup(...install_requires=['azure-common'],...
)
You will only get azure_common
package installed. Maybe. I don't really know, it seems so, but few times that I tried it seemed to only install azure.common
, or maybe I eyeballed it... It's really hard to follow all the manipulations setuptools does on a package.
Hence the question: how can I force setuptools
into installing all packages found in this kind of wheel? Also, the order is important because this garbage needs to be installed some of the times with other packages which also provide azure.something
packages which may overwrite the stuff in azure
directory. So, Ideally, I'd also like to control in which order install_requires
dependencies are processed.
This is where this started: How to specify bracket dependencies in setup.py?
It sounds like only few sub-directories like azure.common
installed into your environment when you installed dependencies via setup.py
with install_requires=['azure-common']
. I tried to reproduce this issue, but failed that all files in this package has been installed.
Here is my steps on my local Windows machine as below, which you can refer to.
- Create a directory
mkdir setuptmp
, and create a virtual environment virtualenv setuptmp
, then to cd setuptmp
.
Create a setup.py
file with the content as below.\
from setuptools import setup, find_packages setup(name = "setuptmp",install_requires = ['azure-common']
)
Activate the virtual environment via Scripts\activate.bat
.
- Run
python setup.py install
to install the dependency described in my setup.py
.
Run python
to open the REPL interpreter to test all packages as you said,
(setuptmp) D:\projects\setuptmp>python
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:57:15) [MSC v.1915 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import azure.common
>>> import azure.profiles
>>> azure.common.__file__
'D:\\projects\\setuptmp\\lib\\site-packages\\azure_common-1.1.16-py3.7.egg\\azure\\common\\__init__.py'
>>> azure.profiles.__file__
'D:\\projects\\setuptmp\\lib\\site-packages\\azure_common-1.1.16-py3.7.egg\\azure\\profiles\\__init__.py'
>>> import azure_common
Traceback (most recent call last):File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'azure_common'
Note: azure_common
is not a module, just an egg info directory.
Check the packages installed in my environment via cd Lib\site-packages
, dir
and tree azure_common-1.1.16-py3.7.egg /F
as below.
(setuptmp) D:\projects\setuptmp\Lib\site-packages>dirVolume in drive D is DataVolume Serial Number is BA4B-64AADirectory of D:\projects\setuptmp\Lib\site-packages2018/12/26 14:48 <DIR> .
2018/12/26 14:48 <DIR> ..
2018/12/26 14:48 <DIR> azure_common-1.1.16-py3.7.egg
2018/12/26 14:48 61 easy-install.pth
2018/12/26 14:46 126 easy_install.py
2018/12/26 14:46 <DIR> pip
2018/12/26 14:46 <DIR> pip-18.1.dist-info
2018/12/26 14:46 <DIR> pkg_resources
2018/12/26 14:48 965 setuptmp-0.0.0-py3.7.egg
2018/12/26 14:46 <DIR> setuptools
2018/12/26 14:46 <DIR> setuptools-40.6.3.dist-info
2018/12/26 14:46 <DIR> wheel
2018/12/26 14:46 <DIR> wheel-0.32.3.dist-info
2018/12/26 14:46 <DIR> __pycache__3 File(s) 1,152 bytes11 Dir(s) 80,896,319,488 bytes free(setuptmp) D:\projects\setuptmp\Lib\site-packages>tree azure_common-1.1.16-py3.7.egg /F
Folder PATH listing for volume Data
Volume serial number is BA4B-64AA
D:\PROJECTS\SETUPTMP\LIB\SITE-PACKAGES\AZURE_COMMON-1.1.16-PY3.7.EGG
├─azure
│ ├─common
│ │ │ client_factory.py
│ │ │ cloud.py
│ │ │ credentials.py
│ │ │ exceptions.py
│ │ │ _version.py
│ │ │ __init__.py
│ │ │
│ │ └─__pycache__
│ │ _version.cpython-37.pyc
│ │ __init__.cpython-37.pyc
│ │
│ └─profiles
│ multiapiclient.py
│ __init__.py
│
└─EGG-INFOPKG-INFORECORDrequires.txttop_level.txtWHEEL
Compare the above with the file structure of azure-common
package downloaded from the link of Pypi website. I decompressed azure_common-1.1.16-py2.py3-none-any.whl
file using 7-Zip
into a temp directory and tree
it.
D:\tmp>tree azure_common-1.1.16-py2.py3-none-any /F
Folder PATH listing for volume Data
Volume serial number is BA4B-64AA
D:\tmp\AZURE_COMMON-1.1.16-PY2.PY3-NONE-ANY
├─azure
│ ├─common
│ │ client_factory.py
│ │ cloud.py
│ │ credentials.py
│ │ exceptions.py
│ │ _version.py
│ │ __init__.py
│ │
│ └─profiles
│ multiapiclient.py
│ __init__.py
│
└─azure_common-1.1.16.dist-infoMETADATARECORDtop_level.txtWHEEL
Then, you will find the file structure of step 6
& 7
is almost same.
Hope it helps. If you have any concern, please feel free to let me know.
I did the same above on Linux and got the same result. I saved the output of tree lib/ > lib_[before|after].txt
of my Linux setuptmp
before and after run python setup.py install
, then to compare them using diff lib_*.txt
as below.
(setuptmp) peter@peterpc:~/setuptmp$ diff lib*.txt
92a93,111
> │ ├── azure_common-1.1.16-py3.6.egg
> │ │ ├── EGG-INFO
> │ │ │ ├── PKG-INFO
> │ │ │ ├── RECORD
> │ │ │ ├── WHEEL
> │ │ │ ├── requires.txt
> │ │ │ └── top_level.txt
> │ │ └── azure
> │ │ ├── common
> │ │ │ ├── __init__.py
> │ │ │ ├── _version.py
> │ │ │ ├── client_factory.py
> │ │ │ ├── cloud.py
> │ │ │ ├── credentials.py
> │ │ │ └── exceptions.py
> │ │ └── profiles
> │ │ ├── __init__.py
> │ │ └── multiapiclient.py
> │ ├── easy-install.pth
827a847
> │ ├── setuptmp-0.0.0-py3.6.egg
1043c1063
< 118 directories, 922 files
---
> 123 directories, 937 files