How to install pyzmq on an Alpine Linux container?

2024/9/28 17:26:50

I have a container with the python:3.6-alpine kernel. I have a problem installing the pyzmq via pip on this:

Dockerfile:

FROM python:3.6-alpineRUN mkdir /code
RUN apk add vim
WORKDIR /
ADD . /code

docker-compose.yml:

version: '3'services:battery_monitoring:build: .image: bm:1.0.0container_name: battery_monitoringrestart: unless-stoppedvolumes:- .:/codetty: true

When I install several Python libraries on this container I don't have any problem, but with pyzmq library, there is an error:

Procedure:

$ docker-compose build
$ docker-compose up -d
$ docker exec -it <This-container-ID> sh

In the container:

pip install pyserial
pip install easydict

The above packages are installed properly, but pyzmq installation has the following error:

pip install pyzmq

A part of the result:

    ----------------------------------------
Command "/usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-545my4q5/pyzmq/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-nbtsgz0b/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-545my4q5/pyzmq/

[NOTE]:

pip -V
pip 18.0 from /usr/local/lib/python3.6/site-packages/pip (python 3.6)which pip
/usr/local/bin/pip
  • There isn't any problem using python:3.6-slim instead of python:3.6-alpine
Answer

Using py3-zmq (renamed py3-pyzmq before August 2022) package

From my experience, python:3.6-alpine is not well suited for installing packages with C extensions because of missing Python headers. The alpine images already offer a Python 3.6 distribution and also a precompiled pyzmq package, so it's already sufficient to do:

FROM alpine:edgeRUN apk update && apk add py3-pyzmqCMD ["/bin/sh"]

Check:

$ docker run --rm -it my/alpine /bin/sh
/ # python3 -c "import zmq; print(zmq.__version__)"
17.1.0

This is the easiest and most reliable way to install pyzmq in an Alpine container.

Building from source with pip install

Alpine is not manylinux1-compatible, so any package containing C extensions must be built from source. This means you have to install the build tools first. Again, I'd use alpine image instead of python:3.6-alpine:

FROM alpine:edgeRUN apk update && apk add build-base libzmq musl-dev python3 python3-dev zeromq-devRUN pip3 install pyzmq# reduce image size by cleaning up the build packages
RUN apk del build-base musl-dev python3-dev zeromq-devCMD ["/bin/sh"]

Check:

$ docker run --rm -it my/alpine /bin/sh
/ # python3 -c "import zmq; print(zmq.__version__)"
17.1.0

If you insist on python:3.6-alpine

Beware that python:3.6-alpine does not install Python via apk, it has Python built from source and located under /usr/local. So when you inherit from python:3.6-alpine, install python3-dev and run pip install pyzmq, you'll end up with building pyzmq for Python 3.6.6 (coming from python:3.6-alpine) using header files from Python 3.6.4 (coming from apk add python3-dev). In general, this shouldn't be an issue (header files are incompatible only between major Python releases), but may become an issue in case the header files were adapted by the distro maintainer.

Edit: exact steps to reproduce, with log

$ docker image rm my/alpine:latest
Untagged: my/alpine:latest
Deleted: sha256:2e613cdc3c90c9d44b23d399bd44069217e5b31c1b4a8fc91e501c5226a4ef6a
Deleted: sha256:d66ac6c96a4fca9c4fe71a73b64a4dd3605a59e570f327974954649b633a7fc5
Deleted: sha256:114efba5527eb4ab23020ef84b6181b6a6ba790059b83ce046c9a1a6c0bdf419$ docker image prune
WARNING! This will remove all dangling images.
Are you sure you want to continue? [y/N] y
Deleted Images:
untagged: alpine@sha256:79c2c5f6db53da44f90bb2731f29f725b5b14c378407a123776b6d3c76e6aebe
untagged: alpine@sha256:ae8a1f9146d74466ddf1def02088ba33544db9aceef01f4b388c674a5ad1d00b
deleted: sha256:5c4fa780951b060bb0a75355765bc58112350d9974970d60561671d552aaf2e2
deleted: sha256:c9e8b5c053a2dda62373bc57fa8cb634230a92ba5f02d2baf5d35b932d04a878Total reclaimed space: 4.148MB$ cat ./Dockerfile
FROM alpine:edgeRUN apk update && apk add py3-pyzmqCMD ["/bin/sh"]$ docker pull alpine:edge
edge: Pulling from library/alpine
a0710691c81a: Pull complete
Digest: sha256:8d9872bf7dc946db1b3cd2bf70752f59085ec3c5035ca1d820d30f1d1267d65d
Status: Downloaded newer image for alpine:edge$ docker build -t my/alpine .
Sending build context to Docker daemon  3.072kB
Step 1/3 : FROM alpine:edge---> 9d1f27787d39
Step 2/3 : RUN apk update && apk add py3-pyzmq---> Running in 0f9bd971b5da
fetch http://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
v3.8.0-1447-g6c9915aaa5 [http://dl-cdn.alpinelinux.org/alpine/edge/main]
v3.8.0-1459-g2ff55fde23 [http://dl-cdn.alpinelinux.org/alpine/edge/community]
OK: 9626 distinct packages available
(1/16) Installing libbz2 (1.0.6-r6)
(2/16) Installing expat (2.2.5-r0)
(3/16) Installing libffi (3.2.1-r4)
(4/16) Installing gdbm (1.13-r1)
(5/16) Installing xz-libs (5.2.4-r0)
(6/16) Installing ncurses-terminfo-base (6.1-r0)
(7/16) Installing ncurses-terminfo (6.1-r0)
(8/16) Installing ncurses-libs (6.1-r0)
(9/16) Installing readline (7.0.003-r0)
(10/16) Installing sqlite-libs (3.24.0-r1)
(11/16) Installing python3 (3.6.4-r1)
(12/16) Installing libgcc (6.4.0-r8)
(13/16) Installing libsodium (1.0.16-r0)
(14/16) Installing libstdc++ (6.4.0-r8)
(15/16) Installing libzmq (4.2.3-r0)
(16/16) Installing py3-zmq (17.1.0-r0)
Executing busybox-1.28.4-r0.trigger
OK: 69 MiB in 29 packages
Removing intermediate container 0f9bd971b5da---> 83a4db72581d
Step 3/3 : CMD ["/bin/sh"]---> Running in b37e3ef8e639
Removing intermediate container b37e3ef8e639---> 558bd6427c77
Successfully built 558bd6427c77
Successfully tagged my/alpine:latest$ docker run --rm -it my/alpine python3 -c "import zmq; print(zmq.__version__)"
17.1.0
https://en.xdnf.cn/q/71315.html

Related Q&A

How to Get Value Out from the Tkinter Slider (Scale)?

So, here is the code I have, and as I run it, the value of the slider bar appears above the slider, I wonder is there a way to get that value out? Maybe let a=that value. ;)from Tkinter import *contr…

Python cassandra-driver OperationTimeOut on every query in Celery task

I have a problem with every insert query (little query) which is executed in celery tasks asynchronously. In sync mode when i do insert all done great, but when it executed in apply_async() i get this:…

Function which returns the least-squares solution to a linear matrix equation

I have been trying to rewrite the code from Python to Swift but Im stuck on the function which should return the least-squares solution to a linear matrix equation. Does anyone know a library written i…

Divide .csv file into chunks with Python

I have a large .csv file that is well over 300 gb. I would like to chunk it into smaller files of 100,000,000 rows each (each row has approximately 55-60 bytes).I wrote the following code:import panda…

Why cant I use operator.itemgetter in a multiprocessing.Pool?

The following program:import multiprocessing,operator f = operator.itemgetter(0) # def f(*a): return operator.itemgetter(0)(*a) if __name__ == __main__:multiprocessing.Pool(1).map(f, ["ab"])f…

Writing a compiler for a DSL in python

I am writing a game in python and have decided to create a DSL for the map data files. I know I could write my own parser with regex, but I am wondering if there are existing python tools which can do …

How to setup Celery to talk ssl to Azure Redis Instance

Using the great answer to "How to configure celery-redis in django project on microsoft azure?", I can configure Celery to use Azure Redis Cache using the non-ssl port, 6379, using the follo…

Cant save data from yfinance into a CSV file

I found library that allows me to get data from yahoo finance very efficiently. Its a wonderful library.The problem is, I cant save the data into a csv file.Ive tried converting the data to a Panda Da…

silhouette coefficient in python with sklearn

Im having trouble computing the silhouette coefficient in python with sklearn. Here is my code :from sklearn import datasets from sklearn.metrics import * iris = datasets.load_iris() X = pd.DataFrame(i…

Force dask to_parquet to write single file

When using dask.to_parquet(df, filename) a subfolder filename is created and several files are written to that folder, whereas pandas.to_parquet(df, filename) writes exactly one file. Can I use dasks t…