As ARM style chips (such as Apple's M1 or AWS's Gravitron) get more popular there's one big problem- not a lot of Python libraries are packaging ARM versions of the code, forcing developers to either sit and wait while their systems compile libraries or use buggy emulation layers. To make this adjustment a bit easier I'm launching a new project, Multi-Py, to publish multiple architecture images so developers can skip all of that compiling.
Multi-Py targets AMD64, ARM64, and ARM v7 and publishes both Ubuntu and Alpine containers (based off of the official Python images).
Initial Packages
This project starts with four initial images, but others are already in development.
Gunicorn
Gunicorn is a WSGI HTTP Server that is extremely popular for hosting Python web applications. It basically acts as a process manager, bringing up instances of the Python application in individual processes to serve requests..
For ARM and Alpine this requires building gevent, inotify, and setproctitle.
Uvicorn
Uvicorn is similar to gunicorn but is designed to take advantage of Python's async features. Rather than spawn multiple processes to serve requests it uses one process and the async functionality to handle multiple requests.
For Arm and Alpine this requires building uvloop.
Gunicorn running Uvicorn
This container runs a common pattern where Gunicorn is used to manage Uvicorn. This allows the container to run multiple processes, managed by Gunicorn, while those processes can handle more requests. This can be used as an alternative to running multiple Uvicorn containers.
This image is made from the other two containers, so it contains all of their compiled libraries.
OSO
OSO is an authorization framework written in Rust with a variety of language bindings, including Python.
This image has to build OSO and it's dependencies.
Other Benefits
Automated Updates
This project uses a custom Github Action, Versionator, to watch for new releases of the upstream packages. Within 30 minutes of a new package is released Versionator will update the workflow for building images to drop the oldest version and add the new one in. This update automatically triggers a new build and even updates the README files for the projects.
As an open source maintainer this is extremely important to me- it doesn't matter if I'm asleep or on vacation, if a security update for a package comes out then the images will be built without requiring any intervention on my behalf.
Small Images
All images are built using multiple stages, so that the libraries are built in one image and then moved into the final published one. This means that the build tools are never even installed in the final image, which seriously shrinks the total image size.
Built with all support versions of Python
Images are built for all of the supported versions of Python (currently 3.6, 3.7, 3.8, 3.9, and 3.10). This support extends back to older versions of each package- in fact, the previous five released versions for the package it supports.
This gives us five package versions, five Python versions, and three variants (Alpine, Ubuntu, and Ubuntu Slim) for 75 maintained tags for each image. Each tag also has three architectures, resulting in 225 individual builds each time images are created and pushed.
No Rate Limits
All of these images are hosted on the Github Container Registry, which does not have any rate limits on public projects and does not require anyone to log in to pull the images.
What's Next
These initial containers were driven by a pretty immediate need on my part, as I've been moving towards more ARM based systems and have a lot of Python web projects I host. These images make hosting Flask and FastAPI apps easy, which covers a lot of my projects.
This isn't the end though- I plan on some additional data science focused containers next, and am completely open to requests!
If you have any questions about this or any of my other projects please feel free to reach out on twitter!.