A review of OpenStack's LOCI and its extensibility
LOCI is a very simple openstack project focusing on building OCI images for OpenStack Services using a Dockerfile and a series of shell scripts.
Currently, 9 projects (Cinder, Glance, Heat, Horizon, Ironic, Keystone, Neutron, Nova, and Octavia) can be built with it.
Flexibility and extensibility
LOCI is relying on passing
arguments to the Docker image building for its flexibility. It requires
you to pass at least the PROJECT
as CLI argument, to tell which
OpenStack project you want to build.
The other parameters are optional. You can pass a base image as starting
point (FROM
) in argument if you do not want to build from the ubuntu
base image. Other variables can be provided, like your personal pip
web mirror, or proxy settings.
The image building’s shell scripts receives the appropriate arguments as environment variables. Many shell scripts do not need any updating to leverage this flexibility. If your use case is not met, I doubt it would be a problem to change the scripts to adapt your use case.
On top of that, you can of course build a child image to your needs.
The build process
The build process currently behaves like this:
python and virtualenv are installed on the base image, using distribution packages.
A system user is created for the project.
virtualenv is upgraded, unconstrained, in a virtualenv. I suppose if you want to have fully reproducible builds with no surprise on the virtualenv version, you could implement a bypass using a docker build argument.
pip get installed in previous virtualenv, unconstrained again. Previous comment applies here.
The bindep pip package and extra pip packages based on the project + profile (coming from a pydep.txt file at the root of the LOCI repo) are installed.
If a wheel built image was created, its upper constraints file is used for the installation of those
pydeps
+ bindep pip packages. These packages are installed in a binary form (only), from your pip mirror of choice.If no pip mirror can be trusted, it would be technically possible to implement a bypass by having an argument that flips the
--only-binary
parameter to--no-binary
, and copying your own constraints file.The destination project is cloned with the user provided reference (default is master branch’s HEAD).
The destination project, and any extra
PIP_PACKAGES
(shouldPIP_PACKAGES
be not empty), are installed based on projects’setup()
content.Distribution packages are installed at this point using bindep.txt file.
Cleanup of installed distro packages (and other extra artifacts) happen, and allows the system to use global packages.
Security and reproducibility
While I did not have the chance to do a thorough security review, here is my opinion: The “requirements” building, while optional, is in my opinion necessary for security.
LOCI can build wheels for all the openstack requirements’ upper-constraints in your environment, which can then be published in a secure location for your next project building. It means you would rely on OpenStack security team’s work, which would eventually update the requirements/constraints of an upstream project by blacklisting an insecure python package. If you do not build the wheels with the LOCI requirements project, there is no reason to think you will pass a constraints file matching your OpenStack projects versions. Therefore, you are probably using anything from PyPI matching each project’s requirements, which can be very variable.
Build speed
The image building itself is quite fast, due to the simplicity of the process. If you have a CI pipeline, you can first build your ‘requirements’ image, then use its artifact for all the other projects building in parallel, which should take around a total of 10 to 20 minutes maximum.
Gating expectations
It is expected in the current gating process to expose the wheels built in the ‘requirements’ image on a web server. You cannot use locally built files, unless you carry your own patch changing the base Dockerfile.
Learning pace
Because the project itself is very simple, it is easy to track what is happening there. It is easy to contribute, and a few minutes/hours should be enough to get a grip of the whole codebase.
Conclusions
LOCI is a very simple and efficient way to build OpenStack images. It relies on Dockerfiles, simple shell scripts, and the power of bindep to install OpenStack software from sources.
If you are looking for a way to build images based on distribution
packages, there is little to no point of using
LOCI, as you could as well build
your image with a simple Dockerfile and a command like
zypper install ${PACKAGE_LIST}
. But that would also mean you will have
to package your software and its dependencies (maybe for two different
versions of Python), handle package conflicts, and other packaging work.
The packaging can be completely avoided if you build from source relying on the python ecosystem, OpenStack creation of upper constraints, and bindep management by upstream team(s).