Build process notes

This section contains notes on complexities in the GitHub Actions build-deploy workflow.

For general build instructions, see Building from source.

Crypto provider

qpdf can get its MD5, RC4, SHA-2 and AES from its own native implementation, from OpenSSL, or from GnuTLS. Which one a wheel ends up with depends on how that platform obtains libqpdf.

On Linux and macOS we compile libqpdf ourselves, and select the provider explicitly:

cmake -DREQUIRE_CRYPTO_NATIVE=1 -DUSE_IMPLICIT_CRYPTO=OFF ...

See build-scripts/posix-build-wheel-deps.bash. This is the only provider compiled in, so those wheels link no TLS library at all.

On Windows we do not build qpdf. build-scripts/win-download-qpdf.ps1 downloads qpdf’s official msvc64 (x64) or msvc-arm64 (ARM64) release, which statically links OpenSSL into qpdf30.dll. Both openssl and native are compiled into that binary and OpenSSL is the default, per qpdf’s gnutls > openssl > native precedence. We have no build-time control over this short of compiling qpdf from source on Windows.

Two reasons to keep the POSIX selection explicit rather than letting USE_IMPLICIT_CRYPTO choose:

  • It is deterministic. The implicit setting picks whatever headers happen to be present in the build image, so a base image that gained gnutls-devel would silently change what we ship.

  • It keeps third-party-licenses/ honest. An implicitly acquired dependency would be vendored into the wheel without appearing in the license manifest.

The user-facing consequences of this choice — what native crypto costs in assurance, and how to get a different provider — are documented in PDF security. Keep the two in sync when changing the build.

Linux qpdf build cache

On Linux, cibuildwheel’s before-all step compiles libqpdf inside each manylinux or musllinux container. The manylinux x86_64 wheels are split into one job per target (see the comment in build.yml), so without a cache several jobs would compile the same qpdf for the same image.

Instead, build.yml restores a per-image archive of the installed qpdf with actions/cache before running cibuildwheel. QPDF_CACHE_DIR tells posix-build-wheel-deps.bash where to find it inside the container, through cibuildwheel’s /host mount of the runner’s filesystem. On a hit the script extracts the archive into /usr/local and skips the download and compile. On a miss it downloads and compiles qpdf as usual, then archives what cmake --install installed, and the job saves that to the cache.

The cache key hashes pyproject.toml (the qpdf version and image choice), build.yml (the cibuildwheel version, which pins the image tag) and the POSIX build scripts, so any change to them rebuilds qpdf once. The yum/apk package installs still run on a hit, since libqpdf links libjpeg and zlib at run time.

macOS generally

Here are the current constraints for building on macOS:

  • General rule for macOS: build on the oldest available macOS runner, and set MACOSX_DEPLOYMENT_TARGET="that version.0", e.g. macos-15 and MACOSX_DEPLOYMENT_TARGET="15.0"

  • QPDF needs at least MACOSX_DEPLOYMENT_TARGET="11.0" since it uses C++20.

  • Homebrew creates binaries with MACOSX_DEPLOYMENT_TARGET=”macos-x”, and delocate vendors Homebrew’s libjpeg into the wheel. Therefore, we must build on the minimum runner, or delocate rejects the wheel.

  • We build arm64 wheels only, on macos-15. GitHub is deprecating the macos-14 runner, so macOS 15 is now the minimum for binary wheels.

  • We no longer build or test Intel (x86_64) wheels, since the platform is end of life. We dropped the macos-15-intel runner. Intel Mac users can build from source.

Taking a quick peek at numpy, it may be easier to build universal2 wheels and use QEMU to confirm that they work. That would be another big build overhaul.

Users who build from source have more options and can likely get functional builds on older macOS versions.

Historical decisions

Background on choices that are no longer current, kept because the reasoning explains how we arrived at the present configuration.

macOS used GnuTLS from v8.11.1 to v10.12.0

macOS wheels once linked GnuTLS. This came from #520: legacy encrypted PDFs stopped opening on macOS, because Homebrew’s OpenSSL had moved RC4 and MD5 into its legacy provider, which is not loaded by default. qpdf needs those algorithms for /R 2, 3 and 4 documents. Switching macOS to GnuTLS restored them, and the change was made on the understanding that it aligned macOS with the Linux builds.

It did not. The manylinux and musllinux images ship neither gnutls-devel nor openssl-devel, so USE_IMPLICIT_CRYPTO had been quietly falling back to native on Linux the whole time. The platforms were never aligned, and Linux wheels had been opening legacy encrypted files on native crypto without complaint.

Native crypto turned out to be a better answer to #520 than either external provider, because it implements the weak algorithms itself and no upstream deprecation policy can withdraw them. Moving macOS to native also dropped nine vendored dylibs (gnutls, nettle, hogweed, gmp, idn2, unistring, tasn1, p11-kit, intl), their LGPL obligations, and about 10 MB per wheel.

macOS used Homebrew’s qpdf before v8.11.1

Before #520 we linked against Homebrew’s prebuilt qpdf rather than building libqpdf ourselves. That is what exposed us to Homebrew’s OpenSSL configuration in the first place. Building libqpdf ourselves on every POSIX platform is what makes the crypto provider ours to choose.