Skip to content

uranusjr/zlig

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CPython Extension Module Support for Flit

This is a PEP 517 build backend piggybacking (and hacking) Flit to support building C extensions.

Mostly a proof-of-concept, but could be further developed into something more generally useful if Flit can better support hooking into the build system.

Features

  • Can build C extensions.
  • Does not require a user to pre-install a compiler.
  • Very good caching; incremental compilation performs much better than setuptools from my totally non-scientific observation.
  • Produces a wheel with appropriate-ish tag for distribution.

Limitations

  • Since Flit only supports one single top-level Python module/package, and enforces the existence of that file/directory, this can only build extensions as a submodule of a package right now.
  • Since Flit's automatic metadata introspection (read version and description from module) needs to import the top-level module/package, you either need to jump through some hooks to make those work without the extension being available, or only write metadata in pyproject.toml.
  • Slower "cold" compilation time compared to setuptools and platform-provided compilers.
  • Does not allow custom compiler flags (possible to implement).
  • Only handles C for now. I believe it's possible to support C++ (and Zig).
  • Probably more, setuptools has so many years behind it and can therefore cover many edge cases I've never dreamt of.

Characteristics

  • Compiles extension modules directly into the top-level Python package. This makes it possible to run the extension "in-place" without installing, which I find useful. But it makes the source tree a bit messy (you probably need to run git clean once in a while to keep things sane).

How-To

There's a minimal example in examples/demo that has all the needed parts.

[build-system]
requires = ["zlig"]
build-backend = "zlig"

# ... Project metadata declaration.

[tool.flit.sdist]
# Exclude extension modules from sdist.
exclude = ["src/demo/*.so"]

[tool.zlig]
# Declare extensions and its sources.
extensions = [{name = "demo.demo", sources = ["src/**/*.c"]}]

Add the following entries to your .gitignore:

/build.zig
/zig-cache/
/zig-out/
# Flit builds things to /dist so add it too.
# Also *.pyd and *.so files but you should've already ignored them.

Details

As a PEP 517 backend, this module simply bridges most of Flit's build API, but re-implements build_wheel to do some additional things:

  • Compile extension modules before handing the package to Flit, which would add all the files (including compiled extensions) into the wheel.
  • When adding a file to the wheel, first check whether the file is an extension's source and exclude it.
  • Override Flit's logic deciding a wheel's file name to use a platform-specific wheel tag instead.

Compilation magic is provided by Zig's build system. A working Zig compiler is installed as a PEP 517 build dependency. During compilation, the backend generates a build script (build.zig) from pyproject.toml, and call the Zig compiler to do the rest. After compilation, those binaries are copied to the location Flit expects to fine modules.

About

CPython Extension Module Support for Flit

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published