Giter Club home page Giter Club logo

wheel-axle's Introduction

Wheel Axle - Python Wheel enhancement library

Gitter Build Status Coverage Status

Wheel Axle Version Wheel Axle Python Versions

Wheel Axle Downloads Per Day Wheel Axle Downloads Per Week Wheel Axle Downloads Per Month

Problem

  1. Python wheel does not support symlinks.
  2. Python wheel does not support overwriting in a convenient way:
    • whether the distribution is pure-Python
    • distribution ABI tag
    • extend Python tag override capability

Solution

WARNING: THIS IS EXPERIMENTAL BETA SOFTWARE. THERE ARE NO WARRANTIES OF ANY KIND. USE AT YOUR OWN RISK. ADDITIONAL INCLUDED DISCLAIMERS ALSO APPLY.

Wheel-Axle (axle, bdist_axle) is a drop-in replacement/augmentation utility for wheel (bdist_wheel) that extends and builds spec-compliant wheels.

During the build, axle is able to capture and record in the Python wheel the symlinks in the following schema paths (locations):

  • purelib
  • platlib
  • scripts
  • headers
  • data

Additionally, Axle is able to customize the Python wheel tags via additional command line options.

While the generated Python wheel is fully spec-compliant, additional symlink functionality is not possible without its companion library Wheel Axle Runtime. Thus, every Python wheel generated by the bdist_axle automatically becomes dependent on wheel-axle-runtime that provides post-install logic required.

Implementation

The body of the library is as ugly and messy as distutils/setuptools are, and consists of, mainly, in hacking/overwriting various setuptools commands to detect, handle and record symlinks and their targets. Once that problem is solved, the list of symlinks is recorded in the .dist-info/symlinks.txt in the following CSV format:

  1. symlink name
  2. symlink target
  3. a boolean (0 or 1) flag indicating whether the target is a directory

NOTE: Symlinks may be relative, absolute and/or broken. Symlink targets are recorded verbatim (even when broken) and are NOT otherwise interpreted. THIS IS INTENTIONAL. Please see Wheel Axle Runtime Security Notice for additional information.

A special <distribution name and version>.pth file is also added to the distribution. When the wheel is installed this .pth file triggers the post-install logic via wheel-axle-runtime.

Usage

python setup.py bdist_wheel <arguments> can be replaced with python setup.py bdist_axle <arguments>. The replacement is drop-in.

Additional functionality is available via the following options:

  --python-tag      Python implementation compatibility tag (default: 'py3')
  --root-is-pure    set to manually override whether the wheel is
                    pure (default: None)
  --abi-tag         set to override ABI tag (default: None)

Using --python-tag, --root-is-pure and --abi-tag allows you to create wheels that carry platform-dependent data while otherwise containing pure-Python libraries.

wheel-axle's People

Contributors

arcivanov avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

wheel-axle's Issues

`[Errno 17] File exists` error for nested symlinks of package data

Say this is in a setup.py to copy C++ shared libraries to a python wheel:

    include_package_data=True,
    package_dir={
        "mypackage/lib": "cmake_install/cpp_libs",
    },
    package_data={
        "mypackage/lib": ["*.so*", "**/*.so*"],
    },

cmake_install/cpp_libs could have libraries of this form, captured by the second regex:

  • prefix/foo.so, which links to...
  • prefix/foo.so.0, which links to...
  • prefix/foo.so.0.1

Because of the prefix directory in the path, the data_files in wheel_axle.bdist_axle.BuildPy.build_package_data (here) have duplicate copy commands:

  • in package install/cpp_libs: prefix/libfoo.so* files
  • in package install/cpp_libs.prefix : libfoo.so* files (without the prefix directory).

When calling BuildPy.copy_file, non-symlinks (like prefix/libfoo.so.0.1) are handled fine since super().copy_file handles it gracefully (no error, just returns false in the return value for "not copied" if it already exists).

However, symlinks (like prefix/foo.so.0 and prefix/foo.so) are handled by copy_link which seems to assume always doing the copy. Due to the duplicate copies from the separate packages listed above, the second copy fails with the following error:

error: [Errno 17] File exists: 'libfoo.so.0' -> './mypackage/lib/prefix/libfoo.so'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.