32

crate2nix - a nix build file generator for rust crates

 5 years ago
source link: https://www.tuicool.com/articles/hit/7j63u2v
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

crate2nix

crate2nix generates nix build files for rust crates using cargo .

Same dependency tree as cargo: It uses cargo_metadata to obtain the dependency tree from cargo. Therefore, it will use the exact same library versions as cargo and respect any locked down version in Cargo.lock .

Smart caching: It uses smart crate by crate caching so that nix rebuilds exactly the crates that need to be rebuilt. Compare that to docker layers...

Nix ecosystem goodness: You can use all things that make the nix/NixOS ecosystem great, e.g. distributed/remote builds, build minimal docker images, deploy your binary as a service to the the cloud with NixOps , ...

Out of the box support for libraries with non-rust dependencies: It builds on top of the buildRustCrate function from NixOS so that native dependencies of many rust libraries are already correctly fetched when needed. If your library with native dependencies is not yet supported, you can create an overlay to add the needed configuration to the defaultCrateOverrides .

Easy to understand nix template: The actual nix code is generated via templates/build.nix.tera so you can fix/improve the nix code without knowing rust if all the data is already there.

Here is a simple example which uses all the defaults and will generate a default.nix file:

# From the project directory.
crate2nix generate

Here is a more elaborate example that uses <nixos-unstable> as the default nixpkgs path and specifies both the path to the Cargo.toml file ( -f ) and the output ( -o ) file explicitly.

crate2nix generate \
    -n '<nixos-unstable>' \
    -f /some/project/dir/Cargo.toml \
    -o /some/project/dir/crate2nix.nix

Use crate2nix help to show all commands and options.

Installation

NOTE: It is Linux-only for now!

For now, clone the repository and then

# Install nix if necessary: https://nixos.org/nix/
cd crate2nix
nix-shell
# you are in a shell with crate2nix

This assumes that the <nixos-unstable> path points to, well, nixos-unstable.

If that doesn't work for you, you can

  1. either add it to your nix channels:
nix-channel --add https://nixos.org/channels/nixos-unstable nixos-unstable
  1. or you override the pkgs argument, e.g.:
nix-shell --arg pkgs 'import <nixos> {config = {}; }'

Known Restrictions

  • Only default crate features are supported. It should be easy to support a different feature set at build generation time since we can simply pass this set to cargo metadata . Feature selection during build time is out of scope for now.
  • Filters all dependencies for the hard-coded "Linux x86_64" target platform . Again, it should be quite easy to support more platforms. To do so completely and at build time (vs build generation time) might be more involved.
  • Only local sources and crates io supported. Again, just requires some work to resolve.
  • Since cargo exposes local paths in package IDs, the generated build file also contain them as part of an "opaque" ID. They are not interpreted as paths but maybe you do not want to expose local paths in there...

Runtime Dependencies

crate2nix use cargo metadata / nix-prefetch-url at runtime so they need to be in the PATH. The default.nix adds the built-time nix/cargo binaries as fallback to the path.

Currently, crate2nix is only tested with nixos-unstable (the future 19.03) since it depends on some new features and bug fixes.

Project Overview / Terminology

If you want to hack on this, it is useful to know that build file generation is broken up into multiple phases:

  1. cargo metadata : Calling cargo metadata via the cargo_metadata crate.
  2. indexing metadata : Indexing the metadata by package ID to enable easy joining of "Node" and "Package" information, resulting in metadata::IndexedMetadata .
  3. resolving : Using the indexed metadata to actually resolve the dependencies and join all needed build information into resolve::CrateDerivation .
  4. pre-fetching : Pre-fetching crates.io packages to determine their sha256, see prefetch module.
  5. rendering : Rendering the data via the build.nix.tera template, see render module.

Related Projects

  • carnix is already widely used in NixOS itself, yet it failed to generate correct builds for my rust projects. After some attempts to fix that, I gave up. That said, big kudos for all the work on buildRustCrate and showing the way!
  • cargo-raze generates BUILD files for bazel.

Contributions

Contributions in the form of documentation and bug fixes are highly welcome. Please start a discussion with me before working on larger features.

I'd really appreciate tests for all new features. Please run cargo test before submitting a pull request.

Feature ideas are also welcome -- just know that this is a pure hobby side project and I will not allocate a lot of bandwidth to this. Therefore, important bug fixes are always prioritised.

By submitting a pull request, you agree to license your changes via all the current licenses of the project.


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK