Comments (11)
Maybe I should have omitted that first sentence because I suppose it wasn't really relevant to the point I was trying to make. I don't want end users (essentially, anyone who is not a crate developer) to have to be concerned with rav1e's internal package structure, and to continue to have just one binary. If there is any splitting within rav1e, it should leave the binary intact as the primary interface for users.
from rav1e.
I think it would make build and rebuild faster so it would be worth looking into it even if just for having the local crates, if we manage to spin out more reusable components is even better.
That said could you please check what is the best practice to publish all the crates? IIRC cargo publish --workspace
hadn't landed yet so there isn't an immediate way to publish all :/
Splitting does not simplify anything, it spreads the complexity across more crates, so we pay some additional burden for some nicer experience rebuilding.
NOTE: In order to not break everybody expectation we should keep cargo install rav1e
working.
from rav1e.
Well one of the reasons it makes builds for some projects faster is that by default LTO across crates is disabled. Because we enable it explicitly for our release builds, the benefit might be smaller than expected.
from rav1e.
Splitting does not simplify anything, it spreads the complexity across more crates, so we pay some additional burden for some nicer experience rebuilding.
To add another perspective here - from a Linux distribution point of view, it is currently quite nice that rav1e is pretty much self-contained. I'm the maintainer of the package in Fedora Linux, and it's always pretty easy to update rav1e to new versions for this reason. When splitting things into multiple sub-components, that makes things harder ... gitoxide is a pretty bad example, it requires updating dozens and dozens of packages for all its subcomponents regularly, which is a huge PITA.
from rav1e.
I would really not like linux packagers to pull in individual components of rav1e, although I'm aware that some distributions prefer going against Rust's static-by-default build methods, but I also very strongly favor rav1e being one binary for end users. The reason I see this being potentially beneficial is primarily around crate development, as mentioned there are crates such as av-scenechange which currently have to pull in the entirety of rav1e. I will say as well, however, from working on av-scenechange--Ideally that crate would host all of the scenechange code, and rav1e would pull in that crate, instead of the other way around, but there is a lot of code in rav1e that is used in multiple places within the codebase, so it's actually not a simple matter to split rav1e into isolated components that have a one-way dependency tree.
from rav1e.
I would really not like linux packagers to pull in individual components of rav1e, although I'm aware that some distributions prefer going against Rust's static-by-default build methods, but I also very strongly favor rav1e being one binary for end users.
I'm not sure what you mean here? Dynamic linking for Rust crates is basically impossible in most contexts. We distribute binaries that are built like usual (with the equivalent of cargo build --release
) ... so, statically linked and just one binary. The only difference is that some distros (primarily debian and Fedora) package crates separately and do not use vendored dependencies. But that does not affect static / dynamic linking, those crate packages contain only source code, no compiled artifacts.
from rav1e.
I would try to use a local subcrate for the asm code, it isn't really impacted by lto and isn't more work due the lack of cargo publish --workspace
). Ideally it would simplify the build.rs for the hash checking and maybe speed up a little rebuilding overall.
from rav1e.
Can you clarify what you mean by "local subcrate"? Support for actual "nested" crates in cargo is not implemented yet: rust-lang/rfcs#3452
from rav1e.
My bad, somehow I assumed it got in.
from rav1e.
Well one of the reasons it makes builds for some projects faster is that by default LTO across crates is disabled. Because we enable it explicitly for our release builds, the benefit might be smaller than expected.
That's fair. I think there could still be some speed improvements from improved build parallelization before LTO, but that might require more work specifically for better build speeds.
That said could you please check what is the best practice to publish all the crates? IIRC cargo publish --workspace hadn't landed yet so there isn't an immediate way to publish all :/
I wasn't aware of this perspective. It seems like there is no official support for this yet, but there are some external tools like cargo-release
(which supports --workspace
) or cargo-workspace
. This is just what I found when searching though, I have no experience with these.
Splitting does not simplify anything, it spreads the complexity across more crates, so we pay some additional burden for some nicer experience rebuilding.
Subcrates shouldn't be created just for the sake of more subcrates. There's no obligation to create more subcrates if there are disadvantages to decoupling things. I think there are a few things that are easy to move into a subcrate though.
If there is any splitting within rav1e, it should leave the binary intact as the primary interface for users.
I agree. Maybe I should've been clearer in my initial post. I didn't mean to suggest that rav1e (as in: the encoder rav1e) should be packaged as multiple components. I would assume that most of these subcrates would be version = 0.0.0
and publish = false
and just exist for organization of code like crates for asm builds. A release build would still create a statically-linked rav1e executable that can be used and distributed like usual.
from rav1e.
On revisiting this, I suggested this under a false assumption. I thought there could be subcrates in this repository which rav1e depends on, which would not need to be published to crates.io separately (I think this would only be possible with the "nested subcrate" feature talked about above).
That changes my stance a bit - I'm not sure having to publish rav1e in X parts without an explicit reason is a good idea given the added workload around publishing these crates. I guess it would make more sense to only split functionality into a separate crate as needed (e.g. av-scenechange, if possible).
from rav1e.
Related Issues (20)
- key_frame_interval capi option inconsistently named with cli
- How is SizeClass1D::X4 initializing row_coeffs? HOT 1
- master branch has older version number than crates.io HOT 1
- crates.io release? HOT 3
- NASM warnings about empty macro parameters
- Can't build v0.7.0 on Alpine Linux HOT 10
- Request to have option to set Pixel aspect ratio
- Rav1e is not honoring input level set for lower resolutions lower bit rates HOT 1
- Feature desync_finder does not compile
- Add WASM tests to CI HOT 2
- Codecov upload from CI is broken
- What WebAssembly targets should be supported? HOT 2
- Build error: thread 'main' panicked, No such file or directory HOT 12
- AlignedBoxedSlice is unsound HOT 1
- blocks placed at incorrect position or time HOT 5
- cargo installation - Permission denied HOT 1
- Error when cross-compiling for Android HOT 8
- Add support for variable framerate encodings HOT 1
- Not possible to change bitrate and other encoder configuration without creating a new encoder context
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rav1e.