Comments (4)
OK, I've tried xargs -P
and it seems to work much more gracefully than other common parallelization tools. I'm OK with closing this issue, and thank you for your help! :-)
from rust-snappy.
gzip
supports this flag, so I'd be in favor of adding it.
from rust-snappy.
I will note thought, that I wonder why you want this. find ./ -type f -print0 | xargs -0 -P8 szip
should work just fine and do it in parallel.
from rust-snappy.
Oh, thank you! I'd forgotten about the -P
flag to xargs
. I always reach for one of the versions of parallel
, and I always conclude, "No, not worth sorting out all the issues."
I'll try this on our next cluster job and see how it goes. If it works well, we should just close this issue.
from rust-snappy.
Related Issues (20)
- Use std::arch for crc checking if available HOT 3
- szip compression on a directory
- Example decompressor fails against Mark Twin sample HOT 7
- Seems to be incompatible with python and java versions of snappy HOT 19
- Reader/Writer inner get_mut() HOT 2
- no binary in releases HOT 1
- FrameDecoder panics on malformed input HOT 6
- [decompress::TagEntry::len]: documentation is wrong
- Is there a reason the the MAX_INPUT_SIZE limit? HOT 2
- Question: PyO3 wrapped snappy, slower than python-snappy HOT 4
- 3 MB of test files are uploaded to crates.io HOT 1
- Async support HOT 1
- decompress_len output size, unexpected value HOT 1
- Read trait for raw::Encoder/Decoder HOT 4
- FrameDecoder returns EOF for any input < 4 bytes HOT 1
- Concatenating multiple Encoder streams HOT 5
- look into failing roundtrip tests on macos HOT 1
- Unexpected Chunk Type Byte 224 HOT 1
- `write::FrameDecoder` HOT 4
- Resetting Encoder structs HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rust-snappy.