Comments (11)
I might take a look at this, it's a cool feature :)
from miniserve.
Yeah, so the way I imagine it to work is to have a link on every page that says "Download as tar | zip" and then it would redirect to <host>/some_dir/.zip
or <host>/some_dir/.tar
. Then we just need to hope that there is no .zip
(and others) file in there which I suppose is a fair assumption to make. We should probably also allow for compression when users choose <host>/some_dir/.tar.(gz,bz,xz,etc)
.
from miniserve.
We'd have to do some streaming. See here at the very bottom.
Also, I think we'd best use a query param like ?download=targz
, ?download=zip
, etc. I'd like to stay Rust native all in all so we'd have to use a compression library that is pure Rust. There is libflate and tar as well as zip-rs.
from miniserve.
Using query parameters is probably better yes, when #44 is merged we'll have the struct to parse this.
from miniserve.
We now have downloading of archives but not streaming as originally intended. I'm going to leave this open for now.
from miniserve.
We really gotta find a crate that allows streaming. That's the last killer feature we lack
from miniserve.
Yes, definitely. Perhaps even add that to tar-rs
.
from miniserve.
So, it seems it's doable with tar-rs, according to a comment on Reddit (thanks a lot to him). I could not try it yet, but here is the link of the a tool which is quite similar to miniserve, but has streaming support for folder downloads using tar-rs: https://github.com/lnicola/rusty-share
I must admit I'm not very experienced with async stuff, so it's not guaranteed I understand how he did it :p (relevant comment: https://www.reddit.com/r/rust/comments/bapows/whats_everyone_working_on_this_week_152019/ekmrzhm/)
cc @vojta7 (you, on the other hand, seem to understand async very well ;))
from miniserve.
I can try to look into it, but I wouldn't call my understanding of async good.
from miniserve.
tar-rs
provide streaming to a Write
. We need to connect this Write
to the actix response - but actix response expects a futures::Stream
, which might potentially be approximated by a Read
.
So what we need to do is have some bytes buffer shared by both a Read
and a Write
. Unsurprisingly, the Write
will write to the pipe (and block/yield when it's full), and the Read
will read from it (and block/yield when it's empty). One "thread" is started with the tar creation using the Write
, and in the meantime the actix response is created using the Read
. Using threads (and blocking on empty/full buffer) is the traditional solution, but if we want to go full-async we might also have them be tasks and yield when appropriate.
So the work to be done is:
- Create a shared buffer with a
Write
and aRead
halves. This is whatPipe
does inrusty-share
: https://github.com/lnicola/rusty-share/blob/master/src/pipe.rs- The
Read
part might just be afutures::sync::mpsc::Receiver<Bytes>
, so it'll already haveStream<Bytes>
implemented.
- The
- Create an adapter to connect the
Read
to actix-web's response, usingBodyStream
. Should be straightforward. - Connect
tar-rs
to theWrite
(possibly wrapped in aflate2::write::GzEncoder
) instead of using aVec
, and start it in a separate thread.- Ideally we wouldn't need a thread, but it would need
tar-rs
to be async-aware and correctly yield (without data corruption) whenever the pipe is full. No idea if/when this will happen.
- Ideally we wouldn't need a thread, but it would need
from miniserve.
Note that zip
streaming may wait on zip-rs/zip#16 - until then, zip-rs
cannot stream the output.
from miniserve.
Related Issues (20)
- Can the compilation target i686-pc-windows-msvc be added to run on a 32-bit Windows? HOT 15
- Panic when serving non-existent file in SPA mode HOT 1
- support setting permission?
- (systemd) error: unexpected argument ' ' found
- panicked at 'modification time must be after epoch: SystemTimeError(28800s)' HOT 6
- Unable to serve files with embedded newline HOT 2
- hope to add ignore file/dir list parameter HOT 1
- Failed test bind_ipv4_ipv6::case_2 HOT 2
- Does it support hiding some files? HOT 6
- Can't Upload HOT 1
- Add cli flag to disable indexing HOT 1
- Not working with reverse proxy in specific directory HOT 2
- Wget footers : hostname:port is repeated
- Feature request: custom subtitle
- rendering markdown files? HOT 6
- Set upload size limit HOT 2
- Save uploaded file atomically HOT 2
- 上传onedrive问题 HOT 4
- How to use HTTPS HOT 1
- Add Support for HSTS to Enhance HTTPS Security HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from miniserve.