Comments (3)
Cool, those might make some numbers suitable to add to "the ecosystem doc" (https://github.com/magic-wormhole/magic-wormhole/blob/master/docs/ecosystem.rst) or somewhere in the general protocols repo
from magic-wormhole.rs.
Wonder what could be the bottleneck.
There could be many reasons why the mac implementation is slower in your specific case. It's impossible to tell, unless you profile it. Could be interesting to know. Just to make sure: You compile against aarch64, not against x86_64?
from magic-wormhole.rs.
I found the potential issue. I reviewed the code of the python and golang version with a debugger and found that the rust implementation has a different chunk size for each send. Python and Golang implementations both send 16KB at a time while the rust version sends 4KB at a time.
After setting chunk size to 16KB I get full speed.
Python
File sending starts from here https://github.com/magic-wormhole/magic-wormhole/blob/02407c4aa4cc3f8d8cd01d549fdc72a5f5d77010/src/wormhole/cli/cmd_send.py#L442
fs = basic.FileSender()
with self._timing.add("tx file"):
with progress:
if filesize:
# don't send zero-length files
yield fs.beginFileTransfer(
self._fd_to_send,
record_pipe,
transform=_count_and_hash)
The chunk size is defined in twisted package twisted.protocols.basic.FileSender
File chunks are read here https://github.com/twisted/twisted/blob/02a2b658cd1ade5d7f41f97d898913686313e615/src/twisted/protocols/basic.py#L892
CHUNK_SIZE
is a constant defined as CHUNK_SIZE = 2**14
(which is 16384bytes, 16kB)
at https://github.com/twisted/twisted/blob/02a2b658cd1ade5d7f41f97d898913686313e615/src/twisted/protocols/basic.py#L857
Golang
The golang implementation also defines chunk size to be 16KB (recordSize := (1 << 14)
)
Rust
Rust is using 4KB.
magic-wormhole.rs/src/transfer/v1.rs
Line 585 in 6082d8b
I changed 4096 to 16384 and build a release build that gives me 117MB/s when sender is M1 pro Mac.
I think the rust implementation can also use 16KB.
from magic-wormhole.rs.
Related Issues (20)
- [security] `send --code` does not verify that a nameplate was provided and uses the entire code as the nameplate instead HOT 7
- RUSTSEC-2023-0037: crate has been renamed to `crypto_secretbox` HOT 1
- Release mac binaries HOT 2
- RUSTSEC-2023-0052: webpki: CPU denial of service in certificate path building
- [Feature request] Drop xsalsa20poly1305 dependency HOT 3
- Adapt help message for renamed binary HOT 1
- Windows binary for v0.6.1 HOT 2
- Enable transfers over link scoped addresses
- Expose the TLS implementation features of async-tungstenite
- CI test failures HOT 1
- For Android HOT 3
- [FR] Please publish binary crate on crates.io, too
- Automatically generate man pages HOT 1
- test_send_many fails on amd64 and armel/hf HOT 6
- RUSTSEC-2024-0357: `MemBio::get_buf` has undefined behavior with empty buffers
- Crate bytemuck 1.16.1 is yanked
- Panic on port forward in version 0.7.1 HOT 1
- No way to get the name/path of the received file when using --noconfirm/--yes HOT 1
- Use the tracing crate for structured logging
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from magic-wormhole.rs.