Comments (2)
upa
commented on September 27, 2024
Thanks for the report.
but I was not expecting 10× worse performance.
So do I. In my environment with Ryzen 9 7950X 16-core CPU, the throughput of a single binary mscp with one connection is about 430MB/s, while the throughput of a normal build is over 1GB/s.
ryzen1 ~/w/m/build > ldd ~/mscp.linux.x86_64.static
not a dynamic executable
ryzen1 ~/w/m/build > ~/mscp.linux.x86_64.static -n 1 ~/5g.img localhost:tmp/
[===============================================] 100% 5.0GB/5.0GB 428.2MB/s 00:13 ryzen1 ~/w/m/build > ldd ./mscp
linux-vdso.so.1 (0x00007fffa1957000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fe6df326000)
libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007fe6deee2000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fe6deec6000)
libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007fe6dee72000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe6dec49000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe6df49b000)
libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007fe6deb7c000)
libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007fe6deb4d000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007fe6deb47000)
libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007fe6deb39000)
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007fe6deb32000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fe6deb1e000)
ryzen1 ~/w/m/build > ./mscp -n 1 ~/5g.img localhost:tmp/
[===============================================] 100% 5.0GB/5.0GB 1.1GB/s 00:05 Does the 10x performance degradation happen on other machines? I guess threadripper would be a cause, but I cannot determine it because I don't have it.
The single binary version of mscp uses musl libc for portability, and it is known that musl libc's memory handling causes performance degradation compared with glibc (ref1, ref2).
from mscp.
baryluk
commented on September 27, 2024
@upa I will test on some other systems soon.
I will also build locally, with glibc and musl (either on Debian, or in docker container), but with same compiler and flags, and see if that it is.
Could be musl memory allocator or pthread support is subpar (glibc probably scales a bit better to more threads and cores), but I would not expect it to perform only only <10 threads.
But, the fact that binary is not showing any thread at 100% does suggest some lock contention (possibly in the allocator).
I will do some profiling with perf later.
from mscp.
Related Issues (20)
- noclobber option HOT 7
- unable to transfer file between RHEL 7 and ubuntu using mscp HOT 6
- Static build will not resolve localhost HOT 5
- No match for method server host key algo HOT 2
- Add support to limit bandwidth HOT 4
- Proxyjump HOT 2
- Issue with resume transfer HOT 6
- Password needs to entered multiple times when resuming from a transfer HOT 2
- Characters in shell are invisible(?) after entering the passwords from resumed transfer HOT 2
- support for zstd compression HOT 1
- CLI documentation / usability / default values HOT 4
- Provide windows build HOT 1
- I added mscp as new port in macports HOT 2
- Hpn-ssh? HOT 3
- Possible to add Resume support? HOT 2
- Unable to parse IPv6 addresses HOT 2
- Configurable intervals between SSH connection attempts HOT 1
- Host runs out of memory resulting in mscp being killed HOT 7
- Possibility to provide password through sshpass HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mscp.