Comments (3)
I tend to be using my computer when I'm running Compactor, so leaving resources free is seen as more of a feature than a bug. This also ties in with future plans to make a set-it-and-forget-it background service.
That said I'll probably get around to adding optional concurrent operation at some point.
from compactor.
For what it's worth, the kernel compressor and decompressor does use multiple cores/threads when acting on different chunks of a file. The problem the compressor has is it doesn't know the final size of each compressed chunk, so it's compressing multiple chunks in parallel but can only write the earliest compressed chunk into the file. When using a very lightweight compression algorithm, I'll bet the parallel compression operations are completing quickly so the single copy/write operation ends up being the bottleneck.
Remember that this engine was designed for Surface RT - it uses multiple cores but it was expecting multiple very slow cores.
Also since the API is per-file, it can only parallelize large files effectively. With a Ryzen 2600 (12 hardware threads) and XPress 4Kb, a file would need to be at least 48Kb before the hardware threads can be completely used, but this still ends up with a draining problem because the file isn't compressed until the last task is done. You should see the cores loaded by compressing a 1Gb file with LZX though, where the compression operation is relatively slow.
For what it's worth, I have a (command-line) tool for this that uses a threadpool so multiple files are compressed in parallel, which works fairly well for this problem if you're interested or want to measure the expected results: https://github.com/malxau/yori/tree/master/compact
from compactor.
For what it's worth, you can run multiple instances of compactor to increase the speed of compression by manually selecting folders to compress.
from compactor.
Related Issues (20)
- High CPU use HOT 2
- Update mechanism
- *.bin/*.cas isn't in the exclusion lists, but Compactor still excludes it HOT 2
- PC Crash HOT 1
- Skipping too many files that can be compressed HOT 2
- Wont open anymore - software exception HOT 4
- Question/Suggestion: Use WimBootCompress.ini as default exclusion list. Allows safe compression of C:\
- Windows crashes with a BSoD upon attempting to decompress HOT 1
- Winget repo
- Compressing Discord app folder would reset the session HOT 3
- Windows Search index could be broken after compression
- Windows Defender history is broken after compression HOT 3
- Decludes so many files that it only compressess 5%
- Compactor Seems to copy files to AppData\Local\VirtualStore\ when compressing files on C drive. HOT 2
- how much slower is lzx for 3600x when playing games HOT 2
- Recompress option, and dry run without analyze. Detect and block Windows 7 or fallback to a compatible algorithm. HOT 1
- Question regarding variable placeholder
- Compactor ignoring excluded folder HOT 2
- Suggestion: sizeof (by the maker of ImDisk) Shows compressed or sparse size rather than virtual size for files or directories HOT 2
- Confusing interaction with LZNT1-compressed files HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from compactor.