Saving the pruned checkpoint? about sparsegpt HOT 4 CLOSED

@AlpinDale , no it does not, unfortunately

from sparsegpt.

The last commit introduces --save parameter so it is possible now.

from sparsegpt.

The last commit introduces --save parameter so it is possible now.

Thanks. I did manage to implement a save function in my fork, but the zeroed weights were kept so the pruned model wasn't cut in half (in terme of disk space). Does this new save function handle that?

from sparsegpt.

@AlpinDale disk storage is the same as for dense model since we save model in the same fashion as original dense model. We could have used some compressed storage format, for high sparsity CSR is quite good (for 50-60% sparsity it is very inefficient) or could store a binary mask indicating the location of non-zero weights, store only nonzero elements as a flattened array and decode it. But note, that this is a research repository and the main message is the ability to prune LLM to 50-60% without significant loss and not about the practical compression.

from sparsegpt.