Comments (7)
hi, i am very confused, I am reproducing opt-1.3b sparsity. the fact is that i can get the same dense model preplexity at 14.62 of wikitext2, but the preplexity after sparsing 50% is 26.71, which is higher than 17.46 in paper. And I didn't modify any code in this repo. I wonder if the hyperparams in paper's experiments are different? look forward to your reply.
@ilmarkov hi, could you reply to me in your spare time ? i am stuck in this problem. I don't know how i reproduce the result as showed in paper. please
from sparsegpt.
Hi, what command are you running? Generally, the code has been running fine for many people; however, it is possible that some recent HuggingFace update broke things.
from sparsegpt.
Hi, what command are you running? Generally, the code has been running fine for many people; however, it is possible that some recent HuggingFace update broke things.
Thank you for your reply. The command I run is the one in the readme. as follows:
python opt.py facebook/opt-1.3b wikitext2 --sparsity .5 --save ./out/opt-1.3b/sparse0.5/
and huggingface version i am using is 0.23.0
I can run your code succuessfully, but the results are slightly different from the results in paper:
look forward to your reply, Thanks
from sparsegpt.
Hi, in the paper we run with calibration data from c4
and not wikitext2
; the former is not only more general but I think the latter can even sometimes lead to overfitting and correspondingly bad results (possibly this is what you are seeing?).
python opt.py facebook/opt-1.3b c4 --sparsity .5 --save ./out/opt-1.3b/sparse0.5/
from sparsegpt.
Hi, in the paper we run with calibration data from
c4
and notwikitext2
; the former is not only more general but I think the latter can even sometimes lead to overfitting and correspondingly bad results (possibly this is what you are seeing?).python opt.py facebook/opt-1.3b c4 --sparsity .5 --save ./out/opt-1.3b/sparse0.5/
Thank you. To verify if the code is wrong, I test more opt models:
as you can see, Apart from opt-1.3b, the other model results differ only slightly from the paper, So I think the code is right and most reproduced results looks normal too .
That is weird. And I can confirm there is not any modify when i test opt-1.3b.
Besides, The table in your paper shows the result on wikitext2:
from sparsegpt.
We evaluate on wikitext but always use c4 as calibration data (this should be noted in the paper). Yes, this is a bit weird, however there can sometimes be outlier models which do not follow the trend.
from sparsegpt.
We evaluate on wikitext but always use c4 as calibration data (this should be noted in the paper). Yes, this is a bit weird, however there can sometimes be outlier models which do not follow the trend.
yes. i realize i made a mistake. i didn't use c4 as calibration data. Now I solve the problem, Thank you!!!!
from sparsegpt.
Related Issues (20)
- How should I verify the speedup effect of the algorithm? HOT 4
- Purpose of this update
- finetuning sparsified LLaMa HOT 5
- Inference Speedup HOT 3
- Dependencies are wrong HOT 3
- Would sparsegpt be available for Llama2? HOT 3
- When would the code for GPT-J-6B be released?
- Adaptation for Pruning Conv2d or Conv3d Layers? HOT 1
- Can SparseGPT be used on BERT ?
- Using llama.py silently fails and occasionally causes system instability
- transformers version is not correct
- Mistral Support HOT 2
- how to use for Baichuan?
- 2:4 sparsity with to_sparse_semi_structured method from pytorch results in memory issue
- Why Hessian can get by activation ($H = XX^T$) ? HOT 4
- Why transpose the input when in case of nn.Linear or nn.Conv1d?
- AttributeError: 'NoneType' object has no attribute 'shape' HOT 7
- AWQ alongside sparsegpt
- [feature request] all models.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sparsegpt.