Comments (3)
似乎找到了原因。mnn量化以后的模型使用netron查看conv和linear(不清出什么原因,mnn转换以后的模型nn.lienar变成了convolution )weight是空的。 mnncompress + pb 文件转换的模型只有conv算子的weight是空的,linear还是fp32的权重。
使用MNNConverter + --weightQuantBits 8
选项以后就正常。
from mnn.
估计是 mnncompress 里面没有识别出 linear 层出来量化
from mnn.
@jxt1234 感谢回复,但是也感觉正常工作了?pb里也可以看到和linear相关的信息。
另外,请问如果想使用量化的linear和conv是不是只能用mnncompress来量化模型,MNNConverter只能做weight only是吗
from mnn.
Related Issues (20)
- Segmentation Fault when loading an invalid buffer
- 使用内部MNN_BUILD_OPENCV出现问题
- mnn本地推理输出全部为0 HOT 2
- error with armv7 in Hi3751V352 chip HOT 2
- https://github.com/wangzhaode/llm-export 导出Qwen-1_8B-Chat生成的MNN,在andoid 中输出失败的问题 HOT 8
- pymnn: Python Module API 推理报错:Reshape error: 202 -> 0,但是相同模型 Python Session API 推理正常 HOT 4
- MNN release 版本上编译LLM引擎libllm 和 llm_demo 出错 HOT 4
- chinese-bert-wwm-ext模型怎么转换成onnx HOT 1
- MNN-LLM多线程计算结果与单线程不一致 HOT 1
- testMNNFromOnnx.py 验证Qwen-1_8B-Chat ONNX 出错 HOT 1
- 在windows上开启vulkan opencl或者opengl时,怎么指定device_id? HOT 2
- ios demo run 崩溃了 HOT 1
- CPU后端`Tanh`算子在Windows平台Release编译情况下推理结果出错
- yolov8s onnx2mnn 转换后与onnxruntime推理精度存在差异
- 请问ios的metal后端支持pytorch平台的PixelUnshuffle和PixelShuffle算子吗
- 使用 llm_demo 运行Qwen-1_8B-Chat模型的llm.mn, 没有交互
- 英伟达T4上运行OpencCL报错
- 文档网页无法访问
- 如何将MNN的numpy对象转换为numpy对象?
- Android跑两个模型,OpenCL只能在一个上面生效。
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mnn.