Comments (2)
jeanniefinks
commented on May 27, 2024
1
@BDHU If you want to be proactively be alerted to when our Llama 2 developments hit, I strongly encourage you and others reading to signup for our newsletter on the bottom right at https://neuralmagic.com/contact/
The traffic is low, I promise! We are very excited to bring these product developments to you!
I will cross-reference your other similar inquiry just in case and close out this thread. Feel free to reopen if you have additional questions.
Best, Jeannie // Neural Magic
from deepsparse.
mgoin
commented on May 27, 2024
Hi @BDHU , it is completely possible to see speedup on llama2 models similar to MPT as they are a very similar architecture! We don't have examples of it sparsified yet but are working on it and hope to share in the coming weeks
from deepsparse.
Related Issues (20)
- Error when compiling YOLO-NAS model HOT 11
- DeepSparse uses 100% of the CPU. HOT 2
- How to use Benchmark feature on custom dataset HOT 1
- Paper: New algorithm for pruning HOT 3
- Does he support stable diffusion or fooocus, or other image generation acceleration? HOT 1
- Segmentation fault (core dumped) HOT 6
- Grrrrrrr HOT 4
- https://www.therapyinsightspractice.com HOT 1
- Adding Contributors Section to readme.md HOT 1
- Research: 4-bit quantization HOT 5
- Assertion `!cache_sizes.empty()' failed HOT 2
- transformers_embedding-extraction for text-generation tasks HOT 3
- Question on quantization size HOT 1
- NM: error: Node (/model/Add_1) Op (Add) [ShapeInferenceError] Incompatible dimension HOT 5
- Using output_value as "token_embeddings" is broken for Sentence Transformer HOT 2
- docker access denied error HOT 8
- Unsupported ONNX type 10 for FP16 HOT 5
- Assertion at src/lib/core/topology.cpp:627 HOT 1
- yolo-v8 in onnx-runtime outperforms deepsparse on iMX8 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deepsparse.