Comments (5)
LoganDark
commented on May 31, 2024
1
Most implementations with this model of encoding, decoding, and reverse prompt (myself included) are incorrect
Hm, why, given that official
tokenizersPython library is used? Or do you mean only theseifstatements likesome_byte_sequence in string?
implementers forget that sometimes encode(str1) + encode(str2) != encode (str1 + str2)
from rwkv.cpp.
walkrunning
commented on May 31, 2024
1
usually it means the model has decided you have reached [endoftext]
Haha, da Lao. I think you are right.
from rwkv.cpp.
BlinkDL
commented on May 31, 2024
usually it means the model has decided you have reached [endoftext]
from rwkv.cpp.
LoganDark
commented on May 31, 2024
Most implementations with this model of encoding, decoding, and reverse prompt (myself included) are incorrect. Implementations of encoding will become almost correct with World's greedy tokenizer, but decoding and reverse prompt will still be buggy. I hope to document how to do this properly in the future
from rwkv.cpp.
saharNooby
commented on May 31, 2024
Most implementations with this model of encoding, decoding, and reverse prompt (myself included) are incorrect
Hm, why, given that official tokenizers Python library is used? Or do you mean only these if statements like some_byte_sequence in string?
from rwkv.cpp.
Related Issues (20)
- Tutorial for python script? HOT 1
- Repetitive, meaningless tokens output HOT 3
- it seems does not support the newly RWKV-4-World-CHNtuned-3B-v1-20230625-ctx4096.bin HOT 2
- Can't build with ggml HOT 3
- Support hipBLAS
- Support Metal in apple macOS?
- [QUESTION] Implementing RNN/LSTM with ggml HOT 4
- crash on GGML_ASSERT: 'rwkv.cpp/ggml/src/ggml.c:5316: ggml_can_repeat_rows(b, a)' HOT 1
- Update new GGML for GGML_MAX_NODES limit? HOT 2
- Support RWKV v5 HOT 1
- llama-node is not working in the moment HOT 1
- Fix extras/CMakeList.txt file for static build
- Support build with cublas and hipblas on github action HOT 3
- CMake Error HOT 1
- hipblas cannot build using cmake on windows with rocm5.7.1 HOT 2
- The linked Huging Face page in the README doesn't have any .bin files HOT 1
- Add mac ARM build as part of the build process
- Replace all assertions in Python code with if statements HOT 7
- Support RWKV v6
- Colab notebook to start faster?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rwkv.cpp.