Comments (4)
AlpinDale
commented on June 11, 2024
Has this been removed and left over in the openai endpoint?
Yes, I haven't updated the OpenAI endpoint (and may remove it soon). For now, I can make a quick update to fix this issue.
from aphrodite-engine.
AWAS666
commented on June 11, 2024
Forked it in the meantime and checking if I find anything else:
https://github.com/AWAS666/aphrodite-engine/tree/patch-1
from aphrodite-engine.
AlpinDale
commented on June 11, 2024
#19 should fix this issue, can you please test?
from aphrodite-engine.
AWAS666
commented on June 11, 2024
Has this been removed and left over in the openai endpoint?
Yes, I haven't updated the OpenAI endpoint (and may remove it soon). For now, I can make a quick update to fix this issue.
Remove the logit_processor or the openai api?
I just gave it a spin on my fork where I just deleted it and it does output, but there seem to be some other issues, not sure if model related so I'll try another one, tried this
But I'll test yours and also download another model to check.
from aphrodite-engine.
Related Issues (20)
- [New Model]: Phi3ForCausalLM
- [Bug]: Fails to start with error UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf8 in position 0: invalid start byte HOT 2
- [Bug]: Cannot load Mixtral GGUF model? HOT 13
- [Installation]: Docker runs out of CPU swap size on 8 GPUs. How to lower swap_space to be less than 4GB per GPU? HOT 1
- [Bug]: Moe's no longer working HOT 3
- [Bug]: [rank0]: KeyError: 'input_ids' HOT 2
- [Usage]: Higher Context Length. HOT 2
- [Feature]: WARNING: Model is quantized. Forcing float16 datatype HOT 4
- [Misc]: INT8 kv quant seems removed.
- [Bug]: unable use all the vram in wsl cuda environment
- [Bug]: /metrics Endpoint Returns 404
- [Feature]: An alternative to `max_tokens` which defaults to `minimum(max_tokens, remaining_tokens)`
- [Bug]: SnowStorm-v1.15-4x8B: Watchdog caught collective operation timeout: WorkNCCL(SeqNum=1, OpType=BROADCAST, NumelIn=128, NumelOut=128, Timeout(ms)=600000)
- [Usage]: OOM crash following Offline Inference setup HOT 3
- [Feature]: Speculative decoding with dual GPUs
- [Bug]: Segmentation fault (core dumped)
- [Bug]: Docker container refuses connection (read ECONNRESET)
- [Installation]: pip installs no executable HOT 3
- [Feature]: Suggestion for build older versions of aphrodite engine's docker images
- [Bug]: Cannot start GGUF FP16 models
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aphrodite-engine.