Comments (2)
lbeurerkellner
commented on May 9, 2024
Hi there :) Happy to hear that you are enjoying LMQL. Thanks for raising this.
Right, step_budget is another parameter that you potentially need to consider. It forms an upper bound to the maximum number of "steps" the underlying decoding algorithms is allowed to take. For argmax this corresponds directly to the sequence length, for others like beam/best_k, it has a slightly different meaning.
To make this a little more convenient, I just pushed a change on 'main' (should be in the next release), such that step_budget is automatically chosen to be at least as high as max_len. So you should not encounter this issue anymore.
from lmql.
nyacg
commented on May 9, 2024
Brilliant, thanks!
from lmql.
Related Issues (20)
- does Azure really work ? HOT 8
- OpenAIStreamError: logprobs, best_of and echo parameters are not available on gpt-35-turbo model.
- [Question] SSLCertVerificationError when Using lmql with llamaindex HOT 1
- LMQL distribution: `NameError: name 'CLASSIFICATION' is not defined` HOT 2
- Do any of the playground examples work ? HOT 6
- Dolphin models doesn't run HOT 2
- Allow specifying model path for hf models HOT 1
- Is it possible to run LMQL with LMStudio locally ? HOT 2
- Is there any way to replicate function calling like in openai assistants api? HOT 1
- lmql.runtime.tokenizer.TokenizerNotAvailableError
- Add support for OpenAI tools/function-calling API HOT 3
- How can I get classification to work with OpenAI models? HOT 5
- Quotes aren't always parsed properly HOT 3
- Unable to use model loaded on gpu and cpu HOT 1
- Client-side (decoding) memory leak with async @lmql.query HOT 3
- [BUG] sub-queries fail with temperature=0 HOT 1
- Possible error with nests of nested queries in Python HOT 2
- Proposal for new KeyWord `using`
- Model Repeats Same Phrase HOT 1
- `--layout` ignores CUDA_VISIBLE_DEVICES
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lmql.