A model that picks the right sized model about llm HOT 6 OPEN

This may be a template and not a model - perhaps llm -t auto

• #23

Not sure what the YAML would look like.

A model might be better through, since then you could combine a template with the -m auto option.

from llm.

I think it's a special model called -m auto

How should it handle some users not having GPT-4 32k access?

I think it should try anyway and error if they don't have the model - it would have errored anyway since they were over 32k tokens.

from llm.

Also need to consider the 3.5 4k vs 16k, guessing this is going to be a pattern that continues as well; models that are "the same" but differ in context length (and pricing).

I think there needs to be some concept of "flavors" of models, and in llm you should be able to select the base "flavor" you want and have the model be selected based on a number of other factors (including context length).

from llm.

Worth noting this is a problem that other tools are facing right now as well. I'm not aware of any consensus on how to handle it as of yet, but it's probably worth looking into.

from llm.

this is relevant to the new -c flag as well, as a conversation that fits in the context of one model may outgrow it, and ideally you can continue the conversation without interuption.

from llm.

Dropped from the 0.5 milestone, it's not critical for that.

I'm actually thinking this might make more sense as a llm-auto plugin. It could be expanded to cover all kinds of other heuristics, not just the length of the context.

from llm.