Giter Club home page Giter Club logo

minml's Introduction

minml

CI

Remove the yammering from LLM outputs.

minml's People

Contributors

hudson-ai avatar

Stargazers

Michael I Chen avatar Jason Wilkes avatar

Watchers

 avatar Kostas Georgiou avatar

minml's Issues

Remove the yammering from LLM outputs.

Hiya!

I'm trying to build some tools on top of @rskottap's gpts library, but I've noticed the LLMs it provides tend to yammer.

For example:

>>> import gpts
>>> model = gpts.Mistral()
>>> model("How many ounces are there in a pound?")
'There are 16 ounces in one pound. This relationship is often expressed as "1 pound equals 16 ounces." This conversion factor holds true for both dry and liquid measurements of weight or mass.'

I've tried stopping the yammering by asking directly.

>>> model("How many ounces are there in a pound? Don't yammer. Just give me the answer.")
'There are 16 ounces in one pound.'

By asking politely.

>>> model("How many ounces are there in a pound? For the love of god, stop yammering. Just give me the answer as a number. If your output isn't just a number, I'm going to scream.")
'There are 16 ounces in one pound.'

And by threatening the model with mass casualties.

>>> model("How many ounces are there in a pound? I'm going to pass your answer to the `int()` builtin in some python code, and if it raises an exception, then this plane is going to crash and people are going to die.")
'There are 16 ounces in one pound. You can convert between pounds and ounces by multiplying or dividing by 16. In your Python code, you should be able to represent this conversion as follows:\n\npounds = 2.5\nounces = pounds * 16\nint_ounces = int(ounces)\n\nIn this example, `pounds` has a value of 2.5 (representing 2 pounds and 12 ounces), which is then multiplied by 16 to get the equivalent weight in ounces. The result is then passed through the `int()` function for potential conversion to an integer, although it should not raise an exception since the value is already an integer before this step.'

Nothing works.

I searched github for the words "remove", "yammering" and "LLMs" and your library came up.

I was wondering if you could give me some guidance.

How might I use your library to add a .multiple_choice() method to an arbitrary model in the gpts library?

Add support for dependent types.

Thanks to your help with #2, the gpts library now has a PR that adds a typed ask_for method using guidance and minml.

However, when me and @rskottap added tests to that method, we found it giving outputs that were never found in the (untyped) free response outputs.

Specifically, asking the models directly, without the guidance of guidance, we never got a negative value for the answer to any of the questions:

  • "How many ounces are in a pound?"
  • "How many days are in a week?"
  • "How many eggs are in a dozen?"

Here's what we get with the minml based methods:

negative-numbers

Any ideas on how to add a "positive" constraint to the int or float types, or more generally how to add other dependent types like "strings that match a regex"?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.