boundaryml / baml Goto Github PK

_{From SyncLinear.com | CLI-20}

null

_MIS-9

Update w/ latest cleaner intro

It would be great in baml files to be able to chain functions together and have some sort of if, then, else logic whereby a model could, for example, run a validation function on a call, and then take action based on it.

Important metric is time to first token

_BAM-10

Only happens in pytest, not thru the actual baml test

null

_BAM-51

VSCode only listens to opened files or files in the baml_src directory. Our current approach of lazily refreshing the list of files in baml_src isn't very robust.

We should just watch for file changes preemptively via a file-watcher mechanism, so we always have the latest state.

Things to keep in mind:

1. If we retrieve a doc from the cache, it should also have the unsaved changes as returned by VSCode onDocumentChanged(..) callbacks. Our playground works off the in-memory content even if you havent saved a file.

Reduces the # of requests to our backend.

I would like to have some separation between azure/anthropic/openai/etc clients and fallback clients

I see fallback clients as a strategy which defines a ruleset for orchestrating various LLM clients. I would prefer if this strategy gave me the ability to set the order of clients (ideally including which ones I can send in parallel), how to deal with various errors (which Baml already provides), and an ability to override default options (such as request timeouts).

Here is a specific example: I have two clients which are both responsible for calling the GPT-3.5-Turbo deployment on Azure given below. The only difference is the request timeout

client<llm> AzureGPT35Turbo {
    provider baml-azure-chat
    retry_policy ZenfetchDefaultPolicy
    options {
      api_key env.AZURE_OPENAI_API_KEY
      api_base env.AZURE_OPENAI_BASE
      engine env.AZURE_GPT_35_TURBO_DEPLOYMENT_NAME
      api_version "2023-07-01-preview"
      api_type azure
      request_timeout 30
    }
}

client<llm> AzureGPT35TurboShortTimeout {
    provider baml-azure-chat
    retry_policy ZenfetchDefaultPolicy
    options {
      api_key env.AZURE_OPENAI_API_KEY
      api_base env.AZURE_OPENAI_BASE
      engine env.AZURE_GPT_35_TURBO_DEPLOYMENT_NAME
      api_version "2023-07-01-preview"
      api_type azure
      request_timeout 5
    }
}

// My version of "strategy"
client<llm> GPTFamilyShortTimeout {
  provider baml-fallback
  options {
    strategy [
      AzureGPT35TurboShortTimeout,
      AzureGPT4TurboShortTimeout
    ]
  }
}

Notice all of the options are the same with the exception of the request_timeout field. The reason I did this is because in certain AI functions, I need the operations to complete quickly, so it's not reasonable to wait for the default 30 second timeout. This leads to a lot of redundancy in my clients (really the "strategies").

With the proposed functionality, I could instead do something like

client<llm> AzureGPT35Turbo {
    provider baml-azure-chat
    retry_policy ZenfetchDefaultPolicy
    options {
      api_key env.AZURE_OPENAI_API_KEY
      api_base env.AZURE_OPENAI_BASE
      engine env.AZURE_GPT_35_TURBO_DEPLOYMENT_NAME
      api_version "2023-07-01-preview"
      api_type azure
      request_timeout 30
    }
}

// My version of "strategy"
client<llm> GPTFamilyShortTimeout {
  provider baml-fallback
  options {
    strategy [
      AzureGPT35Turbo.options(request_timeout=5),
    ]
  }
}

client {
  options {
    strategy [
       // Comments
       GPT4
    ]
  }
}

_MIS-8

Add section on exceptions (which ones are passthrough vs our client-generated ones) and how users should be thinking about them.

null

_BAM-24

null

_BAM-49

If you add an optional field like

function A {
  input (title: string?)
  output string
}

The generated python code always asks you to include the title field, when it should be happy with not providing anything. It's because the parameter to that function call should have a default None value.

_{From SyncLinear.com | CLI-20}

It's a highly requested ask and we should explain how to do this.

null

_BAM-45

Customers want to be able to just iterate on a production request / add it to their test suite.

BAML init should be usable by anyone to add baml to their project.

• Ask for what language they use
• Ask for what package manager (pip or poetry or pnpm or yarn)
• ask for what type of project (empty, tutorial - classification or entity extraction, chatbot, function-calling, etc) they should start with.
• Ask what clients you want (openai, anthropic, azure? a sample of some) [P2]
• Do a confirmation

P0 is just empty with a READMe that just says go to the docs or example repo.

The playground prompt updates on unsaved changes, but the tests do not. So if you run a test you may be running witj and old prompt intil you save your changes.

Find a way to save unsaved .baml changes before running a test so it reflects what is shown in playground.

This causes a confusion when writing

foo 'some value'

then foo == "'some value'" which is bad

_MIS-10

1. Generate baml test cases for a set of N predefined functions
2. Run baml build on json files
3. Run baml test and ensure all tests compile and run

We should have no issues serializing the inputs

Add to CI using github actions

the generated code comes out as:

{"input": "foo"0" "}

which is not parsesable

_MIS-11

We should support an environment variable that can set the stage value of the data.

_BAM-22

Matrix for all toolchains

_{From SyncLinear.com | CLI-18}

_MIS-7

Users are updating the compiler CLI but can forget to regenerate the client code.

In certain situations, I would like to issue multiple LLM calls at once and use whichever client returns first successfully.

I think I would like to have this as part of a strategy option in the client interface.

If LLM outputs a percentage, we should still be able to parse it into an int

Also change the website to have a "copy input".

This is until we have a baml import-test of sorts

current tests require with statements to do mocking, which is ok but adds a lot of unecessary indentention, we should support the same with a decorator at the test level so it works as @patch(...)

_MIS-13

We should find a way to dedupe this information in the analytics dashboard UI.

null

_BAM-53

Some users have said they get confused they have the same name.