Giter Club home page Giter Club logo

typeai's Introduction

TypeAI: An AI Engineering Framework for TypeScript

GitHub top language GitHub

TypeAI Example

TypeAI is a toolkit for building AI-enabled apps using TypeScript that makes things look so simple it seems like magic. More importantly, it makes building with LLMs "feel" like ordinary code with low impedance mismatch.

An example:

import { toAIFunction } from '@typeai/core'

/** @description Given `text`, returns a number between 1 (positive) and -1 (negative) indicating its sentiment score. */
function sentimentSpec(text: string): number | void {}
const sentiment = toAIFunction(sentimentSpec)

const score = await sentiment('That was surprisingly easy!')

Just specify your types and function signatures as you naturally would, and TypeAI will generate the appropriate implementation respecting your type declarations. No loading separate schema files, no prompt engineering, and no manually writing JSON Schema representations of your functions.

Contents

  1. Installation
  2. Usage
    • Using TypeAI to generate functionality
      • AI Models
      • AI Functions
      • AI Classifiers
    • Using TypeAI to expose functionality to an LLM
      • AI "Tool Functions"
  3. Gotchas
  4. How does it work?
  5. Future Direction & TODOs
  6. Acknowledgements
  7. License

Support

Follow me on Twitter: Twitter Follow

Installation

DeepKit is required in order to provide runtime type information on your functions and types.

npm install @typeai/core @deepkit/core

NOTE: For now, automatic extraction of JSDoc @description tags requires these forked npm package builds @deepkit/type and @deepkit/type-compiler

npm install @deepkit/type@npm:@jefflaporte/[email protected]
npm install --save-dev @deepkit/type-compiler@npm:@jefflaporte/[email protected]
# Bash
./node_modules/.bin/deepkit-type-install
# PowerShell
pwsh ./node_modules/.bin/deepkit-type-install.ps1

tsconfig.json

// tsconfig.json
{
  "compilerOptions": {
    // ...

    // Note: DeepKit says that experimentalDecorators is not necessary when using @deepkit/type,
    // but I have found that deepkit's typeOf() does not always work with TypeScript > 4.9
    // without experimentalDecorators set.
    "experimentalDecorators": true
  },
  "reflection": true
}

NOTE: Some runtimes, such as tsx, won't work with Deepkit. See Gotchas for more info.

At execution time

export OPENAI_API_KEY='...'    # currently required for core functionality
export BING_API_KEY='...'      # if using predefined SearchWeb Tool function

TypeAI makes connecting your functions and types to AI APIs like OpenAI's chat completion endpoints lightweight by using runtime type reflection on TypeScript code to generate the JSON schema required by OpenAI's function calling feature, and by handling function dispatch and result delivery to the LLM.

Usage

TypeAI currently provides two main areas of functionality:

  • Generation of "magic" AI-backed functions
    • AI Models
    • AI Functions
    • AI Classifiers
  • Generation and handing of LLM tool function glue
    • AI "Tool Functions"

AI Functions

To create an AI-backed function, write a stub function and pass it to toAIFunction(), which will generate an AI-backed function with the desired behaviour.

/** @description Given `text`, returns a number between 1 (positive) and -1 (negative) indicating its sentiment score. */
function sentimentSpec(text: string): number | void {}
const sentiment = toAIFunction(sentimentSpec)

const score = await sentiment('That was surprisingly easy!')

Functions with complex input and output TypeScript types work too. Here's a more interesting example:

type Patient = {
  name: string
  age: number
  isSmoker: boolean
}
type Diagnosis = {
  condition: string
  diagnosisDate: Date
  stage?: string
  type?: string
  histology?: string
  complications?: string
}
type Treatment = {
  name: string
  startDate: Date
  endDate?: Date
}
type Medication = Treatment & {
  dose?: string
}
type BloodTest = {
  name: string
  result: string
  testDate: Date
}
type PatientData = {
  patient: Patient
  diagnoses: Diagnosis[]
  treatments: Treatment | Medication[]
  bloodTests: BloodTest[]
}

/** @description Returns a PatientData record generate from the content of doctorsNotes notes. */
function generateElectronicHealthRecordSpec(input: string): PatientData | void {}
const generateElectronicHealthRecord = toAIFunction(generateElectronicHealthRecordSpec, {
  model: 'gpt-4',
})

TypeScript enums to AI-backed Classifiers

enum AppRouteEnum {
  USER_PROFILE = '/user-profile',
  SEARCH = '/search',
  NOTIFICATIONS = '/notifications',
  SETTINGS = '/settings',
  HELP = '/help',
  SUPPORT_CHAT = '/support-chat',
  DOCS = '/docs',
  PROJECTS = '/projects',
  WORKSPACES = '/workspaces',
}
const AppRoute = toAIClassifier(AppRouteEnum)

const appRouteRes = await AppRoute('I need to talk to somebody about billing')

AI "Tool Function" Helpers

An AI tool function is a function provided to an LLM for it's own use in generating answers.

Say you have a function and want to provide it's functionality to OpenAI's LLM for use with their Function Calling feature.

See:

TypeAI provides three functions that make exposing your functions and models to GPT-3.5/4, and handling the resulting function call requests from GPT-3/4, transparent:

static ToolFunction.from<R>(
  fn: (...args: any[]) => R,
  options?: ToolFunctionFromOptions
): ToolFunction

static ToolFunction.modelSubmissionToolFor<T>(
  cb: (arg: T) => Promise<void>
): ToolFunction

function handleToolUse(
  openAIClient: OpenAIApi,
  originalRequest: CreateChatCompletionRequest,
  responseData: CreateChatCompletionResponse,
  options?: {
    model?: string,
    registry?: SchemaRegistry,
    handle?: 'single' | 'multiple'
  },
): Promise<CreateChatCompletionResponse | undefined>

They can be used like this:

import {
  OpenAIApi,
  Configuration,
  CreateChatCompletionRequest,
  ChatCompletionRequestMessage,
  ChatCompletionRequestMessageRoleEnum,
} from 'openai'
import { ToolFunction, handleToolUse } from '@typeai/core'
import { getCurrentWeather } from 'yourModule'

// Init OpenAI client
const configuration = new Configuration({ apiKey: process.env.OPENAI_API_KEY })
const openai = new OpenAIApi(configuration)

// Generate JSON Schema for function and dependent types
const getCurrentWeatherTool = ToolFunction.from(getCurrentWeather)

// Run a chat completion sequence
const messages: ChatCompletionRequestMessage[] = [
  {
    role: ChatCompletionRequestMessageRoleEnum.User,
    content: "What's the weather like in Boston? Say it like a weather reporter.",
  },
]
const request: CreateChatCompletionRequest = {
  model: 'gpt-3.5-turbo',
  messages,
  functions: [getCurrentWeatherTool.schema],
  stream: false,
  max_tokens: 1000,
}
const { data: response } = await openai.createChatCompletion(request)

// Transparently handle any LLM calls to your function.
// handleToolUse() returns OpenAI's final response after
// any/all function calls have been completed
const responseData = await handleToolUse(openai, request, response)
const result = responseData?.choices[0].message

/*
Good afternoon, Boston! This is your weather reporter bringing you the latest
updates. Currently, we're experiencing a pleasant temperature of 82 degrees Celsius. The sky is a mix of sunshine and clouds, making for a beautiful day. However, there is a 25% chance of precipitation, so you might want to keep an umbrella handy. Additionally, the atmospheric pressure is at 25 mmHg. Overall, it's a great day to get outside and enjoy the city. Stay safe and have a wonderful time!
*/

Gotchas

Due to the way Deepkit injects it's type-compiler transform, by patching tsc, some runtimes may not work. These are know NOT to work:

  • tsx

How does it work?

TypeAI uses TypeScript runtime type info provided by @deepkit/type to:

  • generate replacement functions with the same signature as your function stubs
  • generate JSON Schema descriptions of your function and dependent types which are provided to the OpenAI API so that it can repect your desired type structure

This results in a coding experience that feels "native".

Example

import { ToolFunction, handleToolUse } from '@typeai/core'

// Your type definitions
// ...
// Your function definitions dependent on your types
// ...
// eg:
const getCurrentWeather = function getCurrentWeather(
  location: string,
  unit: TemperatureUnit = 'fahrenheit',
  options?: WeatherOptions,
): WeatherInfo {
  const weatherInfo: WeatherInfo = {
    location: location,
    temperature: 82,
    unit: unit,
    precipitationPct: options?.flags?.includePrecipitation ? 25 : undefined,
    pressureMmHg: options?.flags?.includePressure ? 25 : undefined,
    forecast: ['sunny', 'cloudy'],
  }
  return weatherInfo
}

// Register your function and type info
const getCurrentWeatherTool = ToolFunction.from(getCurrentWeather)

// Run a completion series
const messages: ChatCompletionRequestMessage[] = [
  {
    role: ChatCompletionRequestMessageRoleEnum.User,
    content: "What's the weather like in Boston? Say it like a weather reporter.",
  },
]
const request: CreateChatCompletionRequest = {
  model: 'gpt-3.5-turbo-0613',
  messages,
  functions: [getCurrentWeatherTool.schema],
  stream: false,
  max_tokens: 1000,
}
const { data: response } = await openai.createChatCompletion(request)
const responseData = await handleToolUse(openai, request, response)
const result = responseData?.choices[0].message
console.log(`LLM final result: ${JSON.stringify(result, null, 2)}`)

Note: The OpenAI completion API does not like void function responses.

Future Direction & TODOs

  • TODO

Acknowledgements

  • the Prefect / Marvin Team
    • The concept of source-code-less AI Functions, Models, and that use function specification and description info to auto-generate their behavior is originally due to the amazing team at PrefectHQ that created prefecthq/marvin for use in Python.
  • Wang Chenyu

License

See LICENSE.txt

typeai's People

Contributors

jefflaporte avatar khill-fbmc avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

typeai's Issues

How to import the module / methods

I followed the installation instructions, and reloaded VSCode yet I am still faced with errors...

image

I copied the sentiment example from the README but I am unsure how to import the library

Loading OpenAI API Key

I am not sure where you want it to show in the README, but I think it would be helpful to include a note stating we (consumers) need to have OPENAI_API_KEY defined as an environment variable at runtime. It took me a minute to deduce that from investigation since I thought the lib was doing the extra lifting for me behind the scenes.

This is my preferred method of an ephemeral run of a TS file:

npx dotenv-cli -e .env tsx index.ts

Another alternative would be to include an exported method that would allow a user to pass in their key.

What are your thoughts?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.