This repo contains a Python utility library for creating and maintaining prompts for Large Language Models (LLMs).

LLMs like GPT-3 and Codex have continued to push the bounds of what AI is capable of - they can capably generate language and code, but are also capable of emergent behavior like question answering, summarization, classification and dialog. One of the best techniques for enabling specific behavior out of LLMs is called prompt engineering - crafting inputs that coax the model to produce certain kinds of outputs. Few-shot prompting is the discipline of giving examples of inputs and outputs, such that the model has a reference for the type of output you're looking for.

Prompt engineering can be as simple as formatting a question and passing it to the model, but it can also get quite complex - requiring substantial code to manipulate and update strings. This library aims to make that easier. It also aims to codify patterns and practices around prompt engineering.

See How to get Codex to produce the code you want article for an example of the prompt engineering patterns this library codifies.

pip install prompt-engine-py

The library currently supports a generic PromptEngine, a CodeEngine and a ChatEngine. All three facilitate a pattern of prompt engineering where the prompt is composed of a description, examples of inputs and outputs and an ongoing "dialog" representing the ongoing input/output pairs as the user and model communicate. The dialog ensures that the model (which is stateless) has the context about what's happened in the conversation so far.

See architecture diagram representation:

Code Engine creates prompts for Natural Language to Code scenarios. See Python Syntax for importing CodeEngine and PythonCodeEngineConfig:

from prompt_engine.code_engine import CodeEngine, PythonCodeEngineConfig

NL->Code prompts should generally have a description, which should give context about the programming language the model should generate and libraries it should be using. The description should also give information about the task at hand:

description = "Natural Language Commands to JavaScript Math Code. The code should log the result of the command to the console."

NL->Code prompts should also have examples of NL->Code interactions, exemplifying the kind of code you expect the model to produce. In this case, the inputs are math queries (e.g. "what is 2 + 2?") and code that console logs the result of the query.

from prompt_engine.interaction import Interaction
examples = [
  Interaction("what's 10 plus 18", "console.log(10 + 18)"),
  Interaction("what's 10 times 18", "console.log(10 * 18)")
]

By default, CodeEngine uses Python as the programming language, but you can create prompts for different languages by passing a different CodeEngineConfig into the constructor. If, for example, we wanted to produce JavaScript prompts, we could have passed CodeEngine a javascript_config specifying the comment operator it should be using:

javascript_config = CodeEngineConfig(description_comment_operator = "/*/", description_comment_close_operator = "/*/", 
                                     comment_operator = "/*", comment_close_operator = "*/")
code_engine = CodeEngine(config = javascript_config, description = description, examples = examples)

With our description and our examples, we can use it to create prompts:

query = "What's 1018 times the ninth power of four?"
prompt = code_engine.build_prompt(query)

The resulting prompt will be a string with the description, examples and the latest query formatted with comment operators and line breaks:

/*/ Natural Language Commands to JavaScript Math Code. The code should log the result of the command to the console. /*/

/* what's 10 plus 18 */
console.log(10 + 18)

/* what's 10 times 18 */
console.log(10 * 18)

/* What's 1018 times the ninth power of four? */

Given the context, a capable code generation model can take the above prompt and guess the next line: print(1018 * (4 ** 9)).

For multi-turn scenarios, where past conversations influences the next turn, Code Engine enables us to persist interactions in a prompt:

...
# Assumes existence of code generation model
code = model.generate_code(prompt)

# Adds interaction
code_engine.add_interaction(query, code)

Now new prompts will include the latest NL->Code interaction:

code_engine.build_prompt("How about the 8th power?")

Produces a prompt identical to the one above, but with the NL->Code dialog history:

...
/* What's 1018 times the ninth power of four? */
console.log(1018 * (4 ** 9))

/* How about the 8th power? */

With this context, the code generation model has the dialog context needed to understand what we mean by the query. In this case, the model would correctly generate print(1018 * (4 ** 8)).

Just like Code Engine, Chat Engine creates prompts with descriptions and examples. See Python Syntax for importing CodeEngine and PythonCodeEngineConfig:

from prompt_engine.chat_engine import ChatEngine, ChatEngineConfig

The difference is that Chat Engine creates prompts for dialog scenarios, where both the user and the model use natural language. The ChatEngine constructor takes an optional config argument, which allows you to define the name of a user and chatbot in a multi-turn dialog:

config = ChatEngineConfig(
    user_name = "Abhishek",
    bot_name = "Gordon"
)

Chat prompts also benefit from a description that gives context. This description helps the model determine how the bot should respond.

description = "A conversation with Gordon the Anxious Robot. Gordon tends to reply nervously and asks a lot of follow-up questions."

Similarly, Chat Engine prompts can have examples interactions:

from prompt_engine.interaction import Interaction
examples = [
  Interaction("Who made you?", "I don't know man! That's an awfully existential question. How would you answer it?"),
  Interaction("Good point - do you at least know what you were made for?", "I'm OK at riveting, but that's not how I should answer a meaning of life question is it?")
]

These examples help set the tone of the bot, in this case Gordon the Anxious Robot. Now we can create our ChatEngine and use it to create prompts:

chat_engine = ChatEngine(chatEngineConfig, description, examples)
user_query = "What are you made of?"
prompt = chat_engine.build_prompt(user_query)

When passed to a large language model (e.g. GPT-3), the context of the above prompt will help coax a good Marvin-like answer from the model, like "Subatomic particles at some level, but somehow I don't think that's what you were asking.". As with Code Engine, we can persist this answer and continue the dialog such that the model is aware of the conversation context:

chatEngine.add_interaction(user_query, "Subatomic particles at some level, but somehow I don't think that's what you were asking.")

Dynamic Prompt Engine is another behaviour constructed on top of prompt engine that enables dynamic retrieval of the relevant examples in order to generate a prompt.

It is developed with the belief that giving more relevant examples to the Large Language Model will enable it to generate a better output and much closer to our examples, rather than leaving it to guess for itself. This also allows the ability to coax multiple behaviours out of the Large Language Model instead of having to maintain mutliple different prompts.

The dynamic prompt engine maintains a prompt bank, which is a collection of embeddings of all the examples and interactions that have been provided to it. When given a new unseen prompt, it queries the prompt bank based on the embeddings to retrieve the Top-k relevant examples and adds them to the examples section of the prompt engine output.

Prompts for Large Language Models generally have limited size, depending on the language model being used. Given that prompt-engine can persist dialog history, it is possible for dialogs to get so long that the prompt overflows. The Prompt Engine pattern handles this situation by removing the oldest dialog interaction from the prompt, effectively only remembering the most recent interactions.

You can specify the maximum tokens allowed in your prompt by passing a max_tokens parameter when constructing the config for any prompt engine:

from prompt_engine.model_config import ModelConfig
config = PromptEngineConfig( ModelConfig(max_tokens=1024) )

The following are the functions available on the PromptEngine class and those that inherit from it:

For more examples and insights into using the prompt-engine library, have a look at the examples folder

It can be useful to represent prompts as standalone files, versus code. This can allow easy swapping between different prompts, prompt versioning, and other advanced capabiliites. With this in mind, prompt-engine offers a way to represent prompts as YAML and to load that YAML into a prompt-engine class. See examples/yaml-examples for examples of YAML prompts and how they're loaded into prompt-engine.

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

This library aims to simplify use of Large Language Models, and to make it easy for developers to take advantage of existing patterns. The package is released in conjunction with the Build 2022 AI examples, as the first three use a multi-turn LLM pattern that this library simplifies. This package works independently of any specific LLM - prompt generated by the package should be useable with various language and code generating models.

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.