AutoCog primary goal is to provide the essential components for constructing comprehensive programming languages that can be effectively executed by language models (LMs). Through the utilization of these programming languages, it will becomes feasible to implement symbolic AI algorithms on top of LM.

At present, AutoCog offers Structured Thoughts Automaton (STA) as its sole Programming Model. STA's language is inherently low-level and may present challenges in terms of maintenance. For those interested in delving deeper, we have also made available a preprint of our submission to CGO-24. See share/cgo24/v1 for the artifacts of that paper.

As a research software, it is important to note that AutoCog undergoes continuous development, which may result in sudden and non-backward compatible changes. Nonetheless, we strive to maintain an up-to-date Demo, offering a glimpse into the capabilities of the framework.

The share directory also contains several notepads, particularly the notebooks for searcher, reader, and writer. The development of these generic STA programs is currently driving the implementation of many features. Particularly, we are building a first set of unit-tests focused on dataflow.

We are developing Finite Thought Automata (FTA) to act as the underlying Machine Model for AutoCog. In a nutshell, FTA enables to express a prompt as a finite automata over the alphabet of the LM. FTA is then used to enforce low-level syntax in the prompt. It ensures that the LM always provides completions that are parseable by the execution model. Initial work on FTA: source and notebook (where we try to force untrained model to ouput numbers as "XX,XXX,XXX.XX" where X is a digit).

As simple as pip install -U git+https://github.com/LLNL/AutoCog.

But, you'll probably want to clone the repository to get the library of programs:

git clone https://github.com/LLNL/AutoCog
pip install -U ./AutoCog

Most of the development is done inside Python notebook (jupiterlab). To get started take a look at the Demo. Eventually, several notebooks demonstrating various part of AutoCog will be provided in the share folder.

python -m autocog --help

usage: __main__.py [-h] [--version] [--lm LM] [--program PROGRAM] [--tool TOOL] [--prefix PREFIX] [--tee TEE] [--fmt FMT] [--serve] [--host HOST] [--port PORT] [--debug] [--command COMMAND]
                   [--opath OPATH]

options:
  -h, --help         show this help message and exit
  --version          show program's version number and exit
  --lm LM            Inlined JSON or path to a JSON file: `{ "text" : { "cls" : "OpenAI", ... } }` see TODO for details. (default: None)
  --program PROGRAM  Inlined JSON or path to a JSON file: `{ "writer" : { "filepath" : "./library/writer/simple.sta", ... } }` see TODO for details. (default: None)
  --tool TOOL        Inlined JSON or path to a JSON file: `{ "search" : { "cls" : "SerpApi", ... } }` see TODO for details. (default: None)
  --prefix PREFIX    String to identify this instance of AutoCog (used when displaying and saving the prompts) (default: autocog)
  --tee TEE          Filepath or `stdout` or `stderr`. If present, prompts will be append to that file as they are executed. (default: None)
  --fmt FMT          Format string used to save individual prompts to files. If present but empty (or `default`), `{p}/{c}/{t}-{i}.txt` is used. `p` is the prefix. `c` is the sequence id of the call. `t`
                     is the prompt name. `i` is the prompt sequence id. WARNING! This will change as the schema is obsolete! (default: None)
  --serve            Whether to launch the flask server. (default: False)
  --host HOST        Host for flask server. (default: localhost)
  --port PORT        Port for flask server. (default: 5000)
  --debug            Whether to run the flask server in debug mode. (default: False)
  --command COMMAND  Inlined JSON or path to a JSON file: `{ 'callee' : 'writer', ... }` see TODO for details. (default: None)
  --opath OPATH      If present, JSON outputs of the commands will be stored in that file. If missing, they are written to stdout. (default: None)

For example, we run commands using:

python3 -m autocog --program '{ "writer" : { "filepath":"./library/writer/simple.sta", "content" : "report", "R" : 3, "P" : 2, "N" : 3 } }' \
                   --lm '{ "text"     : { "cls" : "OpenAI", "config" : { "max_tokens" : 20, "temperature" : 0.4 } } }' \
                   --lm '{ "thought"  : { "cls" : "OpenAI", "config" : { "max_tokens" : 15, "temperature" : 1.0 } } }' \
                   --lm '{ "sentence" : { "cls" : "OpenAI", "config" : { "max_tokens" : 30, "temperature" : 0.7 } } }' \
                   --command '{ "tag" : "writer", "idea" : "using finite automaton to constrain the production of auto-regressive language models"  }'

This command:

• build the architecture with one program and 3 LM wrapppers
• execute the command
• write resutls.json

The goal is to provide a development environment. Particularly, the ability to inspect and edit/replay frames. These are created for each execution of an Automaton (nested when an Automaton call another Automaton). Upon ending, the execution trace of the Automaton is saved in the corresponding frame.

Eventually, we want to use these traces for two purposes:

• replay: edit part of the trace then restart the program from that point
• finetuning: select "succesful" frames to finetune models

Run the command below at the root of the repository to launch a server. It uses quart.

python3 -m autocog --serve --host 0.0.0.0 --port 1001 --program tests/library.json --lm tests/openai.json

tests/library.json instantiates a few programs from the library. tests/openai.json instantiates OpenAI's GPT 3.5 (text-davinci-003).

]

Minimal testing with pipenv run tests/runall.sh:

• Unit-tests
• Llama.cpp wrapper (must set $LLAMA_CPP_MODEL_PATH)
• OpenAI wrapper (must set $OPENAI_API_KEY)

Currently only pushes to master trigger GitHub actions. It tests pip install and STA dataflow (only set of unit-tests so far). Looking for way to tests the LLM on GitHub (don't want to expose an API key or move the 4GB minimum of Llama 7B q4).

Contributions are welcome and encouraged! So far there is only one rule: linear git history (no merge commits). Only the master branch have stable commits, other branches might be rebased without notice.

Version number will increase for each push to master and have a matching tag.

AutoCog is distributed under the terms of the Apache License (Version 2.0) with LLVM exceptions.

All new contributions must be made under Apache-2.0 license (with LLVM exceptions).

SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception

LLNL-CODE-850523