A set of unofficial APIs for various TavernAI extensions.

You need to run the lastest version of my TavernAI mod. Grab it here: Direct link to ZIP, Git repository

All modules require at least 6 Gb of VRAM to run. With Stable Diffusion disabled, it will probably fit in 4 Gb. Alternatively, everything could also be run on the CPU.

Try on Colab (runs KoboldAI backend and TavernAI Extras server alongside):

Default requirements.txt contains only basic packages for text processing

If you want to use the most advanced features (like Stable Diffusion, TTS), change that to requirements-complete.txt in commands below. See Modules section for more details.

You must specify a list of module names to be run in the --enable-modules command (caption provided as an example). See Modules section.

• Open colab link
• Select desired "extra" options and start the cell
• Wait for it to finish
• Get an API URL link from colab output under the ### TavernAI Extensions LINK ### title
• Start TavernAI with extensions support: set enableExtensions to true in config.conf
• Navigate to TavernAI settings and put in an API URL and tap "Connect" to load the extensions

• Install Miniconda: https://docs.conda.io/en/latest/miniconda.html
• Install git: https://git-scm.com/downloads
• Before the first run, create an environment (let's call it extras):

conda create -n extras
conda activate extras
conda install pytorch torchvision torchaudio pytorch-cuda=11.7 git -c pytorch -c nvidia
git clone https://github.com/SillyLossy/TavernAI-extras
cd TavernAI-extras
pip install -r requirements.txt

• Run python server.py --enable-modules=caption
• Get the API URL. Defaults to http://localhost:5100 if you run locally.
• Start TavernAI with extensions support: set enableExtensions to true in config.conf
• Navigate to TavernAI settings and put in an API URL and tap "Connect" to load the extensions
• To run again, simply activate the environment and run the script:

conda activate extras
python server.py

• Install Python 3.10: https://www.python.org/downloads/release/python-31010/
• Install git: https://git-scm.com/downloads
• Clone the repo:

git clone https://github.com/SillyLossy/TavernAI-extras
cd TavernAI-extras

• Run pip install -r requirements.txt
• Run python server.py --enable-modules=caption
• Get the API URL. Defaults to http://localhost:5100 if you run locally.
• Start TavernAI with extensions support: set enableExtensions to true in config.conf
• Navigate to TavernAI settings and put in an API URL and tap "Connect" to load the extensions

Name	Description	Required Modules	Screenshot
Image Captioning	Send a cute picture to your bot! Picture select option will appear beside "Message send" button.	caption
Character Expressions	See your character reacting to your messages! You need to provide your own character images! 1. Create a folder in TavernAI called public/characters/<name>, where <name> is a name of your character. 2. For base emotion classification model, put six PNG files there with the following names: joy.png, anger.png, fear.png, love.png, sadness.png, surprise.png. Other models may provide another options. 3. Images only display in desktop mode.	classify
Memory	Chatbot long-term memory simulation using automatic message context summarization.	summarize
Floating Prompt	Adds a string to your scenario after certain amount of messages you send. Usage ideas: reinforce certain events during roleplay. Thanks @Ali឵#2222 for suggesting me that!	None

GET /api/extensions

None

{"extensions":[{"metadata":{"css":"file.css","display_name":"human-friendly name","js":"file.js","requires":["module_id"]},"name":"extension_name"}]}

GET /api/script/<name>

Extension name in a route

File content

GET /api/style/<name>

Extension name in a route

File content

GET /api/asset/<name>/<asset>

Extension name and assert name in a route

File content

POST /api/caption

{ "image": "base64 encoded image" }

{ "caption": "caption of the posted image" }

POST /api/summarize

{ "text": "text to be summarize", "params": {} }

{ "summary": "summarized text" }

POST /api/classify

{ "text": "text to classify sentiment of" }

{
    "classification": [
        {
            "label": "joy",
            "score": 1.0
        },
        {
            "label": "anger",
            "score": 0.7
        },
        {
            "label": "love",
            "score": 0.6
        },
        {
            "label": "sadness",
            "score": 0.5
        },
        {
            "label": "fear",
            "score": 0.4
        },
        {
            "label": "surprise",
            "score": 0.3
        }
    ]
}

NOTES

1. Sorted by descending score order
2. List of categories defined by the summarization model
3. Value range from 0.0 to 1.0

POST /api/keywords

{ "text": "text to be scanned for key phrases" }

{
    "keywords": [
        "array of",
        "extracted",
        "keywords",
    ]
}

POST /api/prompt

{ "name": "character name (optional)", "text": "textual summary of a character" }

{ "prompts": [ "array of generated prompts" ] }

POST /api/image

{ "prompt": "prompt to be generated" }

{ "image": "base64 encoded image" }