andrewveee / nucleo-ai Goto Github PK

An AI assistant beyond the chat box.

License: Apache License 2.0

Python 29.68% HTML 43.76% Shell 0.24% JavaScript 5.25% Vue 19.50% SCSS 1.58%

nucleo-ai's Introduction

Nucleo AI Alpha

An AI assistant app that mixes AI features with traditional personal productivity. The AI can work in the background to answer multiple chats, handle tasks, and stream/feed entries.

Here's a demo video (recorded using OpenHermes Mistral 7b with llama.cpp):

nucleo-alpha.mp4

Warnings:

This is "alpha" software. It is new. There are bugs and issues.

Some features (esp the assistant mode) can use a lot of tokens. Be aware when you are trying this with paid APIs. This is an early release and there are bugs and improvements needed.

Install and Setup

You should have Python 3 and git installed. I've tried to keep the app light-weight, but RAG requires downloading some large files (about 850mb for the embed and cross encoder).

The app does not include the LLM engine. You can connect it to any openai-compatible endpoint, like llama.cpp, oobabooga with the --api flag, and newer versions of ollama. I use llama-cpp-python and together ai quite a bit.

Step 1

If you're using Windows, see this install note.

$ git clone [email protected]:AndrewVeee/nucleo-ai.git

# Create python venv, install torch cpu-only and requirements.txt. Copy config file.
$ ./setup.sh

Step 2

Edit the config file in ./data/config.toml

Find the [llm] block and set your openai_base_url (something like http://localhost:5000/v1) and the openai_api_key if required. You can also set the secret_key at the top if you want to require auth to connect to Nucleo.

Step 3

Run ./start.sh to start the app. The first run will take a bit to download the SentenceTransformers models for RAG support.

Once the app has started, you will see a line like:

* Running on http://127.0.0.1:4742

Open the link in your browser to start using Nucleo!

Step 4

Share your initial feedback on r/nucleoai if you have time - it will help improve the app!

Features

Regular Chat

Nucleo supports regular chat. You can create multiple chats, and choose saved or temporary (throwaway) chat when you create it.

Assistant

The Assistant chat mode has the AI make a list of tasks from the request and perform them. It can perform web searches, add to do entries, create docs, and respond.

This is very early, and most models will make mistakes. Mixtral tends to do pretty well. I personally use OpenHermes Mistral 7b for testing, and it has ~50% success rate unless you are very careful with the wording.

Researcher

In the stream, you can click the "Research" option from "Add to your stream..." to let the AI research a topic in-depth.

It will generate an initial list of research topics, and allow you to configure max depth, max research time, whether it should search the web, and other options.

The researcher will generate a doc with research on the topic, including links it used at the bottom.

Docs (RAG)

You can upload PDFs (I recommend < 5mb right now) in the Docs tab or upload plain text/markdown files. You can create new docs at the top, or paste in a URL to download and convert it to markdown.

With files uploaded, you can enable the Docs chat mode and it should do a decent job of finding relevant content and responding. Of course, this is dependent on your model. For small context models, make sure to set the "Past Messages" chat option low or there won't be much room for RAG results.

Stream

The stream tab shows a list of actions the assistant has performed. You can also add one-shot chats to your stream - Assistant Task, Quick Chat, or Message (paste emails, etc to categorize and summarize). When you create entries, you can replay them later.

To Do

There is a very basic to do list in the app, so the AI can create entries for you. You can add, edit, or remove entries through the tab.

What's Next?

Right now, I'm primarily looking for feedback and ideas, to know how far it is from a daily driver for others.

I have some ideas I really want to try:

Researcher: In the stream, enter a topic and let the AI run for a few minutes, choosing subtopics, running web searches, and writing its findings in a "live document".
Assistant Modes: There are many ways to implement the assistant mode, from quick 1- or 2-shot requests, to complex ReAct LLM loops. I want to experiment with multiple options, some for tiny models, some for large models.
Stream/Feed: I want to have useful integrations with things like email, calendar, and other personal data sources, so it can all be in one place, and the AI can reference it.
More Functions: Each function makes the assistant more useful, and I'd like to have a list of popular requests to build.
Fine Tuned Models: It would be really nice to have a tiny fine-tuned model (like TinyLlama, Phi, etc) that can handle an assistant mode.

Developers

My long term goal for Nucleo is a platform that helps developers build and test ideas and make them available to others to try easily. I'm not a fan of most LLM frameworks, so I'd prefer Nucleo as a collection of tools that you can pick and choose as you need.

The code is pretty messy, but here are some key points to play around:

You can take a look at app/functions/functions.py if you want to add your own function.
Nucleo is API-first. You can connect to it at /api/v1/chat/completions with an OpenAI client.
Nucleo uses "virtual" models. You can set the model to 'assistant' to chat in assistant mode, 'chat' to chat in basic mode, or 'message' to summarize messages.
With chat mode, you can also append a message with {'role': 'config', 'content': '{"rag": true}'} to enable RAG in the chat.
One of the key future features is being able to create your own virtual models, but it's currently a pretty messy process. Take a look at backend/app/ai_models/default_model.py to see the chat model, or assistant_model2.py for a complex example.

There's so much work to do. I'm happy to help if you want to contribute.

Getting Help

Need help?

I've set up a subreddit to discuss issues and ideas: https://www.reddit.com/r/nucleoai/

You can post in the issues tab as well, but the community might give a faster response.

nucleo-ai's People

Contributors

Stargazers

Watchers

nucleo-ai's Issues

Use of tomllib (and not toml)

I hope I don't regret choosing toml - it's supposed to be a simpler config format, but it's not used much.

To be honest I hate toml, not the format itself, but the fact that tomllib only exists in python >= 3.11. And most distributions still use python 3.10 (including me :-) as it's the remcommended stable (and for stable diffusion) version.
You can maybe use the toml module, instead of tomllib? Not sure if it's ok with python >= 3.11

Originally posted by @AndrewVeee in #5 (comment)

Text is not visible in light mode on Windows

Hey, I just got the program up and running on Windows (thanks again to @AndrewVeee for helping with troubleshooting!) and the website starts in light mode, but most of the text isn't visible. I was worried something was broken until I hit the dark mode switch in the top right. It seems like a lot of text is set to be white even in light mode.

I've tried this in Firefox and Chrome, and the problem seems to be consistent across the two browsers.

App should fail to start if config file has errors

If the config file isn't loaded correctly, the app prints an error message but it isn't very obvious.

It should probably stop, otherwise the app will basically look broken without the user settings.

can you make a video on installing everyhting and setting up claude 3 api, Im new

Documenting Setup on Linux (almost made it)

Stumbled across the repo and was interested in trying out the assistant concept with some beefed up local model settings.

My env is quite different than what's used here and my goal was to document my journey to help with docs.

Unfortunately, I couldn't get the UI to communicate correctly with my local LLM, and ran out of the extra time I had to play with it.

Everything below is an artifact of that process, in case it helps someone else!

Stumbled across this and wanted to share my manual setup as my local env is a little different:

using Linux Mint (Ubuntu-based)
using https instead of ssh for git
using conda instead of venv
using torch w/ GPU support
using Miqu-70b w/ 32k context through Textgen Web UI
using npm instead of yarn (managed through nvm)

Backend setup:

git clone https://github.com/AndrewVeee/nucleo-ai.git
cd nucleo-ai
conda create -n nucleo python=3.11
conda activate nucleo
pip install torch
pip install -r backend/requirements.txt 
cp sample/config-sample.toml data/config.toml

Modified config:

server_host = "127.0.0.1"
server_port = 4742
log_level = 3

[llm]
name = 'local-model'
default = true
type = 'openai'

# Set this to the port of your local instance or update to your API service and key.
openai_base_url = 'http://localhost:5000/v1'
openai_api_key = 'none'
openai_model = 'gpt-3.5-turbo'

# NOTE: Since a proper tokenizer isn't used, you should set this to about 2/3 of your
# actual max context size.
context_size = 21000

# Maximum number of completions at a time.
# For local servers (llama.cpp, oobabooga, etc), this should be set to 1, otherwise 
# it might cut off a response to start a new one.
# If you're using an API/serving infrastructure, you can set this higher.
max_concurrent = 1

[embed]
# If you change the embedding model, change this name so Chroma will keep working.
db_name = "bge-large-en-v1.5"
embed_model = "BAAI/bge-large-en-v1.5"
rank_model = "BAAI/bge-reranker-large"

Frontend setup:

Note: Just running npm install failed due to dependency issues between vite and vite-plugin-singlefile. I could've forced it but explored just updating the minor versions of the packages with npm-check-updates first:

$ cd frontend
$ npx npm-check-updates --target minor -u
Upgrading /home/ash/futuristic/nucleo-ai/frontend/package.json
[====================] 7/7 100%

 sass                   ^1.69.5  →  ^1.71.1
 vite                    ^2.0.5  →  ^2.9.17
 vite-plugin-vue2        ^2.0.1  →   ^2.0.3
 vue-template-compiler  ^2.7.14  →  ^2.7.16

After the minor upgrade this worked to get it running and accessible on my local network.

Warning

Using the code below will expose the UI to any network connection. Do not use with internet-facing services unless you know what you're doing.

npm install --force
npx vite serve --host 0.0.0.0 --port 4743

The UI was working but had white bg + white text so was really hard to read.

Switched to dark mode, everything looks good but was showing "Connection Error" top right and the settings didn't pick up the correct IP.

Narrowed it down to setting my local IP plus port 5000 for the OpenAI API. The connection error went away.

Chats still weren't working though, and after deleting something I'm stuck with an undefined list that I can't add to.

I also see this error in the console running vite, looks like it's using port 4742 for something?

  vite v2.9.17 dev server running at:

  > Local:    http://localhost:4743/
  > Network:  http://192.168.0.178:4743/

  ready in 164ms.

3:47:46 AM [vite] http proxy error:
Error: connect ECONNREFUSED 127.0.0.1:4742
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16)
3:47:46 AM [vite] http proxy error:
Error: connect ECONNREFUSED 127.0.0.1:4742
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16) (x2)
3:47:46 AM [vite] http proxy error:
Error: connect ECONNREFUSED 127.0.0.1:4742
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16) (x3)
3:47:46 AM [vite] http proxy error:
Error: connect ECONNREFUSED 127.0.0.1:4742
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16) (x4)
3:47:46 AM [vite] http proxy error:
Error: connect ECONNREFUSED 127.0.0.1:4742
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16) (x5)

Maybe next time!

Check for search results length before trying to download first result

If there are no results for a search query, there is a List index out of range error.

Check the length before retrieving a page to fix the error.

setup.sh does not fully install dependencies (MacOS/Apple Silicon)

After running setup.sh, I had to manually install the following packages in the nucleo venv in order to launch:
flask
fitz
frontend
tools
markdownify
duckduckgo_search
openai
sentence-transformers
chromadb

I tried running pip install -r requirements.txt inside the venv using the list from the repo but that did not work. Putting all the missing packages in one pip install command did, however, work. Nucleo launches now but several features are broken. I suspect there are more things missing.

Also, argon2 refuses to compile on my system. Attempting to install it returns the following error:

(env) (base) quentin@MBP5 nucleo-ai % pip install argon2 --use-pep517 Collecting argon2 Using cached argon2-0.1.10.tar.gz (28 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Building wheels for collected packages: argon2 Building wheel for argon2 (pyproject.toml) ... error error: subprocess-exited-with-error × Building wheel for argon2 (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [26 lines of output] Warning: 'classifiers' should be a list, got type 'tuple' running bdist_wheel running build running build_py creating build creating build/lib.macosx-11.0-arm64-cpython-310 copying argon2.py -> build/lib.macosx-11.0-arm64-cpython-310 running build_ext building '_argon2' extension creating build/temp.macosx-11.0-arm64-cpython-310 creating build/temp.macosx-11.0-arm64-cpython-310/phc-winner-argon2 creating build/temp.macosx-11.0-arm64-cpython-310/phc-winner-argon2/src creating build/temp.macosx-11.0-arm64-cpython-310/phc-winner-argon2/src/blake2 creating build/temp.macosx-11.0-arm64-cpython-310/src clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/homebrew/Caskroom/miniforge/base/include -arch arm64 -fPIC -O2 -isystem /opt/homebrew/Caskroom/miniforge/base/include -arch arm64 -DHAVE_CONFIG_H -DHAVE_POSIX_MEMALIGN=1 -DHAVE_SYSCTL_HW_USERMEM=1 -I./phc-winner-argon2 -I./phc-winner-argon2/src -I./phc-winner-argon2/src/blake2 -I/Users/quentin/Development/nucleo-ai/env/include -I/opt/homebrew/Caskroom/miniforge/base/include/python3.10 -c ./phc-winner-argon2/src/argon2.c -o build/temp.macosx-11.0-arm64-cpython-310/./phc-winner-argon2/src/argon2.o clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/homebrew/Caskroom/miniforge/base/include -arch arm64 -fPIC -O2 -isystem /opt/homebrew/Caskroom/miniforge/base/include -arch arm64 -DHAVE_CONFIG_H -DHAVE_POSIX_MEMALIGN=1 -DHAVE_SYSCTL_HW_USERMEM=1 -I./phc-winner-argon2 -I./phc-winner-argon2/src -I./phc-winner-argon2/src/blake2 -I/Users/quentin/Development/nucleo-ai/env/include -I/opt/homebrew/Caskroom/miniforge/base/include/python3.10 -c ./phc-winner-argon2/src/blake2/blake2b.c -o build/temp.macosx-11.0-arm64-cpython-310/./phc-winner-argon2/src/blake2/blake2b.o clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/homebrew/Caskroom/miniforge/base/include -arch arm64 -fPIC -O2 -isystem /opt/homebrew/Caskroom/miniforge/base/include -arch arm64 -DHAVE_CONFIG_H -DHAVE_POSIX_MEMALIGN=1 -DHAVE_SYSCTL_HW_USERMEM=1 -I./phc-winner-argon2 -I./phc-winner-argon2/src -I./phc-winner-argon2/src/blake2 -I/Users/quentin/Development/nucleo-ai/env/include -I/opt/homebrew/Caskroom/miniforge/base/include/python3.10 -c ./phc-winner-argon2/src/core.c -o build/temp.macosx-11.0-arm64-cpython-310/./phc-winner-argon2/src/core.o clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/homebrew/Caskroom/miniforge/base/include -arch arm64 -fPIC -O2 -isystem /opt/homebrew/Caskroom/miniforge/base/include -arch arm64 -DHAVE_CONFIG_H -DHAVE_POSIX_MEMALIGN=1 -DHAVE_SYSCTL_HW_USERMEM=1 -I./phc-winner-argon2 -I./phc-winner-argon2/src -I./phc-winner-argon2/src/blake2 -I/Users/quentin/Development/nucleo-ai/env/include -I/opt/homebrew/Caskroom/miniforge/base/include/python3.10 -c ./phc-winner-argon2/src/encoding.c -o build/temp.macosx-11.0-arm64-cpython-310/./phc-winner-argon2/src/encoding.o clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/homebrew/Caskroom/miniforge/base/include -arch arm64 -fPIC -O2 -isystem /opt/homebrew/Caskroom/miniforge/base/include -arch arm64 -DHAVE_CONFIG_H -DHAVE_POSIX_MEMALIGN=1 -DHAVE_SYSCTL_HW_USERMEM=1 -I./phc-winner-argon2 -I./phc-winner-argon2/src -I./phc-winner-argon2/src/blake2 -I/Users/quentin/Development/nucleo-ai/env/include -I/opt/homebrew/Caskroom/miniforge/base/include/python3.10 -c ./phc-winner-argon2/src/ref.c -o build/temp.macosx-11.0-arm64-cpython-310/./phc-winner-argon2/src/ref.o clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/homebrew/Caskroom/miniforge/base/include -arch arm64 -fPIC -O2 -isystem /opt/homebrew/Caskroom/miniforge/base/include -arch arm64 -DHAVE_CONFIG_H -DHAVE_POSIX_MEMALIGN=1 -DHAVE_SYSCTL_HW_USERMEM=1 -I./phc-winner-argon2 -I./phc-winner-argon2/src -I./phc-winner-argon2/src/blake2 -I/Users/quentin/Development/nucleo-ai/env/include -I/opt/homebrew/Caskroom/miniforge/base/include/python3.10 -c ./phc-winner-argon2/src/thread.c -o build/temp.macosx-11.0-arm64-cpython-310/./phc-winner-argon2/src/thread.o clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/homebrew/Caskroom/miniforge/base/include -arch arm64 -fPIC -O2 -isystem /opt/homebrew/Caskroom/miniforge/base/include -arch arm64 -DHAVE_CONFIG_H -DHAVE_POSIX_MEMALIGN=1 -DHAVE_SYSCTL_HW_USERMEM=1 -I./phc-winner-argon2 -I./phc-winner-argon2/src -I./phc-winner-argon2/src/blake2 -I/Users/quentin/Development/nucleo-ai/env/include -I/opt/homebrew/Caskroom/miniforge/base/include/python3.10 -c ./src/argon2_py.c -o build/temp.macosx-11.0-arm64-cpython-310/./src/argon2_py.o clang -bundle -undefined dynamic_lookup -Wl,-rpath,/opt/homebrew/Caskroom/miniforge/base/lib -L/opt/homebrew/Caskroom/miniforge/base/lib -Wl,-rpath,/opt/homebrew/Caskroom/miniforge/base/lib -L/opt/homebrew/Caskroom/miniforge/base/lib build/temp.macosx-11.0-arm64-cpython-310/./phc-winner-argon2/src/argon2.o build/temp.macosx-11.0-arm64-cpython-310/./phc-winner-argon2/src/blake2/blake2b.o build/temp.macosx-11.0-arm64-cpython-310/./phc-winner-argon2/src/core.o build/temp.macosx-11.0-arm64-cpython-310/./phc-winner-argon2/src/encoding.o build/temp.macosx-11.0-arm64-cpython-310/./phc-winner-argon2/src/ref.o build/temp.macosx-11.0-arm64-cpython-310/./phc-winner-argon2/src/thread.o build/temp.macosx-11.0-arm64-cpython-310/./src/argon2_py.o -lc_argon2 -o build/lib.macosx-11.0-arm64-cpython-310/_argon2.cpython-310-darwin.so ld: warning: duplicate -rpath '/opt/homebrew/Caskroom/miniforge/base/lib' ignored ld: library 'c_argon2' not found clang: error: linker command failed with exit code 1 (use -v to see invocation) error: command '/usr/bin/clang' failed with exit code 1 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for argon2 Failed to build argon2 ERROR: Could not build wheels for argon2, which is required to install pyproject.toml-based projects [notice] A new release of pip available: 22.3.1 -> 24.0 [notice] To update, run: pip install --upgrade pip

This appears to be something of a common problem. Other implementations like argon2-cffi install without issue.
M1 Max 32GB, MacOS 13.6.3
Apologies if there are trivial solutions to these problems. I am very new to Python.

Connection Test UI

Nucleo needs a UI for testing the connection.

It should do the following:

Determine if it connected to the nucleo backend.
Send a simple chat message, like "Hi"
Either display the response or show a detailed exception.

Steps 1 and 3 should provide more information, possibly link to an FAQ that can grow as more reports are given. Can mention common issues with ollama, oobabooga, external APIs, etc.

Add speech recognition

This is an idea that just popped up, while I was trying to search something.

Maybe we can integrate whisper.cpp or something like that, for interaction? It would be awesome if the Assistant can search the web then, and pass it through the summarization engine.

Advanced web search capabilities

The Researcher tool has a very limited view into the web right now, and the library below handles multiple searches, uses selenium for better scraping, and LLMs to extract snippets:
https://github.com/myrakrusemark/llm-gpt4-browser

This would also be useful for the assistant chat mode.

missing title error: args["title"] -> args[" title"]

Need to change args['title'] to args[' title'] or change dictionary to use 'title' instead of ' title'

Error:

Exception in thread Thread-9 (handler):                                                                                                                                                                              
Traceback (most recent call last):                                                                                                                                                                                   
  File "/usr/lib/python3.11/threading.py", line 1045, in _bootstrap_inner                                                                                                                                            
    self.run()                                                                                                                                                                                                       
  File "/usr/lib/python3.11/threading.py", line 982, in run                                                                                                                                                          
    self._target(*self._args, **self._kwargs)                                                                                                                                                                        
  File "/home/epyc/ai/nucleo-ai/backend/app/ai_models/assistant_model2.py", line 156, in handler                                                                                                                     
    chain.run()                                                                                                                                                                                                      
  File "/home/epyc/ai/nucleo-ai/backend/app/ai_models/ai_model_base.py", line 98, in run                                                                                                                             
    res = self.instance.run_next()                                                                                                                                                                                   
          ^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                   
  File "/home/epyc/ai/nucleo-ai/backend/ai_tools/prompt_chain/chain.py", line 75, in run_next                                                                                                                        
    result = entry.run(self)                                                                                                                                                                                         
             ^^^^^^^^^^^^^^^                                                                                                                                                                                         
  File "/home/epyc/ai/nucleo-ai/backend/ai_tools/prompt_chain/entry.py", line 100, in run                                                                                                                            
    result = self.runner(instance)                                                                                                                                                                                   
             ^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                   
  File "/home/epyc/ai/nucleo-ai/backend/app/ai_models/assistant_model2.py", line 67, in runner                                                                                                                       
    runner.run()                                                                                                                                                                                                     
  File "/home/epyc/ai/nucleo-ai/backend/app/ai_models/ai_model_base.py", line 98, in run               
    res = self.instance.run_next()                                                                     
          ^^^^^^^^^^^^^^^^^^^^^^^^                                                                     
  File "/home/epyc/ai/nucleo-ai/backend/ai_tools/prompt_chain/chain.py", line 75, in run_next          
    result = entry.run(self)                                                                           
             ^^^^^^^^^^^^^^^                                                                           
  File "/home/epyc/ai/nucleo-ai/backend/ai_tools/prompt_chain/entry.py", line 100, in run              
    result = self.runner(instance)                                                                     
             ^^^^^^^^^^^^^^^^^^^^^                                                                     
  File "/home/epyc/ai/nucleo-ai/backend/ai_tools/prompt_chain/entry.py", line 122, in runner           
    self.fn(instance, response, self.opts)                                                             
  File "/home/epyc/ai/nucleo-ai/backend/app/ai_models/function_chain_simple.py", line 71, in call_function
    result = fn.handler(inst, inst.get_opt('function_arguments'))                                      
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                      
  File "/home/epyc/ai/nucleo-ai/backend/app/functions/functions.py", line 187, in run                  
    "result": f"Created doc: {args['title']}",                                                         
                              ~~~~^^^^^^^^^                                                            
KeyError: 'title'                                                                                      
^C^CException ignored in atexit callback: <bound method Client.join of <posthog.client.Client object at 0x7fe952d82d50>>
Traceback (most recent call last):                                                                     
  File "/home/epyc/ai/nucleo-ai/env/lib/python3.11/site-packages/posthog/client.py", line 416, in join 
    consumer.join()                                                                                    
  File "/usr/lib/python3.11/threading.py", line 1119, in join                                          
    self._wait_for_tstate_lock()                                                                       
  File "/usr/lib/python3.11/threading.py", line 1139, in _wait_for_tstate_lock                         
    if lock.acquire(block, timeout):                                                                   
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^       
KeyboardInterrupt:

Stable Diffusion support

AUTOMATIC1111 has an API:
https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/API

This looks really easy to build with! Here are some notes:

The function definition would look something like:

class CreateImage:
  def __init__(self, app):
    self.app = app
    self.reg = app.ai_functions.define(self.run, 'create_image', 'Create a new image',
        short_description="create images")
    self.reg.add_argument('description', 'An in-depth one line description of the image to create', question="What is the description of the image?")
    self.reg.add_argument('count', 'Number of sample images to create (default is 1)', question="How many images did the user request?")
    self.reg.add_to('Artist')

Add a config file section for it:

[a1111]
base_url = 'http://localhost:7860/' # App will append /sdapi/v1/txt2img

Then it's just a matter of sending the description to the API, writing the images to the data/uploads/ folder, and sending the response back, like f"I created an image for you: "

TypeError: can only concatenate str (not "NoneType") to str

Research task leads to this error in the terminal and the UI hangs without any information.

n\nWrite 2 paragraphs completely covering the main topics based on what you know and use the web snippets to help with specifics.\n\nUse your expert abilities to write content, summarize what we've written, and to provide a complete, useful answer to the main topic."}]
Job Exception: timed out
Exception in thread Thread-25 (handler):
Traceback (most recent call last):
  File "/home/user/miniconda3/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/home/user/miniconda3/lib/python3.11/threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "/home/user/Downloads/LLM_Applications/nucleo-ai/backend/app/ai_models/researcher_model.py", line 293, in handler
    researcher.run()
  File "/home/user/Downloads/LLM_Applications/nucleo-ai/backend/app/ai_models/researcher_model.py", line 257, in run
    content = self.result.generate_md()
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Downloads/LLM_Applications/nucleo-ai/backend/app/ai_models/researcher_model.py", line 31, in generate_md
    md += self.content
TypeError: can only concatenate str (not "NoneType") to str

Researcher Virtual Model

Overview:

A research model that takes a topic and loops endlessly to research the topic, write its findings, generate sub-topics, and run for a set period of time (or maximum depth).

User clicks a Research button, enters a topic. Examples: "Beach vacations within 6 hour flight of Los Angeles", "Computer cases by size and price", "What is the meaning of life?"...
LLM generates an initial list of topics to explore. User has a chance to add/remove/edit the topics. They can also choose max run time, and max depth.

Researcher quick improvements

Some quick improvements that can be made to the researcher:

Try to prompt it not to write a conclusion paragraph for the research topics.
Add a UI field for research instructions (like "Write for an expert on the subject").
Add a UI field for final summary instructions (like "Write a code sample in python", "List the most likely options", ...)

AttributeError: 'NoneType' object has no attribute 'split'

Hi, nice tool!
The installation went flawlessly.
I then tried the "Research" stream and it returns the error AttributeError: 'NoneType' object has no attribute 'split'.

Here is the bash console log, up to just before my query:

(base) 18:20:26 piero@gl753vd  nucleo-ai $ ./start.sh 
Fri Mar  8 18:20:35 2024 * Loading modules
[Fri Mar  8 18:20:38 2024] * Loading config
[Fri Mar  8 18:20:38 2024] * Initializing database
[Fri Mar  8 18:20:38 2024] * Initializing RAG
[Fri Mar  8 18:20:40 2024] * Initializing webserver
[Fri Mar  8 18:20:40 2024] * Loading LLM
[Fri Mar  8 18:20:40 2024]   http://192.168.111.181:14434/v1 [key len=4] [model=mistral]
[Fri Mar  8 18:20:40 2024] * Starting job queue
[Fri Mar  8 18:20:40 2024] * Loading functions
[Fri Mar  8 18:20:40 2024]   Secretary: to do lists - ['add_todo']
[Fri Mar  8 18:20:40 2024]   Writer: responses, write docs - ['respond_user', 'create_doc']
[Fri Mar  8 18:20:40 2024]   Artist: take photos, draw, edit media - []
[Fri Mar  8 18:20:40 2024]   Researcher: find, research, look ups, web searches - ['web_search']
[Fri Mar  8 18:20:40 2024]   General: tasks not handled - []
[Fri Mar  8 18:20:40 2024] * Loading Models
[Fri Mar  8 18:20:40 2024]   Models: ['chat', 'assistant-orig', 'assistant', 'message', 'researcher', 'hi']
[Fri Mar  8 18:20:40 2024] * Starting!
 * Serving Flask app 'app.web_server'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:4742
Press CTRL+C to quit
127.0.0.1 - - [08/Mar/2024 18:20:57] "GET / HTTP/1.1" 302 -
127.0.0.1 - - [08/Mar/2024 18:20:57] "GET /index.html HTTP/1.1" 304 -
127.0.0.1 - - [08/Mar/2024 18:20:58] "POST /api//store/list HTTP/1.1" 308 -
127.0.0.1 - - [08/Mar/2024 18:20:58] "POST /api//store/list HTTP/1.1" 308 -
127.0.0.1 - - [08/Mar/2024 18:20:58] "POST /api//store/list HTTP/1.1" 308 -
127.0.0.1 - - [08/Mar/2024 18:20:58] "POST /api//store/list HTTP/1.1" 308 -
127.0.0.1 - - [08/Mar/2024 18:20:58] "POST /api//state/check_auth HTTP/1.1" 308 -
127.0.0.1 - - [08/Mar/2024 18:20:58] "GET /icons/help.svg HTTP/1.1" 304 -
127.0.0.1 - - [08/Mar/2024 18:20:58] "GET /icons/moon.svg HTTP/1.1" 304 -
127.0.0.1 - - [08/Mar/2024 18:20:58] "GET /icons/settings.svg HTTP/1.1" 304 -
[Fri Mar  8 18:20:58 2024] SELECT: select * from user_store where data_type=? order by created_at desc [['todo']]
[Fri Mar  8 18:20:58 2024] SELECT: select id, user_id, data_type, subtype, root_id, parent_type, parent_id, name, metadata, pinned, source, embed_model, ai_created, created_at, updated_at from user_store where data_type=? order by created_at desc [['doc']]
127.0.0.1 - - [08/Mar/2024 18:20:58] "POST /api/store/list HTTP/1.1" 200 -
[Fri Mar  8 18:20:58 2024] SELECT: select * from user_store where data_type=? order by created_at desc [['stream']]
[Fri Mar  8 18:20:58 2024] SELECT: select * from user_store where data_type=? order by created_at desc [['chat']]
127.0.0.1 - - [08/Mar/2024 18:20:58] "POST /api/store/list HTTP/1.1" 200 -
127.0.0.1 - - [08/Mar/2024 18:20:58] "POST /api/store/list HTTP/1.1" 200 -
127.0.0.1 - - [08/Mar/2024 18:20:58] "POST /api/state/check_auth HTTP/1.1" 200 -
127.0.0.1 - - [08/Mar/2024 18:20:58] "POST /api/store/list HTTP/1.1" 200 -

This the log after my query List 5 supplements to improve Insulin Resistance. :

127.0.0.1 - - [08/Mar/2024 18:23:44] "POST /api//v1/chat/completions HTTP/1.1" 308 -
127.0.0.1 - - [08/Mar/2024 18:23:44] "GET /icons/loader.svg HTTP/1.1" 304 -
[Fri Mar  8 18:23:44 2024] Using model: researcher ({'topic': 'List 5 supplements to improve Insulin Resistance.'})
[Fri Mar  8 18:23:44 2024] [Researcher:run_llm] [{'role': 'system', 'content': 'You are an advanced topic researcher. Given this topic:\n```\nList 5 supplements to improve Insulin Resistance.\n```\n\nList the 3-7 main research topics for the text. Keep them short.'}]
Job Exception: Connection error.
Exception in thread Thread-23 (handler):
Traceback (most recent call last):
  File "/usr/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.11/threading.py", line 982, in run
    self._target(*self._args, **self._kwargs)
  File "/mnt/00aadc36-3e91-4512-b272-3e84356ac527/Piero/AI_Lab/Github/nucleo-ai/backend/app/ai_models/researcher_model.py", line 305, in handler
    tq.queue.append(self.generate_block("\n".join(chain.parse_list(job.result, None))))
                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/00aadc36-3e91-4512-b272-3e84356ac527/Piero/AI_Lab/Github/nucleo-ai/backend/ai_tools/prompt_chain/chain.py", line 12, in parse_list
    for line in output.split("\n"):
                ^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'split'

EDIT: forgot to include my config:

# If you make the app externally accessible, you should set a good
# secret key.
#auth_key = 'my s3cret k33y'
server_host = "127.0.0.1"
server_port = 4742
log_level = 3

[llm]
name = 'mistral'
default = true
type = 'openai'

# Set this to the port of your local instance or update to your API service and key.
openai_base_url = 'http://192.168.111.181:14434/v1'
openai_api_key = 'none'
openai_model = 'mistral'

# NOTE: Since a proper tokenizer isn't used, you should set this to about 2/3 of your
# actual max context size.
context_size = 2048

# Maximum number of completions at a time.
# For local servers (llama.cpp, oobabooga, etc), this should be set to 1, otherwise
# it might cut off a response to start a new one.
# If you're using an API/serving infrastructure, you can set this higher.
max_concurrent = 1

[embed]
# If you change the embedding model, change this name so Chroma will keep working.
db_name = "bge-small-en-v1.5"
#embed_model = "BAAI/bge-small-en-v1.5"
#rank_model = "cross-encoder/ms-marco-MiniLM-L-6-v2"

Thank you,
Piero

Disable chromadb telemetry

https://docs.trychroma.com/telemetry

Log the model used for chat messages

It would be nice to store additional metadata for each chat message to know which model was used.

This would probably be 2 fields: virtual_model (eg assistant, chat, etc), llm_model (from the actual config.toml)

It should also include an option to display the model in the chat somewhere.

Assistant Mode: Improve response function calling

(Model: OpenHermes-Mistral-7b)
Given a task like "Tell me about the best python job queues," the assistant often creates the right task list to look up then tell the user. But once it gets to the tell the user task, it chooses to perform a lookup instead of respond.

This also applies to the create_doc tool.

Need to:

Collect similar examples
Update the prompt to increase the chances that it selects the correct function.

Add Send Button for Improved Message Sending Experience

Issue Description:
Currently, users face inconvenience when trying to send messages consisting of multiple lines in the chat interface. Upon selecting multiple lines of text and hitting enter, the message input field moves to a new line instead of sending the message. This behavior interrupts the flow of communication and requires additional steps to send the message, leading to a suboptimal user experience.

Proposed Solution:
To address this issue and enhance the message sending experience, I propose adding a send button adjacent to the chat input field. The send button will provide users with a convenient way to send messages after composing them, irrespective of whether the message spans multiple lines. Users will have the option to either hit enter or click the send button to transmit their messages, providing flexibility and improving usability.

Expected Behavior:

A send button will be displayed next to the chat input field.
Users can compose messages in the input field, including multi-line messages.
After composing the message, users can click the send button to transmit the message.
Clicking the send button will send the entire message, regardless of whether it spans multiple lines.
The send button will complement the existing functionality of hitting enter to send messages, providing users with an alternative method for message transmission.

Benefits:

Improved user experience: Users can easily send messages without the need to hit enter repeatedly or navigate through multi-line text.
Enhanced usability: The addition of a send button offers users more control over when to send their messages, reducing friction in the communication process.
Increased efficiency: With a dedicated send button, users can quickly transmit messages, leading to
smoother interactions and increased productivity.

Implementation Details:

Implement a send button adjacent to the chat input field in the application interface.
Develop functionality to handle message transmission upon clicking the send button.
Ensure seamless integration with existing chat functionality and support for multi-line messages.