AI Chatbot support for Q&A and Actions, aka ChatGPT for Monica about monica HOT 3 OPEN

I'd be happy to have the option as it could be a great way to quickly summarize content.

It should be possible to switch between different LLMs easily as local ones will get better and not everyone wants to share the information with large LLM providers which only provide APIs.

from monica.

Just adding a thought. To do this, one would have to send some or all of the database to the LLM as part of the prompt, this would be a large amount of tokens for each request, and the more tokens, the more cost. As such you would first have to do some context discovery (e.g the name of the person) and run a search first, before then only sending the product of that search, however this would severely limit the scope of the chat requests.

from monica.

@HarvsG most of those problems can actually be mitigated by giving the LLM pseudo-functions to run (SEARCH_USERS, USER_INFO, CONVERSATION_HISTORY_FOR_USER) and letting the LLM go through a reasoning loop (meaning multiple round trips) and at each step letting the LLM evaluate if it has enough information to satisfy the request, or if a given function should be run.

This also allows for extensibility to execute modification actions (INSERT_CONVERSATION_ENTRY, UPDATE_USER_DETAILS, etc).

from monica.