kaiokendev / superbig Goto Github PK

Some other user mentioned they want to integrate this into SillyTavern, so I will fix these issues to get it to a usable state, and add the UI for adding sources

• if I try to run anything else before loading the URL, it I get an index out of range error

• after loading the URL (I can load basically any URL without issue) to clear the error, it does not appear to make new sources if I just copy-paste a bunch of text in

• never tries to chunk the context without the addition of a URL

• When dealing with URLs alone, I tried loading a fairly heavy Reddit thread (using old.reddit) and got the following error: chromadb.errors.NotEnoughElementsException: Number of requested results 3 cannot be greater than number of elements in index 2

• Further attempts to ask questions didn't lead to a reattempt at scanning the URL just use an old URL if prompted

• until the Settings and Sources UI tabs are completed. I'd consider those a high priority for baseline usability. Without it, it's hard to know what sources are being considered without reading the console output

When I try to load a pdf it says:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe2 in position 10: invalid continuation byte
Sometimes it changes the byte number, but the rest of the message remains constant.

Is this an error or a feature that could be added?

Maybe it's easier than what I get, but wit hthe explanation in the main page I'm not sure how to use this.

I have a large amount of html pages that I want to feed as information, it's software documentation, I have thm both online and offline (as local html files).

How can I feed all that information and make it available to the AI so it can answer correctly questions related to it?
Is there an external file where I can put my sources?
Also after the sources are loaded, do making a related question automatically uses that data or should I specify something?

Thanks!

https://huggingface.co/Babelscape/rebel-large