helixml / helix Goto Github PK

currently the qapair thing seems to translate non-english input data to english, however we have users who want to be able to do it all in, say, french

when working get back to french user on crisp

E.g 1080

If the user uploads their own qapairs, skip the qapair generation phase

Support multiple GPUs on a single node. Initially we can workaround this just by running N runners with CUDA_VISIBLE_DEVICES passed through to the runner python processes

(e.g. text & image inference to start with)

basically show the curl equivalent of the UI action - i.e. make it clear that you can use the API for each of these actions

it goes to notfound

when done, reply to user

(i.e. train an image and text and combine them into one chat)

There is some useMemo that is reloading (possibly from keycloak) that is causing the "the session has changed scroll to the bottom" behaviour even when the session clearly has not changed - it's annoying because you are actively scrolling up and down just reading and then it will just jump to the bottom of the page

If you put a small bit of text like:

Bob lives at 6 Crow Terrace

It will generate a single question / answer pair and then axotl complains there are too few questions in the training data set

this could be in same session or not

can you make it use GiB not GB? as in, gibibytes 1GB = 1024 * 1024 * 1024 bytes

If I start a conversation / generation with the Create option selected, then it opens and saves a session in the sidebar, but this does not happen when I use the Finetune option.

I cannot find a way to restore my Fine Tune session

also, make it automatically proceed to training when qapairs is completed

For fully private deploys

Let's test out 4bit mixtral, and adapt pkg/dataprep/text/dynamic.go to call into it

some URLs are just javascript and break unstructured - we need a better error: https://www.reuters.com/legal/colorado-ballot-case-adds-fuel-trumps-nomination-drive-2023-12-20/

we need a way to see what happened to a single session in the dashboard

https://mlops-community.slack.com/archives/C0675EX9V2Q/p1701725991656799

See discord. In particular what URL to put in the browser to open the app

Having submitted a document (random doc, outline of a fictional story), then asking what a character should do in the story, I keep being met with "Character should continue fine-tuning the data to improve the accuracy of the model." This seemed to be an inescapable answer, no matter how I posed the question.

It also does not appear to learn from any further conversation I have after the dataset is submitted.

the logic for whether a model instance is stale is currently in 3 places (search for stale := and nonStale :=)

move it to one

if you put a URL to a file in the URL box - detect the bloody mime type so we don't split docs that are downloaded

the URL box should download files first

Vote for this from a user on discord and karolis

in all caps

we should probably distinguish between ACTIVE and IDLE sessions and not kill the active ones

https://mlops-community.slack.com/archives/C0675EX9V2Q/p1701727773319809

Things we did whilst using the old "list"

• when continuing a cloned session, the messages are missing
• if there are no files - the "view files" button shows an error
• "add new documents" button at bottom of text session (add more documents, dataprep new ones into jsonl qa-pairs, concatenate qa-pairs, retrain model)
• retry button for errors
• plugin sentry
• share mode where original training data is not copied
• auto-scroll broken
• put the name of the session in topbar
• rather than system as the username, put the name of the session
• sessions are updating other sessions https://mlops-community.slack.com/archives/C0675EX9V2Q/p1702476943225859
• add a restart button whilst doing a fine-tune so if things get stuck we can restart
  • possibly only show this if we've not seen any progress for > 30 seconds (fixed by the error throwing an error if runner reports job still active)
• dashboard not showing finetune interactions
• performance of auto-save before login (image fine tune text is slow)
• for session updates check we are on the same page
  • whilst we are on one page and another session is processing - it's updating the page we are on with the wrong session
• react is rendering streaming updates to the sessions slowly
• progress bars on text fine tuning
• fork session (fork from an interaction)
• add data after the model is trained
• pdfs are broken in production
• for HTML conversion, use pupetteer to render the page into a PDF then convert the PDF into plain text
• reliable and fast, scale to 5 concurrent users (Luke)
  • Dockerize the runner & deploy some on vast.ai / runpod.io
• finish and deploy dashboard
• logged out state when trying to do things - show a message "please register"
• fix bug with "create image" dropdown etc not working
• fix bug with openAI responding with "GPT 4 Answer: Without providing a valid context, I am unable to generate 50 question and answer pairs as requested"
  • make it so user can see whole message from OpenAI
• replace the thinking face with a spinning progress (small horizontal bouncing three dots)
• there is a dashboard bug where where runner model job history reverses itself
• you lose keyboard focus when the chat box disables and re-enables
• make the chatbox have keyboard focus the first time you load the page
• pasting a long chunk of text into training text box makes the box go taller than the screen and you cannot scroll
• create images says “chat with helix” should say “describe what you want to see in an image”
• enforce min-width on left sidebar
• the event cancel handler on drop downs is not letting you click the same mode
• hide technical details behind "technical details" button ?
  • where it currently says "Session ...." - put the session title
  • put a link next to "View Files" called "Info" that will open a model window with more session details
  • e.g. we put the text summary above in the model along with the ID and other things we want to show
  • in the text box say "Chat with Helix" <- for txt models
  • in the text box say "Make images with Helix" <- for image models
• edit session name (pencil icon to left of bin icon)
• obvious buttons (on fine tuning)
  • in default starting state - make both buttons (add docs / text) - blue and outlined
  • the the default starting state - make the files button say "or choose files"
  • when you start typing in the box make the "Add Text" button pink and make the upload files not pink
  • once there are > 0 files - make the "choose more files" button outlined so the "upload docs" is the main button
• performance on text fine tuning (add concurrency to openAI calls)
• URL to fetch text for text fine tuning
• homepage uncomment buttons
• re-train, will add more interactions to add files to
• we should keep previous Lora files at the interaction level
• we hoist lora_dir from the latest interaction to the session

if it's more than 5 seconds

we already have the "this is taking a while" window - this is to show the place in the queue also

make it more obvious what session we are looking at

we are getting nignx 500's in the runner "load session from api" handler https://mlops-community.slack.com/archives/C0675EX9V2Q/p1702369315736539

If a user does a private deploy they probably don't want our GA or crisp widget

We should enable an opt out version check though

show a dot next to sessions that are currently active or have new replies

probably doesn't need that many epochs, or, at least you could choose to tune it for longer if not happy with the results

make it clear that URLs need to be of text content - for example a youtube URL will not work

when the queue depth for a certain model on a certain runner is high, we might actually want to start multiple instances of a model on a single gpu

In the llm model go code (e.g. here) we build up a prompt that is a formatted string based on the chat template associated with the model.

We could instead store a generic json-ised version of the chat history in task.Prompt, like:

[{"role": "user", "content": "What's the capital of France'?"}, {"role": "assistant", "content": "It's Paris."}]

and the use the model's tokenizer to format the message for us inside axolotl at inference time:

messages = json.loads(json_messages)
tokenizer = AutoTokenizer.from_pretrained(model_name)
encoded_messages = tokenizer.apply_chat_template(new_messages, tokenize=False)

This will reduce the effort needed to add subsequent models with potentially different chat templates.

make sure we are seeing things date ordered

instead we need to truncate the start of the session and allow the user to carry on chatting

reply to user when issue is fixed

quite often there's a model ready to serve and a new one gets spun up on the other node - maybe the clocks are drifting between the machines so the 2 second head start doesn't work? or the python processes aren't polling every 100ms or something?

why? do we need to automatically restart things if they haven't started in a timeout?

once they're done they should exit right?

for each one:

• are there errors? if so, add issues to github. calculate which issues caused the most errors
• is there a trained model with no interactions? if so add to chris's spreadsheet and ping him. also, #43
• were they successful at doing anything?

overall: what % of sessions were successful and what were the biggest pain points? categorise the use cases

example: fine tune on https://arxiv.org/pdf/2311.14465.pdf
no authors show up in plain text version, so of course qapairs is unable to generate questions about who the authors are (or even what the paper is called)

when pasting the link into text fine tuning https://techycompany.com/ the text file has no name

detect when we did not manage to extract any text and tell the user that is the error

autoscale spot runpod instances to match our queue depth: https://graphql-spec.runpod.io/ https://docs.runpod.io/recipes/