Giter Club home page Giter Club logo

assistance's Introduction

Assistance

The Python AI assistance library.

Installation

WASI Python Sandbox

Install wasmtime:

curl https://wasmtime.dev/install.sh -sSf | bash

Download python.wasm:

wget https://github.com/assistancechat/assistance/releases/download/python-wasm/python-3.11.1.wasm -O ~/.assistance/wasm/python.wasm

Test that it works:

wasmtime \
  --env PYTHONPATH=/assistance:/assistance/.venv/lib/python3.11/site-packages \
  --mapdir /assistance::$HOME/git/assistance \
  ~/.assistance/wasm/python.wasm \
  -- -c "print('hello world!')"

Acknowledgment to https://wasmlabs.dev/articles/python-wasm32-wasi/ where much of the above was inspired from.

Server Hosting

Supervisor

First time setup of supervisor:

sudo ln -s $HOME/git/assistance/dev/server/supervisor.conf /etc/supervisor/conf.d/assistance.conf
sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl status assistance

Restart supervisorctl

sudo supervisorctl restart assistance

Nginx

First time setup of nginx:

sudo ln -s $HOME/git/assistance/dev/server/nginx-site.conf /etc/nginx/sites-enabled/assistance
sudo nginx -s reload

Notes

Dev Tooling

Poetry Python version

poetry env use $(which python)

Sync data locally

rsync -r assistance:~/.assistance/* ~/.assistance/
rsync -r server:~/.assistance/* ~/.assistance/

Jupyter

poetry run python -m ipykernel install --user --name assistance
jupyter lab --ip=10.0.0.163

assistance's People

Contributors

simonbiggs avatar kennie12345 avatar camacaccount avatar

Stargazers

 avatar  avatar KittoZheng avatar  avatar  avatar Jarrian Gojar avatar  avatar

Watchers

 avatar

assistance's Issues

Focus Trap

@Kennie12345, not sure if this is worth focusing on at the moment. But sometimes clicking the "contact us" button and then clicking off the model results in the following warning:

There are no focusable elements inside the

Along with the page freezing up and not being able to be scrolled anymore.

Model switching issue

Currently if one does the following steps:

  • Chat (login)
  • Presses question
  • Closes model
  • Opens Enquire

Then when the chat responds it force closes enquire and opens the chat. Need to fix this.

Raymond's Rules on Concurrency

From https://youtu.be/9zinZmE3Ogk?t=2920
and https://web.archive.org/web/20180110194745/http://pybay.com/site_media/slides/raymond2017-keynote/threading.html

RR 1000

ANY shared resource SHALL be run in EXACTLY ONE thread. ALL communication with that thread SHALL be done using an atomic message queue: typically the Queue module, email, message queues like RabiitMQ or ZeroMQ, interstingly you can commincate via a database as well.

Resources that need this technique: global variables, user input, output devices, files, sockets, etc.

Some resources that already have locks inside (thread-safe): logging module, decimal module (thread local variables), databases (reader locks and writer locks), email (this is an atomic message queue).

RR 1001

One category of sequencing problems is to make sure that step A and step B happen sequentially. The solution is to put both in the same thread where all actions proceed sequentially.

RR 1002

To implement a "barrier" that waits for parallel threads to complete, just join() all of the threads

RR 1003

You can't wait on daemon threads to complete (they are infinite loops). Instead, you join() on the queue itself. It waits until all the requested tasks are marked as being done.

RR 1004

Sometimes you need a global variable to communicate between functions. Global variables work great for this purpose in a single threaded program. In multi-threaded code, mutable global state is a disaster. The better solution is to use a threading.local() that is global WITHIN a thread but not without.

RR 1005

Never try to kill a thread from something external to that thread. You never know if that thread is holding a lock. Python doesn't provide a direct mechanism to kill threads externally; however, you can do it using ctypes, but that is a recipe for a deadlock.

RR 1006

Locks don'y lock anything. They are just flags and can be ignored. It is a cooperative tool, not an enforced tool.

RR 1006

In general, locks should be considered a low level primitive that is difficult to reason about in non-trivial examples. For more complex applications, you're almost always better off with using atomic message queues.

RR 1007

The more locks you acquire at one time, the more you lose the advantage of concurrency.

Handle model timeout errors

Jan 20 08:34:54 api python[70963]: INFO: 141.168.162.42:0 - "POST /chat/student/continue HTTP/1.0" 500 Internal Server Error
Jan 20 08:34:54 api python[70963]: ERROR: Exception in ASGI application
Jan 20 08:34:54 api python[70963]: Traceback (most recent call last):
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 419, in run_asgi
Jan 20 08:34:54 api python[70963]: result = await app( # type: ignore[func-returns-value]
Jan 20 08:34:54 api python[70963]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in call
Jan 20 08:34:54 api python[70963]: return await self.app(scope, receive, send)
Jan 20 08:34:54 api python[70963]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/fastapi/applications.py", line 270, in call
Jan 20 08:34:54 api python[70963]: await super().call(scope, receive, send)
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/starlette/applications.py", line 124, in call
Jan 20 08:34:54 api python[70963]: await self.middleware_stack(scope, receive, send)
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in call
Jan 20 08:34:54 api python[70963]: raise exc
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in call
Jan 20 08:34:54 api python[70963]: await self.app(scope, receive, _send)
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/starlette/middleware/cors.py", line 92, in call
Jan 20 08:34:54 api python[70963]: await self.simple_response(scope, receive, send, request_headers=headers)
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/starlette/middleware/cors.py", line 147, in simple_response
Jan 20 08:34:54 api python[70963]: await self.app(scope, receive, send)
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in call
Jan 20 08:34:54 api python[70963]: raise exc
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in call
Jan 20 08:34:54 api python[70963]: await self.app(scope, receive, sender)
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in call
Jan 20 08:34:54 api python[70963]: raise e
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in call
Jan 20 08:34:54 api python[70963]: await self.app(scope, receive, send)
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/starlette/routing.py", line 706, in call
Jan 20 08:34:54 api python[70963]: await route.handle(scope, receive, send)
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle
Jan 20 08:34:54 api python[70963]: await self.app(scope, receive, send)
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/starlette/routing.py", line 66, in app
Jan 20 08:34:54 api python[70963]: response = await func(request)
Jan 20 08:34:54 api python[70963]: ^^^^^^^^^^^^^^^^^^^
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/fastapi/routing.py", line 235, in app
Jan 20 08:34:54 api python[70963]: raw_response = await run_endpoint_function(
Jan 20 08:34:54 api python[70963]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/fastapi/routing.py", line 161, in run_endpoint_function
Jan 20 08:34:54 api python[70963]: return await dependant.call(**values)
Jan 20 08:34:54 api python[70963]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 20 08:34:54 api python[70963]: File "/home/simon/git/assistance.chat/src/python/assistance/api/routers/chat.py", line 103, in student_chat_continue
Jan 20 08:34:54 api python[70963]: response = await run_student_chat_response(
Jan 20 08:34:54 api python[70963]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 20 08:34:54 api python[70963]: File "/home/simon/git/assistance.chat/src/python/assistance/conversations/student.py", line 115, in run_student_chat_response
Jan 20 08:34:54 api python[70963]: response = await call_gpt_and_store_as_transcript(
Jan 20 08:34:54 api python[70963]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 20 08:34:54 api python[70963]: File "/home/simon/git/assistance.chat/src/python/assistance/conversations/student.py", line 133, in call_gpt_and_store_as_transcript
Jan 20 08:34:54 api python[70963]: completions = await openai.Completion.acreate(prompt=prompt, **model_kwargs)
Jan 20 08:34:54 api python[70963]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/openai/api_resources/completion.py", line 45, in acreate
Jan 20 08:34:54 api python[70963]: return await super().acreate(*args, **kwargs)
Jan 20 08:34:54 api python[70963]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 217, in acreate
Jan 20 08:34:54 api python[70963]: response, _, api_key = await requestor.arequest(
Jan 20 08:34:54 api python[70963]: ^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/openai/api_requestor.py", line 311, in arequest
Jan 20 08:34:54 api python[70963]: resp, got_stream = await self._interpret_async_response(result, stream)
Jan 20 08:34:54 api python[70963]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/openai/api_requestor.py", line 646, in _interpret_async_response
Jan 20 08:34:54 api python[70963]: self._interpret_response_line(
Jan 20 08:34:54 api python[70963]: File "/home/simon/.cache/pypoetry/virtualenvs/assistance-zIqiKnAa-py3.11/lib/python3.11/site-packages/openai/api_requestor.py", line 680, in _interpret_response_line
Jan 20 08:34:54 api python[70963]: raise self.handle_error_response(
Jan 20 08:34:54 api python[70963]: openai.error.RateLimitError: The model is currently overloaded with other requests (timed out after generating some tokens) - please retry with smaller completion length or fewer completions (requested 1 prompts, 2 completions for each)

Affiliate link implementation

Heya @Kennie12345,

You might be interested. The affiliate link implementation is actually not saving anything anywhere. Instead, it is creating a token that contains the affiliate's email address within it, and then signing and encrypting the token with our secret key. That way, only we can create these tokens with these email addresses contained within them.

https://github.com/assistancechat/assistance.chat/blob/b1334c8f43a727d5ae90e385ca0a136ea399b897/src/python/assistance/_admin/apps/affiliate.py#L49-L61

It also means that the information regarding the affiliate link is actually all contained within the affiliate link itself. (as long as we have our secret key to decrypt and verify it)

Handle Rate Limit Errors

2023-02-14 17:16:23.205 ERROR: Task exception was never retrieved
future: <Task finished name='Task-4' coro=<_react_to_email() done, defined at /home/simon/git/assistance.chat/src/python/assistance/_api/routers/email.py:61> exception=RateLimitError(message='The server had an error while processing your request. Sorry about that!', http_status=429, request_id=None)>
Traceback (most recent call last):
  File "/home/simon/git/assistance.chat/src/python/assistance/_api/routers/email.py", line 96, in _react_to_email
    await task(email)
  File "/home/simon/git/assistance.chat/src/python/assistance/_agents/email/create.py", line 177, in create_agent
    completions = await openai.Completion.acreate(
  File "/home/simon/git/assistance.chat/src/python/.venv/lib/python3.10/site-packages/openai/api_resources/completion.py", line 45, in acreate
    return await super().acreate(*args, **kwargs)
  File "/home/simon/git/assistance.chat/src/python/.venv/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 217, in acreate
    response, _, api_key = await requestor.arequest(
  File "/home/simon/git/assistance.chat/src/python/.venv/lib/python3.10/site-packages/openai/api_requestor.py", line 311, in arequest
    resp, got_stream = await self._interpret_async_response(result, stream)
  File "/home/simon/git/assistance.chat/src/python/.venv/lib/python3.10/site-packages/openai/api_requestor.py", line 646, in _interpret_async_response
    self._interpret_response_line(
  File "/home/simon/git/assistance.chat/src/python/.venv/lib/python3.10/site-packages/openai/api_requestor.py", line 680, in _interpret_response_line
    raise self.handle_error_response(
openai.error.RateLimitError: The server had an error while processing your request. Sorry about that!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.