pathwaycom / llm-app Goto Github PK

Steps to reproduce

Is there any way to pass all outputs to Slack for alert?

Relevant log output

Sorry, it is not a bug, but rather an ask for a favor

What did you expect to happen?

No matter if it is alert or not, pass it to slack

Version

Current main branch

Docker Versions (if used)

No response

OS

Linux

On which CPU architecture did you run Pathway?

x86-64

I assume that currently, the LLM App API client wrapper for OpenAI API does not support this streaming completions feature.

It is nice to have it where we can stream ChatGPT final responses into Pathway's output connectors such as Kafka, Redpanda or Debezium.

References:

https://github.com/openai/openai-cookbook/blob/main/examples/How_to_stream_completions.ipynb
https://platform.openai.com/docs/api-reference/completions/create#completions/create-stream

Logs don't show any activity. Contexfull works as expected

Hi,

In prompt engineering, flexibility comes from two directions: how users correctly define instructions, and how LLMs respond to these instructions. The ambiguity in generated responses by LLMs seems to be still a problem that many LLM app developers are trying to mitigate. In this context, how Pathway LLM app can solve this problem systematically?

Steps to reproduce

Which option to use if the usecase is :

1. Using local HF model
2. Auto update index from specific folder
3. Store embeddings in database

Love this project !

Relevant log output

Not available

What did you expect to happen?

Easier way to deploy and use

Version

1

Docker Versions (if used)

No response

OS

MacOS

On which CPU architecture did you run Pathway?

ARM64 (AArch64, Apple silicon)

I have installed several quantized open source LLM (e.g. LLaMA.cpp, ChatGLM2-6B-int4 ..etc) for private Chatbot apps and I want to use some locally installed embedding model (e.g. SentenceTransformers) instead of OpenAI embedding API.
My question is:
Can Pathway llm-app be used for open source LLM without OpenAI embedding API?
Any example or tutorials on how to do that?

Thanks
Benny

As reported by @Pipboyguy on discord, the Pathway website should be updated to speak of code execution examples, in line with the llm-app/README.

Currently we have:

Flatten data structures in a Jsonline file can not be mapped to structured schemas automatically.

For example, list_price and current_price mapping to the scheme is failing:

{"position": 1, "link": "https://www.amazon.com/Avia-Resistant-Restaurant-Service-Sneakers/dp/B0BJY1FN8F", "asin": "B0BJXSKK9L", "is_lightning_deal": false, "deal_type": "BEST_DEAL", "is_prime_exclusive": false, "starts_at": "2023-08-14T07:00:08.270Z", "ends_at": "2023-08-21T06:45:08.270Z", "type": "multi_item", "title": "Avia Anchor SR Mesh Slip On Black Non Slip Shoes for Women, Comfortable Water Resistant Womens Food Service Sneakers - Black, Blue, or White Med or Wide Restaurant, Slip Resistant Work Shoes Women", "image": "https://m.media-amazon.com/images/I/3195IpEIRpL._SY500_.jpg", "deal_price": 39.98, "list_price": {"value": 59.98, "currency": "USD", "symbol": "$", "raw": "59.98", "name": "List Price"}, "current_price": {"value": 39.98, "currency": "USD", "symbol": "$", "raw": "39.98", "name": "Current Price"}, "merchant_name": "Galaxy Active", "free_shipping": false, "is_prime": true, "is_map": false, "deal_id": "34f3da97", "seller_id": "A3GMJQO0HY62S", "description": "Avia Anchor SR Mesh Slip On Black Non Slip Shoes for Women, Comfortable Water Resistant Womens Food Service Sneakers - Black, Blue, or White Med or Wide Restaurant, Slip Resistant Work Shoes Women", "rating": 4.16, "ratings_total": 1148, "old_price": 59.98, "currency": "USD"}

In this data schema:

class Price(pw.Schema):
    value: float
    currency: str
    symbol: str
    raw: str
    name: str


class DealResult(pw.Schema):
    position: int
    link: str
    asin: str
    is_lightning_deal: bool
    deal_type: str
    is_prime_exclusive: bool
    starts_at: str
    ends_at: str
    type: str
    title: str
    image: str
    deal_price: Price
    list_price: Price
    current_price: Price
    merchant_name: str
    free_shipping: bool
    is_prime: bool
    is_map: bool

The error I got:

Read data parsed unsuccessfully. field deal_price with no JsonPointer path specified is absent in

Hello 👋

I run a security community that finds and fixes vulnerabilities in OSS. A researcher (@m0kr4n3) has found a potential issue, which I would be eager to share with you.

Could you add a SECURITY.md file with an e-mail address for me to send further details to? GitHub recommends a security policy to ensure issues are responsibly disclosed, and it would help direct researchers in the future.

Looking forward to hearing from you 👍

(cc @huntr-helper)

What key challenges of productionizing LLM applications can Pathway solve and make the process easier?

Steps to reproduce

I am using this repository in a remote mode using ssh and pem key, therefore, I can not use a local address for UI.
Is there any approach to skip the URL part of the curl command or force it to run without opening the URL?

Thank you

Relevant log output

Connection time out

What did you expect to happen?

Run the program without opening url

Version

current main version

Docker Versions (if used)

No response

OS

Linux

On which CPU architecture did you run Pathway?

x86-64

Currently, there is no way to send the data to the indexing process without creating a doc column from the input.

Need to fix the indexing error:

AttributeError: Table has no column with name doc.
Occurred here:
    Line: query_context = index.query(embedded_query, k=3).select(
    File: /home/bumurzokov/llm-app/src/prompt.py:14

When no doc column defined, it always fails at the index stage:

# Compute embeddings for each document using the OpenAI Embeddings API
embedded_data = contextful(context=documents, data_to_embed=documents.doc)

With APP_VARIANT=local, I'm getting the following with the docker workflow:

[+] Building 1.5s (14/14) FINISHED
 => [llm-app-examples internal] load build definition from Dockerfile                                                      0.0s
 => => transferring dockerfile: 579B                                                                                       0.0s
 => [llm-app-examples internal] load .dockerignore                                                                         0.0s
 => => transferring context: 105B                                                                                          0.0s
 => [llm-app-examples internal] load metadata for docker.io/library/python:3.11                                            0.9s
 => [llm-app-examples 1/9] FROM docker.io/library/python:3.11@sha256:9a1b705aecedc76e8bf87dfca091d7093df3f2dd4765af6c2501  0.0s
 => [llm-app-examples internal] load build context                                                                         0.5s
 => => transferring context: 4.54MB                                                                                        0.5s
 => CACHED [llm-app-examples 2/9] WORKDIR /app                                                                             0.0s
 => CACHED [llm-app-examples 3/9] RUN pip install poetry                                                                   0.0s
 => CACHED [llm-app-examples 4/9] RUN poetry config installer.max-workers 10                                               0.0s
 => CACHED [llm-app-examples 5/9] COPY ./pyproject.toml ./pyproject.toml                                                   0.0s
 => CACHED [llm-app-examples 6/9] COPY ./poetry.lock ./poetry.lock                                                         0.0s
 => CACHED [llm-app-examples 7/9] RUN if [ "local" = "local" ] ; then     poetry install --no-root --with examples --no-i  0.0s
 => CACHED [llm-app-examples 8/9] COPY . .                                                                                 0.0s
 => CACHED [llm-app-examples 9/9] RUN poetry install --only-root                                                           0.0s
 => [llm-app-examples] exporting to image                                                                                  0.0s
 => => exporting layers                                                                                                    0.0s
 => => writing image sha256:945b1565afc71f847839a04c11329ebaeea8e274b6187a8177b78a51855c9732                               0.0s
 => => naming to docker.io/library/llm-app-llm-app-examples                                                                0.0s
Traceback (most recent call last):
  File "/app/./run_examples.py", line 144, in <module>
    cli.main()
  File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/./run_examples.py", line 102, in wrapper
    return func(**kwargs)
           ^^^^^^^^^^^^^^
  File "/app/./run_examples.py", line 112, in local
    return run(**kwargs)
           ^^^^^^^^^^^^^
  File "/app/examples/pipelines/local/app.py", line 58, in run
    embedder = SentenceTransformerTask(model=embedder_locator)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/llm_app/model_wrappers/sentence_transformer/embedding.py", line 11, in __init__
    from sentence_transformers import SentenceTransformer
ModuleNotFoundError: No module named 'sentence_transformers'