aws-samples / aws-lex-conv-faq Goto Github PK

Demonstration of LLM integration into a lex bot using Lambda codehooks and a Sagemaker endpoint.

License: MIT No Attribution

Python 94.87% Dockerfile 3.80% Batchfile 1.33%

aws-cdk aws-lex aws-lex-lambda aws-sagemaker large-language-models huggingface llama-index

aws-lex-conv-faq's Issues

[Error] when uploading txt file to source materials bucket

Revieve following error from the read-source-and-build-index Lambda function when I try to upload attached .txt file to the source materials bucket. Able to execute with a pdf instead
allergies_faq.txt

[ERROR] TypeError: __init__() takes 1 positional argument but 2 were givenTraceback (most recent call last):  File "/var/task/index_creation_app.py", line 63, in handler    documents = [Document(t) for t in text_list]  File "/var/task/index_creation_app.py", line 63, in <listcomp>    documents = [Document(t) for t in text_list] | [ERROR] TypeError: __init__() takes 1 positional argument but 2 were given Traceback (most recent call last):   File "/var/task/index_creation_app.py", line 63, in handler     documents = [Document(t) for t in text_list]   File "/var/task/index_creation_app.py", line 63, in <listcomp>     documents = [Document(t) for t in text_list]
-- | --

[ERROR] Runtime.ImportModuleError: Unable to import module 'index_creation_app': cannot import name 'BaseCache' from 'langchain'

Read-source-and-build-index-fn Lambda Function is failing with:

[ERROR] Runtime.ImportModuleError: Unable to import module 'index_creation_app': cannot import name 'BaseCache' from 'langchain' (/var/task/langchain/__init__.py)Traceback (most recent call last): | [ERROR] Runtime.ImportModuleError: Unable to import module 'index_creation_app': cannot import name 'BaseCache' from 'langchain' (/var/task/langchain/__init__.py) Traceback (most recent call last):
-- | --

[Error] exec ./entrypoint.sh: exec format error

When I run cdk synth on an M1 Mac, my endpoint health checks fail with the following CloudWatch logs:

exec ./entrypoint.sh: exec format error

I have read that this is due to architecture incompatibilities when building on an M1 chip - however I'm not quite sure where in the repository I should update the target platform configuration (one of the included Docker images? An HuggingFace image change? Something else?)

Thanks for any advice.

unable to redeploy

I've made a successfully build of the demo, once and I've deleted all the resources at the end. Now, I'm trying to do the same process again and it doesn't creates all resources as it made in the first round.
Already deleted the CDKToolkit template in cloudformation

Deleted all buckets
Deleted all Lambdas
Deleted the Lex bot
Deleted the ECR Images and Repository
Deleted the Instance in Sagemaker with configuration and model
Deleted the SSM configuration

Now it just creates the Asset bucket + ECR Repository + Images and thats it. when I run the cdk deploy
and I'm attempting to do it in the same acocunt and the same region as before.

Interaction between lex-codehook-fn and Lex Test Chat

When I ask a question in Lex using the Test Chatbot - it sends the query to lex-codehook-fn and the Lambda function starts working. Using DEBUG I can see that is gets my question, produces a Prompt for the LLM and even returns the Top 5 internally. but for some reason (even when I make the threshold 0.1) I never get an answer returned?

query_engine = RetrieverQueryEngine(retriever=retriever, response_synthesizer=synth)
query_input = event["inputTranscript"]

try:
    answer = query_engine.query(query_input)
    if answer.source_nodes[0].score < RETRIEVAL_THRESHOLD:
        answer = OUT_OF_DOMAIN_RESPONSE
except:
    answer = OUT_OF_DOMAIN_RESPONSE

In fact I changed the OUT_OF_DOMAIN_RESPONSE for try and except and what is happening is that the "try" is failing over to except. - this seems to mean that the model does not have confidence in the answers even though they are coming in with a confidence of .6 and higher?

For this reason I am NEVER getting a response in the Chatbot Test except '"I'm sorry, but I am only able to give responses regarding the source topic"

This inspite of the fact that the Function actually did work and has an 'answer' ???

Cloudformation Issues

Hi Guys,

I`m having a Cloudformation issue on the step 2/9 while creating the StaginBucket during "cdk boostrap". Any suggestion?

ValueError: Unsupported model type falcon

I am getting the following error. Will update with FLAN instead and rety.

#033[2m2023-10-06T18:32:49.087661Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Args { model_id: "tiiuae/falcon-7b-instruct", revision: None, sharded: None, num_shard: Some(1), quantize: None, trust_remote_code: false, max_concurrent_requests: 128, max_best_of: 2, max_stop_sequences: 4, max_input_length: 1000, max_total_tokens: 1512, max_batch_size: None, waiting_served_ratio: 1.2, max_batch_total_tokens: 32000, max_waiting_tokens: 20, port: 8080, shard_uds_path: "/tmp/text-generation-server", master_addr: "localhost", master_port: 29500, huggingface_hub_cache: Some("/tmp"), weights_cache_override: None, disable_custom_kernels: false, json_output: false, otlp_endpoint: None, cors_allow_origin: [], watermark_gamma: None, watermark_delta: None, env: false }
--
#033[2m2023-10-06T18:32:49.087762Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Starting download process.
#033[2m2023-10-06T18:32:51.646683Z#033[0m #033[33m WARN#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m No safetensors weights found for model tiiuae/falcon-7b-instruct at revision None. Downloading PyTorch weights.
#033[2m2023-10-06T18:32:51.722142Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download file: pytorch_model-00001-of-00002.bin
#033[2m2023-10-06T18:33:02.752506Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Downloaded /tmp/models--tiiuae--falcon-7b-instruct/snapshots/cf4b3c42ce2fdfe24f753f0f0d179202fea59c99/pytorch_model-00001-of-00002.bin in 0:00:11.
#033[2m2023-10-06T18:33:02.752599Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download: [1/2] -- ETA: 0:00:11
#033[2m2023-10-06T18:33:02.752840Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download file: pytorch_model-00002-of-00002.bin
#033[2m2023-10-06T18:33:07.745517Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Downloaded /tmp/models--tiiuae--falcon-7b-instruct/snapshots/cf4b3c42ce2fdfe24f753f0f0d179202fea59c99/pytorch_model-00002-of-00002.bin in 0:00:04.
#033[2m2023-10-06T18:33:07.745585Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download: [2/2] -- ETA: 0
#033[2m2023-10-06T18:33:07.745660Z#033[0m #033[33m WARN#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m No safetensors weights found for model tiiuae/falcon-7b-instruct at revision None. Converting PyTorch weights to safetensors.
#033[2m2023-10-06T18:33:07.745780Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Convert /tmp/models--tiiuae--falcon-7b-instruct/snapshots/cf4b3c42ce2fdfe24f753f0f0d179202fea59c99/pytorch_model-00001-of-00002.bin to /tmp/models--tiiuae--falcon-7b-instruct/snapshots/cf4b3c42ce2fdfe24f753f0f0d179202fea59c99/model-00001-of-00002.safetensors.
#033[2m2023-10-06T18:33:19.153703Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Convert: [1/2] -- Took: 0:00:11.407687
#033[2m2023-10-06T18:33:19.153776Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Convert /tmp/models--tiiuae--falcon-7b-instruct/snapshots/cf4b3c42ce2fdfe24f753f0f0d179202fea59c99/pytorch_model-00002-of-00002.bin to /tmp/models--tiiuae--falcon-7b-instruct/snapshots/cf4b3c42ce2fdfe24f753f0f0d179202fea59c99/model-00002-of-00002.safetensors.
#033[2m2023-10-06T18:33:24.271356Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Convert: [2/2] -- Took: 0:00:05.117383
#033[2m2023-10-06T18:33:24.829104Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Successfully downloaded weights.
#033[2m2023-10-06T18:33:24.829294Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Starting shard 0
#033[2m2023-10-06T18:33:28.178304Z#033[0m #033[31mERROR#033[0m #033[1mshard-manager#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Error when initializing model
Traceback (most recent call last):  File "/opt/conda/bin/text-generation-server", line 8, in <module>    sys.exit(app())  File "/opt/conda/lib/python3.9/site-packages/typer/main.py", line 311, in __call__    return get_command(self)(*args, **kwargs)  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1130, in __call__    return self.main(*args, **kwargs)  File "/opt/conda/lib/python3.9/site-packages/typer/core.py", line 778, in main    return _main(  File "/opt/conda/lib/python3.9/site-packages/typer/core.py", line 216, in _main    rv = self.invoke(ctx)  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1657, in invoke    return _process_result(sub_ctx.command.invoke(sub_ctx))  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1404, in invoke    return ctx.invoke(self.callback, **ctx.params)  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 760, in invoke    return __callback(*args, **kwargs)  File "/opt/conda/lib/python3.9/site-packages/typer/main.py", line 683, in wrapper    return callback(**use_params)  # type: ignore  File "/opt/conda/lib/python3.9/site-packages/text_generation_server/cli.py", line 67, in serve    server.serve(model_id, revision, sharded, quantize, trust_remote_code, uds_path)  File "/opt/conda/lib/python3.9/site-packages/text_generation_server/server.py", line 155, in serve    asyncio.run(serve_inner(model_id, revision, sharded, quantize, trust_remote_code))  File "/opt/conda/lib/python3.9/asyncio/runners.py", line 44, in run    return loop.run_until_complete(main)  File "/opt/conda/lib/python3.9/asyncio/base_events.py", line 634, in run_until_complete    self.run_forever()  File "/opt/conda/lib/python3.9/asyncio/base_events.py", line 601, in run_forever    self._run_once()  File "/opt/conda/lib/python3.9/asyncio/base_events.py", line 1905, in _run_once    handle._run()  File "/opt/conda/lib/python3.9/asyncio/events.py", line 80, in _run    self._context.run(self._callback, *self._args)
> File "/opt/conda/lib/python3.9/site-packages/text_generation_server/server.py", line 124, in serve_inner    model = get_model(model_id, revision, sharded, quantize, trust_remote_code)  File "/opt/conda/lib/python3.9/site-packages/text_generation_server/models/__init__.py", line 314, in get_model    raise ValueError(f"Unsupported model type {model_type}")
ValueError: Unsupported model type falcon #033[2m#033[3mrank#033[0m#033[2m=#033[0m0#033[0m
#033[2m2023-10-06T18:33:28.832440Z#033[0m #033[31mERROR#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Shard 0 failed to start:
Traceback (most recent call last):  File "/opt/conda/bin/text-generation-server", line 8, in <module>    sys.exit(app())  File "/opt/conda/lib/python3.9/site-packages/text_generation_server/cli.py", line 67, in serve    server.serve(model_id, revision, sharded, quantize, trust_remote_code, uds_path)  File "/opt/conda/lib/python3.9/site-packages/text_generation_server/server.py", line 155, in serve    asyncio.run(serve_inner(model_id, revision, sharded, quantize, trust_remote_code))  File "/opt/conda/lib/python3.9/asyncio/runners.py", line 44, in run    return loop.run_until_complete(main)  File "/opt/conda/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete    return future.result()  File "/opt/conda/lib/python3.9/site-packages/text_generation_server/server.py", line 124, in serve_inner    model = get_model(model_id, revision, sharded, quantize, trust_remote_code)  File "/opt/conda/lib/python3.9/site-packages/text_generation_server/models/__init__.py", line 314, in get_model    raise ValueError(f"Unsupported model type {model_type}")
 

<br class="Apple-interchange-newline">

MemorySize issue with lambda function

while running the code in the step of "cdk deploy LexGenAIDemoFilesStack" i receive error , below is the error

❌ LexGenAIDemoFilesStack failed: Error: The stack named LexGenAIDemoFilesStack failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Resource handler returned message: "'MemorySize' value failed to satisfy constraint: Member must have value less than or equal to 3008 (Service: Lambda, Status Code: 400, Request ID: c44f0e2d-457d-4c11-837e-35da515dfa2f)

how to resolve it,
i also requested in service quota of lambda service to increase concurrency , but after increment still problem is there.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.