lambdalabsml / falconpdf Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
I am running it out of the box with Falcon 7b and the output is very slow. 5-10 second per word.
I am unable to get this code to function on a local deployment. I'm actually using a g5.2xlarge (provides A10 GPU) instance on AWS running Ubuntu 22.04.
One suggestion (not related to the core issue) would be to include a requirements.txt file so others can easily import the necessary Python libraries. I ran 'app.py' but it still omitted things like the pdf2image and pdfminer libraries.
The core of my issue though is that I am unable to get any response from the model:
chat mode: basic
/home/ubuntu/.venv/lib/python3.10/site-packages/transformers/generation/utils.py:1411: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation )
warnings.warn(
/home/ubuntu/.venv/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:362: UserWarning:do_sample
is set toFalse
. However,temperature
is set to0
-- this flag is only used in sample-based generation modes. You should setdo_sample=True
or unsettemperature
.
warnings.warn(
Settingpad_token_id
toeos_token_id
:11 for open-end generation.
Traceback (most recent call last):
File "/home/ubuntu/.venv/lib/python3.10/site-packages/gradio/routes.py", line 488, in run_predict
output = await app.get_blocks().process_api(
File "/home/ubuntu/.venv/lib/python3.10/site-packages/gradio/blocks.py", line 1435, in process_api
result = await self.call_function(
File "/home/ubuntu/.venv/lib/python3.10/site-packages/gradio/blocks.py", line 1121, in call_function
prediction = await utils.async_iteration(iterator)
File "/home/ubuntu/.venv/lib/python3.10/site-packages/gradio/utils.py", line 350, in async_iteration
return await iterator.anext()
File "/home/ubuntu/.venv/lib/python3.10/site-packages/gradio/utils.py", line 343, in anext
return await anyio.to_thread.run_sync(
File "/home/ubuntu/.venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/home/ubuntu/.venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "/home/ubuntu/.venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
result = context.run(func, *args)
File "/home/ubuntu/.venv/lib/python3.10/site-packages/gradio/utils.py", line 326, in run_sync_iterator_async
return next(iterator)
File "/home/ubuntu/.venv/lib/python3.10/site-packages/gradio/utils.py", line 695, in gen_wrapper
yield from f(*args, **kwargs)
File "/home/ubuntu/FalconPDF/run.py", line 165, in bot
for new_text in streamer:
File "/home/ubuntu/.venv/lib/python3.10/site-packages/transformers/generation/streamers.py", line 223, in next
value = self.text_queue.get(timeout=self.timeout)
File "/usr/lib/python3.10/queue.py", line 179, in get
raise Empty
_queue.Empty
I attempted to both remove 'temperature=temperature' in the run.py script (did not work, led to other odd errors) as well as adding 'do_sample=True' and setting temperature=0.5. This also led to odd errors.
It would be awesome if you could provide some insight as to what could be going on. It would be a bonus if you had working code wrapped as a container. :)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.