Simple web app example serving a PyTorch model using streamlit and FastAPI

Home Page: https://davidefiocco.github.io/streamlit-fastapi-ml-serving

License: MIT License

Dockerfile 11.50% Python 88.50%

fastapi streamlit docker-compose pytorch pycones

streamlit-fastapi-model-serving's People

Contributors

Stargazers

Watchers

Forkers

robmarkcole dorcasoj shir1917 gaylonalfano sooheang abusallam sevaroy jairoandrescastaneda detectiveosint mtvu thias42 sax-dot vishwa360 sultanmu xiskoct riccardo1980 tkahng dunovank zdwang suri1155 hwangjohn v4gadkari chingweiyu ducdh1210 puja129 danielchudc emg110 sanogotech tonykuo222 kanhaiyakumarrai banditelol hartigran minseo0214 hello2tomoki tanglespace golesuman prajamani asrulsibaoel ciepielajan iteimouri obeshor juancaob yangxin1994 larssonhampus reejungkim ccwu0918 cybermedusa annewallace jomorodi cjayb mikful marouane1206 milindsoorya rams502 shivagowri1928 ilyoungkim qqr1 andrewssobral straitrobot jatintyagi-dev shenrun seengel dev-strender duyta506 kidmam networkbiology-graphml lynxye chandrakanth-jp endikamt ppeterlongotg goyalpramod aktienautobahn haizibinbin ax2l 5l1v3r1 jackman337 nadia-el bombzoo ricgu8086 juanvulcano rbtrsv jorgeavilacartes trancongtuyetxuanthu nhantapcode hyunyang-seo feihu618

streamlit-fastapi-model-serving's Issues

after deployment on ubuntu the Streamlit page always shows "Please Wait"

I use ubuntu virtual machine 18.04 to deploy your program.

I use following steps to finish this.

sudo apt install docker.io
sudo apt install docker-compose
systemctl start docker
git clone the project code and enter the directory
sudo docker-compose build
sudo docker-compose up

then the streamlit page always display "Please Wait", when I refresh the page, the terminal logs a problem I search for a long time but I can not resolve. Thanks for your work and waiting for your reply.

streamlit    | Exception in thread ScriptRunner.scriptThread:
streamlit    | Traceback (most recent call last):
streamlit    |   File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
streamlit    |     self.run()
streamlit    |   File "/usr/local/lib/python3.7/threading.py", line 870, in run
streamlit    |     self._target(*self._args, **self._kwargs)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 172, in _process_request_queue
streamlit    |     self._run_script(data)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 272, in _run_script
streamlit    |     self.on_event.send(ScriptRunnerEvent.SCRIPT_STARTED)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/blinker/base.py", line 267, in send
streamlit    |     for receiver in self.receivers_for(sender)]
streamlit    |   File "/usr/local/lib/python3.7/site-packages/blinker/base.py", line 267, in <listcomp>
streamlit    |     for receiver in self.receivers_for(sender)]
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/report_session.py", line 285, in _on_scriptrunner_event
streamlit    |     self._enqueue_new_report_message()
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/report_session.py", line 383, in _enqueue_new_report_message
streamlit    |     _populate_user_info_msg(imsg.user_info)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/report_session.py", line 706, in _populate_user_info_msg
streamlit    |     msg.installation_id = Installation.instance().installation_id
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/metrics_util.py", line 129, in instance
streamlit    |     cls._instance = Installation()
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/metrics_util.py", line 133, in __init__
streamlit    |     self.installation_id_v1 = str(uuid.uuid5(uuid.NAMESPACE_DNS, _get_machine_id()))
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/metrics_util.py", line 49, in _get_machine_id
streamlit    |     subprocess.run(["sudo", "dbus-uuidgen", "--ensure"])
streamlit    |   File "/usr/local/lib/python3.7/subprocess.py", line 488, in run
streamlit    |     with Popen(*popenargs, **kwargs) as process:
streamlit    |   File "/usr/local/lib/python3.7/subprocess.py", line 800, in __init__
streamlit    |     restore_signals, start_new_session)
streamlit    |   File "/usr/local/lib/python3.7/subprocess.py", line 1551, in _execute_child
streamlit    |     raise child_exception_type(errno_num, err_msg, err_filename)
streamlit    | FileNotFoundError: [Errno 2] No such file or directory: 'sudo': 'sudo'
streamlit    | 
streamlit    | Exception in thread ScriptRunner.scriptThread:
streamlit    | Traceback (most recent call last):
streamlit    |   File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
streamlit    |     self.run()
streamlit    |   File "/usr/local/lib/python3.7/threading.py", line 870, in run
streamlit    |     self._target(*self._args, **self._kwargs)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 172, in _process_request_queue
streamlit    |     self._run_script(data)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 272, in _run_script
streamlit    |     self.on_event.send(ScriptRunnerEvent.SCRIPT_STARTED)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/blinker/base.py", line 267, in send
streamlit    |     for receiver in self.receivers_for(sender)]
streamlit    |   File "/usr/local/lib/python3.7/site-packages/blinker/base.py", line 267, in <listcomp>
streamlit    |     for receiver in self.receivers_for(sender)]
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/report_session.py", line 285, in _on_scriptrunner_event
streamlit    |     self._enqueue_new_report_message()
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/report_session.py", line 383, in _enqueue_new_report_message
streamlit    |     _populate_user_info_msg(imsg.user_info)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/report_session.py", line 706, in _populate_user_info_msg
streamlit    |     msg.installation_id = Installation.instance().installation_id
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/metrics_util.py", line 129, in instance
streamlit    |     cls._instance = Installation()
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/metrics_util.py", line 133, in __init__
streamlit    |     self.installation_id_v1 = str(uuid.uuid5(uuid.NAMESPACE_DNS, _get_machine_id()))
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/metrics_util.py", line 49, in _get_machine_id
streamlit    |     subprocess.run(["sudo", "dbus-uuidgen", "--ensure"])
streamlit    |   File "/usr/local/lib/python3.7/subprocess.py", line 488, in run
streamlit    |     with Popen(*popenargs, **kwargs) as process:
streamlit    |   File "/usr/local/lib/python3.7/subprocess.py", line 800, in __init__
streamlit    |     restore_signals, start_new_session)
streamlit    |   File "/usr/local/lib/python3.7/subprocess.py", line 1551, in _execute_child
streamlit    |     raise child_exception_type(errno_num, err_msg, err_filename)
streamlit    | FileNotFoundError: [Errno 2] No such file or directory: 'sudo': 'sudo'
streamlit    | 
streamlit    | Exception in thread ScriptRunner.scriptThread:
streamlit    | Traceback (most recent call last):
streamlit    |   File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
streamlit    |     self.run()
streamlit    |   File "/usr/local/lib/python3.7/threading.py", line 870, in run
streamlit    |     self._target(*self._args, **self._kwargs)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 172, in _process_request_queue
streamlit    |     self._run_script(data)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 272, in _run_script
streamlit    |     self.on_event.send(ScriptRunnerEvent.SCRIPT_STARTED)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/blinker/base.py", line 267, in send
streamlit    |     for receiver in self.receivers_for(sender)]
streamlit    |   File "/usr/local/lib/python3.7/site-packages/blinker/base.py", line 267, in <listcomp>
streamlit    |     for receiver in self.receivers_for(sender)]
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/report_session.py", line 285, in _on_scriptrunner_event
streamlit    |     self._enqueue_new_report_message()
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/report_session.py", line 383, in _enqueue_new_report_message
streamlit    |     _populate_user_info_msg(imsg.user_info)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/report_session.py", line 706, in _populate_user_info_msg
streamlit    |     msg.installation_id = Installation.instance().installation_id
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/metrics_util.py", line 129, in instance
streamlit    |     cls._instance = Installation()
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/metrics_util.py", line 133, in __init__
streamlit    |     self.installation_id_v1 = str(uuid.uuid5(uuid.NAMESPACE_DNS, _get_machine_id()))
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/metrics_util.py", line 49, in _get_machine_id
streamlit    |     subprocess.run(["sudo", "dbus-uuidgen", "--ensure"])
streamlit    |   File "/usr/local/lib/python3.7/subprocess.py", line 488, in run
streamlit    |     with Popen(*popenargs, **kwargs) as process:
streamlit    |   File "/usr/local/lib/python3.7/subprocess.py", line 800, in __init__
streamlit    |     restore_signals, start_new_session)
streamlit    |   File "/usr/local/lib/python3.7/subprocess.py", line 1551, in _execute_child
streamlit    |     raise child_exception_type(errno_num, err_msg, err_filename)
streamlit    | FileNotFoundError: [Errno 2] No such file or directory: 'sudo': 'sudo'
streamlit    | 
streamlit    | Exception in thread ScriptRunner.scriptThread:
streamlit    | Traceback (most recent call last):
streamlit    |   File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
streamlit    |     self.run()
streamlit    |   File "/usr/local/lib/python3.7/threading.py", line 870, in run
streamlit    |     self._target(*self._args, **self._kwargs)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 172, in _process_request_queue
streamlit    |     self._run_script(data)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 272, in _run_script
streamlit    |     self.on_event.send(ScriptRunnerEvent.SCRIPT_STARTED)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/blinker/base.py", line 267, in send
streamlit    |     for receiver in self.receivers_for(sender)]
streamlit    |   File "/usr/local/lib/python3.7/site-packages/blinker/base.py", line 267, in <listcomp>
streamlit    |     for receiver in self.receivers_for(sender)]
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/report_session.py", line 285, in _on_scriptrunner_event
streamlit    |     self._enqueue_new_report_message()
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/report_session.py", line 383, in _enqueue_new_report_message
streamlit    |     _populate_user_info_msg(imsg.user_info)
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/report_session.py", line 706, in _populate_user_info_msg
streamlit    |     msg.installation_id = Installation.instance().installation_id
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/metrics_util.py", line 129, in instance
streamlit    |     cls._instance = Installation()
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/metrics_util.py", line 133, in __init__
streamlit    |     self.installation_id_v1 = str(uuid.uuid5(uuid.NAMESPACE_DNS, _get_machine_id()))
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/metrics_util.py", line 49, in _get_machine_id
streamlit    |     subprocess.run(["sudo", "dbus-uuidgen", "--ensure"])
streamlit    |   File "/usr/local/lib/python3.7/subprocess.py", line 488, in run
streamlit    |     with Popen(*popenargs, **kwargs) as process:
streamlit    |   File "/usr/local/lib/python3.7/subprocess.py", line 800, in __init__
streamlit    |     restore_signals, start_new_session)
streamlit    |   File "/usr/local/lib/python3.7/subprocess.py", line 1551, in _execute_child
streamlit    |     raise child_exception_type(errno_num, err_msg, err_filename)
streamlit    | FileNotFoundError: [Errno 2] No such file or directory: 'sudo': 'sudo'

fastapi exited with code 137

Hi Davide
I am unfamiliar with FastAPI, but it looks very powerful!
On running docker-compose up and uploading an image via the UI I get an error:

ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Traceback:

File "/usr/local/lib/python3.7/site-packages/streamlit/ScriptRunner.py", line 322, in _run_script
    exec(code, module.__dict__)
File "/streamlit/ui.py", line 37, in <module>
    segments = process(image, url + endpoint)
File "/streamlit/ui.py", line 30, in process
    timeout=8000,
File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)

And in the logs:

MacBook-Pro:streamlit-fastapi-model-serving robin$ docker-compose up
Starting fastapi ... done
Starting streamlit ... done
Attaching to fastapi, streamlit
streamlit    | 
streamlit    |   You can now view your Streamlit app in your browser.
streamlit    | 
streamlit    |   Network URL: http://172.18.0.3:8501
streamlit    |   External URL: http://2.26.108.101:8501
streamlit    | 
fastapi      | Using cache found in /root/.cache/torch/hub/pytorch_vision_v0.6.0
fastapi      | INFO:     Started server process [1]
fastapi      | INFO:     Waiting for application startup.
fastapi      | INFO:     Application startup complete.
fastapi      | INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
fastapi exited with code 137

Have confirmed api is on 8000.
Possibly I need to assign more memory to the container?
Cheers

url is hardcoded to http://fastapi:8000 but how to handle not using docker compose?

The current approach of hardcoding http://fastapi:8000 requires the use of docker compose, but how do we also handle the case that users are running the containers seperately? A situation where this occurs might be if you want to run the pytorch model on a dedicate GPU machine, and have this resource shared by multiple services (e.g.a web UI, some analytics tools). One option I used here is to pass the machine IP in via an environment variable like:

    environment: 
      - DEEPSTACK_IP=deepstack

Anyway I raise this in the hope you have an even better suggestion. It would be great to identify the optimum solutions and share this back with the community :-)

streamlit exited with code 1 due to TypeError in python3.7/site-packages/google/protobuf/descriptor.py

Thanks for the example of how to tie up FastAPI and streamlit in docker.

On two of my machines streamlit exited with code 1, as mentioned in the title.
The full trace is pasted below.
The solution I found was to upgrade the packages in streamlit/requirements.txt to their most recent versions:

-- original requirements.txt ---

streamlit==1.9.0 
requests==2.24.0
requests-toolbelt==0.9.1

-- modified requirements.txt ---

streamlit==1.10.0 
requests==2.28.1
requests-toolbelt==0.9.1

--- Screen output---

~/Downloads/streamlit-fastapi$ docker-compose up
Creating network "streamlit-fastapi_deploy_network" with driver "bridge"
Creating fastapi ... done
Creating streamlit ... done
Attaching to fastapi, streamlit
streamlit    | Traceback (most recent call last):
streamlit    |   File "/usr/local/bin/streamlit", line 5, in <module>
streamlit    |     from streamlit.cli import main
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/__init__.py", line 48, in <module>
streamlit    |     from streamlit.proto.RootContainer_pb2 import RootContainer
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/proto/RootContainer_pb2.py", line 37, in <module>
streamlit    |     create_key=_descriptor._internal_create_key),
streamlit    |   File "/usr/local/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 755, in __new__
streamlit    |     _message.Message._CheckCalledFromGeneratedFile()
streamlit    | TypeError: Descriptors cannot not be created directly.
streamlit    | If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
streamlit    | If you cannot immediately regenerate your protos, some other possible workarounds are:
streamlit    |  1. Downgrade the protobuf package to 3.20.x or lower.
streamlit    |  2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
streamlit    | 
streamlit    | More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
streamlit exited with code 1
fastapi      | Downloading: "https://github.com/pytorch/vision/archive/v0.6.0.zip" to /root/.cache/torch/hub/v0.6.0.zip
fastapi      | Downloading: "https://download.pytorch.org/models/resnet101-5d3b4d8f.pth" to /root/.cache/torch/hub/checkpoints/resnet101-5d3b4d8f.pth
100.0%i      | 
fastapi      | Downloading: "https://download.pytorch.org/models/deeplabv3_resnet101_coco-586e9e4e.pth" to /root/.cache/torch/hub/checkpoints/deeplabv3_resnet101_coco-586e9e4e.pth
100.0%i      | 
fastapi      | INFO:     Started server process [1]
fastapi      | INFO:     Waiting for application startup.
fastapi      | INFO:     Application startup complete.
fastapi      | INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
fastapi      | INFO:     172.19.0.1:51794 - "GET / HTTP/1.1" 404 Not Found

How to attach the dockhero url to the heroku app url

The approach described using dockhero to deploy leads to a separate dockhero url for the streamlit and fastapi apps, which maybe accessed at the respective ports: 8501 and 8000.

I have three questions:

Is there any way to access the streamlit side of the app using the Heroku url? I may be missing something here, but I have only been able to access it from dockhero.
I would like the fastapi side of the app to only be exposed to the streamlit side of the app. Is there a way to implement this?
More broadly, have you discovered a way to deploy the two docker images to the Heroku docker container, while allowing the streamlit to access the get or post outputs internally without exposing the fastapi app to the external web?

Streamlit UI throws error if no image is uploaded

If no image is uploaded and segmentation is requested, the UI shows:

This case should be handled more cleanly with a clearer message.

Do not store a temporary file

Great article and tutorial @davidefiocco!

Just a tiny detail but I'm pretty sure you can just send the binary data from the image using https://pillow.readthedocs.io/en/latest/reference/Image.html#PIL.Image.Image.tobytes and a raw starlette.Response

Would love to read more about those subjects in the future!

Streamlit requests may fail if FastAPI backend hasn't initialized yet

As the app is started with docker-compose up, the streamlit service is likely the fastest to come online (the fastapi service needs to do the PyTorch model load). If the user visits the streamlit page and fires a call, the result may be

because the fastapi service is not responding yet.

The user should just wait a bit and then results should be returned OK, but it would be better to implement some sort of healthcheck so that streamlit requests can be handled more gracefully.

"FileNotFoundError: [Errno 2] No such file or directory: 'sudo': 'sudo'" on Docker build

Hi,

I try to test yout repo, but there's an error on the docker build :

Creating fastapi ... done
Creating streamlit ... done
Attaching to fastapi, streamlit
streamlit    | Generate machine-id
streamlit    | Traceback (most recent call last):
streamlit    |   File "/usr/local/bin/streamlit", line 5, in <module>
streamlit    |     from streamlit.cli import main
streamlit    |   File "/usr/local/lib/python3.7/site-packages/streamlit/__init__.py", line 74, in <module>
streamlit    |     subprocess.run(["sudo", "dbus-uuidgen", "--ensure"])
streamlit    |   File "/usr/local/lib/python3.7/subprocess.py", line 488, in run
streamlit    |     with Popen(*popenargs, **kwargs) as process:
streamlit    |   File "/usr/local/lib/python3.7/subprocess.py", line 800, in __init__
streamlit    |     restore_signals, start_new_session)
streamlit    |   File "/usr/local/lib/python3.7/subprocess.py", line 1567, in _execute_child
streamlit    |     raise child_exception_type(errno_num, err_msg, err_filename)
streamlit    | FileNotFoundError: [Errno 2] No such file or directory: 'sudo': 'sudo'

have you an idea please ?
Thanks

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

davidefiocco / streamlit-fastapi-model-serving Goto Github PK