How can you use schemas in this project with celery about flask-restplus-server-example HOT 4 CLOSED

I design my services so every piece of data is owned by only one application. This means that either scheduled tasks own their own data in their own DB (well, the DB server can be shared, but services use separate databases), OR scheduled tasks make queries to the API server, OR introduce a separate "internal" API service, so the public API server and the scheduled tasks operate via this internal API service.

I have had the experience implementing API server and scheduled tasks in a single code base, and, mind you, you don't want to make that mistake; that goes cumbersome really quickly especially with Celery [I also strongly recommend avoid Celery; take a look at Dramatiq or Kuyruk].

I usually go with the second option (a single API server completely owns the data, and I make regular API calls from the scheduled tasks using an "internal" role for that).

P.S. I close this issue to mark it as answered, but feel free to leave your comments, and let us discuss the topic if necessary.

from flask-restplus-server-example.

Thanks for the quick answer
I use Celery because I need to schedule tasks every 15 minutes, and to have a simple dashboard (flower does it perfectly) to know if a task has failed etc... I m not sure Kuyruk does this, it looks more like a distributed task manager. Right?

Having an "internal" API service may be an acceptable solution, I think I will go for it.

THank you

from flask-restplus-server-example.

it looks more like a distributed task manager. Right?

You are right. It is just like Celery in that respect. If you are just running some regular tasks, you may just as well run a thread inside your Flask application and do the job using something like https://github.com/dbader/schedule.

from flask-restplus-server-example.

Thanks for the advice.
I don't think it is good to mix businesses. Schedule job has nothing to do on a web Rest API for me and should be done aside.
But as you said, I will call the "internal" api from celery. And thanks to flower I will have all my tasks status and logs accessible easily.

Thank you for the help.

from flask-restplus-server-example.