OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

How should a flask application running on gunicorn handle cpu intensive tasks?

  • Thread starter Thread starter user14530855
  • Start date Start date
U

user14530855

Guest
My flask app runs on gunicorn that has 1 worker and 8 threads. At startup the application reads a few million lines of JSON to load into an in-memory database. To do this, I spawn a gevent that does the read. I yield several times (I use gevent.sleep(0.1) with the monkey patch) and in the data-read thread but the kube pod still misses answering liveness and readyness probes enough times to get killed by kubernetes.

I tried using 2 (and more) workers. That solved my probe issue since a worker was always available for them. But every worker also now launches the read event, and I have multiple gevents trying to do the same thing.

Can I "pin" a gevent to a worker? Like only worker #0 does this event? Should I use gthreads/thread pools instead? I'm not terribly familiar with gunicorn so I'm sorry if this is a silly question.
<p>My flask app runs on gunicorn that has 1 worker and 8 threads. At startup the application reads a few million lines of JSON to load into an in-memory database. To do this, I spawn a gevent that does the read. I yield several times (I use <code>gevent.sleep(0.1)</code> with the monkey patch) and in the data-read thread but the kube pod still misses answering liveness and readyness probes enough times to get killed by kubernetes.</p>
<p>I tried using 2 (and more) workers. That solved my probe issue since a worker was always available for them. But every worker also now launches the read event, and I have multiple gevents trying to do the same thing.</p>
<p>Can I "pin" a gevent to a worker? Like only worker #0 does this event? Should I use gthreads/thread pools instead? I'm not terribly familiar with gunicorn so I'm sorry if this is a silly question.</p>
 

Latest posts

A
Replies
0
Views
1
Ashrik Ahamed
A
A
Replies
0
Views
1
Ashrik Ahamed
A

Online statistics

Members online
0
Guests online
4
Total visitors
4
Top