You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have an idea that consists on triggering an Argo workflow that spins up a pod to run a Celery task every time Celery distributes a queue through a Redis broker. That way I could capture the task we want to run and the parameters required and run a pod that runs a task until it is completed rather than having Celery workers running permanently.
So far, the only way I can configure Redis event sources is through streams or channels which I don't think is the way Celery distributes the queues.
I don't know whether there is a simple way to watch Redis broker events or if the right way to do this would be to replace all Celery functionality with Argo Events (Websockets & Cronjob) and Workflows (which it would require more effort and collaboration).
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
I have an idea that consists on triggering an Argo workflow that spins up a pod to run a Celery task every time Celery distributes a queue through a Redis broker. That way I could capture the task we want to run and the parameters required and run a pod that runs a task until it is completed rather than having Celery workers running permanently.
So far, the only way I can configure Redis event sources is through streams or channels which I don't think is the way Celery distributes the queues.
I don't know whether there is a simple way to watch Redis broker events or if the right way to do this would be to replace all Celery functionality with Argo Events (Websockets & Cronjob) and Workflows (which it would require more effort and collaboration).
Beta Was this translation helpful? Give feedback.
All reactions