Use python-RQ to distribute jobs

[matteocao]

Is your feature request related to a problem? Please describe.

The current basic way of distributing jobs in K8 is not effective. we need automation
Describe the solution you'd like

The solution would be to use python-rq , deploy many workers on K8, and every time one wants to crush heavy HPOs, simply enqueue the jobs and let the workers work.

This requires also a mysql DB so that optima can keep track of the progress and properly distribute the computations.
Describe alternatives you've considered

Using kubctl and volumes, but it creates race conditions easily.
Additional context