You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was working on a Rust rewrite of OCDS Merge (need to add a CLI and compare performance to the Python version). Once that's done, I can also explore replacing Kingfisher Process (that would mean using OCDS Kit to upgrade the few collections that are OCDS 1.0 – if that's too slow, I can write a Rust version).
Having fewer items in the queues should increase the stability of RabbitMQ. (If RabbitMQ continues to fall over, maybe we switch to Redis, where enabling its durability features won't be as much of a performance concern, as we'll just have 1 message per job task per publication).
The main reason for a Rust version, then, is that for large collections, we end up using a lot of CPU (triggering alerts), with RabbitMQ queueing over 1M messages in the release compiler. A Rust version means we use less CPU and finish faster.
I was working on a Rust rewrite of OCDS Merge (need to add a CLI and compare performance to the Python version). Once that's done, I can also explore replacing Kingfisher Process (that would mean using OCDS Kit to upgrade the few collections that are OCDS 1.0 – if that's too slow, I can write a Rust version).
Having fewer items in the queues should increase the stability of RabbitMQ. (If RabbitMQ continues to fall over, maybe we switch to Redis, where enabling its durability features won't be as much of a performance concern, as we'll just have 1 message per job task per publication).
Similar issue for data support server: open-contracting/deploy#437
The text was updated successfully, but these errors were encountered: