FAI-15182 | CircleCI optimization for tests stream API calls #1928
Conversation
dbruno21
force-pushed
the
db/fai-15182
branch
from
dbe82e1 to
b33edcd
Compare
|
@chalenge wdyt? For better PR readability, hide whitespace from diff |
dbruno21
requested review from
cjwooo,
ypc-faros and
willmarks
as code owners
| } | ||
| } | ||
|
|
||
| getUpdatedState( |
There was a problem hiding this comment.
On the connector version using memoize, the memoized values corresponds to the pipelines stream state (the one that was used for making the API calls 1 2). So even if it's needed it wasn't being used.
In theory on a successful sync, this state would end up matching with the one saved on the pipelines stream. The only problem is that if a sync breaks before finishing the tests stream. The state for pipelines stream will be already saved, losing tests records. Not sure how to handle that case. I decided to do this instead of combining the streams, because it seemed faster and easier, but it has this problem (that also happens with the memoized version)
|
Description
Prevent calling the same API calls for
pipelinesandtestsstreams (both call pipelines -> workflows -> jobs). Memoization was removed on #1918 to prevent OOM issues. It was thought to not be used, but it was useful after all (memoize by default uses only the first function param as key).This change stores, when the
pipelinesstream runs, just the necessary jobs data to be used later on thetestsstream.Type of change
Related issues
Migration notes
Extra info