-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
documentationImprovements or additions to documentationImprovements or additions to documentation
Description
Curifactory has no way internally of dealing with partial stage cache hits (e.g. one thing in cache for a stage is found but not another)
A way to deal with this manually is through record.stage_cachers:
@cf.stage(None, ["thing1", "thing2"], [PickleCacher]*2)
def make_things(record):
# in the case where thing1 is in cache but thing2 is not, the whole stage will re-run
# if we know this is a possibility, and maybe thing1 takes a very long time to compute, we could
# manually handle this inside the stage (assuming it is not cleaner to separate out thing1 and thing2 production
# into separate stages)
if record.stage_cachers[0].check():
thing1 = record.stage_cachers[0].load()
else:
# create thing1 the normal way
...
# create thing2
...
return thing1, thing2Metadata
Metadata
Assignees
Labels
documentationImprovements or additions to documentationImprovements or additions to documentation