Cache with no datasource, contrived or ok?

[steveoh]

I have a need to seed an L1/L2 cache from a BigQuery table. BigQuery has restrictive transactional quotas since it's an analytics database so acting as a "normal" data source for a cache is not a good fit.

I white boarded some ideas and I'd like to share them to see if fusion is a good match and if there is a clean path forward and possibly one that is not uncommon. I'm excited to replace my current implementation to get all the wonderful features of fusion including stampede protection.

I was thinking about using a dotnet background service to query BigQuery on startup and seed the L1 and L2 memorystore/redis cache. This would involve inserting < 5000 items. This data is necessary for the application to function and refreshes weekly.

Every request to the api will take a piece of user input and look for an item in the cache. There are occasions where the input is not going to exist and should be set to empty. This is analogous to checking the spelling of a word in a dictionary and if the input is incorrect the app won't have an answer for it. Therefore there is no reason to query the datasource as it won't have an answer.

In certain scenarios when a cache miss happens, we want to get all the keys, which I assume we will duplicate in one cache entry for ease of access, and perform a Levenshtein distance to the values and add a new cache entry if we find one that matches closely. I assumed we could do that in the code of the cache.GetOrSetAsync method by accessing the L1 or L2 cache values?

Matches to the cache will simply follow the standard route and return the data associated.

Let me know if this makes sense and I can clear up any confusion. I'd like to confirm this is a good candidate for fusion.

As I write this out, it's starting to sound more like dynamic/remote configuration than something that requires a hybrid cache. 🤔

[jodydonetti]

Hi @steveoh , thanks for sharing your scenario!

I have a need to seed an L1/L2 cache from a BigQuery table. BigQuery has restrictive transactional quotas since it's an analytics database so acting as a "normal" data source for a cache is not a good fit.

I white boarded some ideas and I'd like to share them to see if fusion is a good match and if there is a clean path forward and possibly one that is not uncommon. I'm excited to replace my current implementation to get all the wonderful features of fusion including stampede protection.

Let's see what I can do 🙂

I was thinking about using a dotnet background service to query BigQuery on startup and seed the L1 and L2 memorystore/redis cache. This would involve inserting < 5000 items. This data is necessary for the application to function and refreshes weekly.

First thing to notice is that I suppose this bg service would run in the same app that will use FusionCache.
If this is the case I'm thinking about 2 possible things:

• at every restart of the app, you'll be querying BigQuery to repopulate it, which is a waste of resources. If the call is done via GetOrSet() then no big deal, but then I'm wondering why the extra bg service and not just the normal usage
• if you scale horizontally (eg: multiple nodes) the same issue will be even worse, because 10 instances = 10 initial massive load

Can you help me understand this part better?

Every request to the api will take a piece of user input and look for an item in the cache. There are occasions where the input is not going to exist and should be set to empty. This is analogous to checking the spelling of a word in a dictionary and if the input is incorrect the app won't have an answer for it. Therefore there is no reason to query the datasource as it won't have an answer.

Ok, for this you have 2 ways:

• if you want to SET an empty value in the cache, you can call GetOrSet() and, in the factory, just return the empty value
• if you only want to GET the value and have back empty without setting it, you can call GetOrDefault() (you can even specify which default value to use, or it will fall back to default(T) automatically)

One of these 2 approaches should cover this part.

In certain scenarios when a cache miss happens, we want to get all the keys, which I assume we will duplicate in one cache entry for ease of access, and perform a Levenshtein distance to the values and add a new cache entry if we find one that matches closely. I assumed we could do that in the code of the cache.GetOrSetAsync method by accessing the L1 or L2 cache values?

Correct.
Just avoid having a single cache entry that is too big (don't have any specific number, and it also depends on the specific distributed cache, but still): for example with Redis there's the famous Conveyor Belt Problem, where one big command will block other small commands from executing fast.
Apart from this, looks good.

Let me know if this makes sense and I can clear up any confusion. I'd like to confirm this is a good candidate for fusion.

For my understanding of the scenario: yes, it looks good!
If you'll follow along let me know how it goes, in case of issues I can try to help.

As I write this out, it's starting to sound more like dynamic/remote configuration than something that requires a hybrid cache. 🤔

Eh, there are scenarios that are not necessarily 100% this or 100% that, but more like middle grounds.
Also, caching is not something strictly tied to a particular scenario, but more like a tool that you can use in many of them: a lot of people I know are using caching to access rate-limited APIs (eg: Stripe, etc) so what you are thinking about is definitely something not esoteric.

Hope this helps!

[steveoh]

This is what I came up with if you're interested. https://github.com/agrc/api.mapserv.utah.gov/pull/452/files

[jodydonetti]

Hi @steveoh , I'm looking at it right now.

One quick thing I noticed is that in the "places" cache setup you are doing this:

Duration = TimeSpan.FromDays(7),

FailSafeMaxDuration = TimeSpan.FromHours(1),
FailSafeThrottleDuration = TimeSpan.FromSeconds(30),

This is saying "cache for 7 days, and in case of problems during reresh cache it again as state (expired) for 30 seconds at a time, until the total duration in the cache is at most 1 hour": the fail-safe max duration is the total duration, meaning "how long you want something to be in cache, in total, at most", and is not the part that gets added "at the end" of the normal Duration.

In any case FusionCache takes care of normalizing incoherent options to avoid problems, and in doing so it emits a log message such as this:

"FUSION [N={CacheName} I={CacheInstanceId}] (O={CacheOperationId} K={CacheKey}): FailSafeMaxDuration {FailSafeMaxDuration} was lower than the Duration {Duration} on {Options}. Duration has been used instead."

If you enable logging and look for it you should find it.

Hope this helps.

[jodydonetti]

Btw the use case looks cool!
Is it being used in production?

[steveoh]

Ok, so I'll need to do 7 days + 1 hour. I'm working to push it to staging now.