How does HybridCache in .NET 9 affect FusionCache?

[davidhenley]

How does the new HybridCache in .NET 9 affect FusionCache?

Will FusionCache leverage the new HybridCache? What does FusionCache offer on top of HybridCache?

[jodydonetti]

Hi @davidhenley good question, and one that I'm receiving from a lot of people, publicly and in private.

I'll try to articulate here my thoughts, including a small intro about each one for the people that don't know them well already.

Hope this helps.

FusionCache

FusionCache is a hybrid cache that has been battle-tested in production for years.

It has a ton of features that allows it to be quite flexible, like adaptive caching, conditional refresh, named caches and more.

I think it goes beyond a "normal" hybrid cache in that it also includes a lot of advanced resiliency features like fail-safe, soft/hard timeouts, auto-recovery and others.

Having lived on my skin for years and years how complex a multi-level cache can be, and what a pain it can be to handle transient issues in production, even more so with distributed systems composed of a lot of moving pieces that can intermittently fail at any time, I decided to put that experience into FusionCache so others would not have to handle them manually.

In general, the goal behind FusionCache is to have a project that protect from cache stampede, transparently handles L1 or L1+L2 caching (the hybrid part), is fast, flexible and automatically manages the resiliency issues mentioned above, all out of the box in a (hopefully) "it just works" way.

I also wanted to create an easy to grasp, accessible, thorough and nice looking documentation, since I think that's something both really helpful and frequently overlooked. Also, it is my first OSS project so I just wanted to "do it well".

Overall its packages have ben downloaded 5M+ times, it got me a couple of awards (Google and Microsoft) but mostly, based on what people tell me, it is working quite well for them so I am definitely happy about it 🙂

It is also being used in a lot of projects out in the wild, both OSS, private and commercial, even including some products from Microsoft itself like Data API Builder, where it is used as the caching engine, which is definitely nice.

I'll continue working on FusionCache to add new features, optimize it more and I'll keep listening to the community for requests.

HybridCache

This is the Microsoft version of an L1/L1+L2 cache with stampede protection.

The team working on HybridCache has top notch people in it, the lead is Marc Gravell and, I mean: StackExchange, Dapper, Protobuf-net and so on, should I say more?

I'm collaborating with the team by sharing my experience with caching in general, FusionCache in particular, some ideas, gotchas, potential traps, suggestions, naming (I suggested the HybridCache name) and so on, hoping it can be of some help. The way I'm trying to help is usually via all-in-one mega-comments like this: they seem to be reading them and sometimes they answer, sometimes not (or not yet).

A nice thing about this whole HybridCache effort is that it will not be only an implementation: it may become a new shared abstraction for caching in .net, like IDistributedCache already is. It would basically define the base features and api surface area, so that others may implement it differently.

Will they be able to pull this off? I don't know yet, but the pieces of the puzzle seem to be aligning, so let's hope.

Note

the team choose an abstract class instead of an interface for HybridCache to be better positioned to extend it in the future without breaking backward compatibility. Another option was default interface methods, but it felt not stable enough yet (UPDATE after Marc's note) it was not possible since that feature requires runtime support and the team want to support down-level TFMs.

Differences

Currently HybridCache is not out yet, so this is based on what is currently known: in the end it may contain less features or more, we don't know yet.

Common features (both FusionCache and HybridCache):

• Cache Stampede protection
• L1 only (memory)
• L1+L2 (memory + distributed)
• multi-node notifications (in FusionCache it is the Backplane)
• Tagging

FusionCache-only features:

• Fail-Safe
• Soft/Hard Timeouts
• Adaptive Caching
• Conditional Refresh
• Eager Refresh
• Auto-Recovery
• Multiple Named Caches
• Advanced Logging
• Events
• Background Distributed Operations
• OpenTelemetry (not sure about this, maybe HybridCache will have this, too)
• All methods available as both sync+async (HybridCache will be async-only)

HybridCache-only features (kind of, see below for more):

• Compression

Regarding eviction by tags: I'm experimenting about it from some time now, but it's a complex topic to get right, and I have some doubts about the current design, doubts that I highlighted here.

🔔 UPDATE 2024-10-20: Evict By Tags is happening!
🔔 UPDATE 2025-01-20: FusionCache v2 is out, including Tagging, Clear and native HybridCache support!

Regarding compression, I'm also experimenting on that, but with a slightly different approach which I'll highlight as soon as the pieces will be in a good shape.

There are also minor differences, like the fact that HybridCache will allow different serializers to be used for specific cache keys while FusionCache does not: on the other hand FusionCache supports multiple named caches (while HybridCache does not) and allows for each named cache to use different serializers, which I feel is overall a more robust/easier to use design. But still, minor things.

FusionCache + HybridCache

FusionCache will not leverage the new HybridCache, in the sense that it will not be built "on top of it", but it will do 2 main things:

1. it may leverage some of the new bits being created for it
2. will be available as an implementation of HybridCache

Let's expand on these a little bit.

New Bits

As an example, with FusionCache I had the need to come up with a generic "binary serializer" abstraction to allow users to serialize cache entries for the distributed cache in different formats.
Therefore I created an IFusionCacheSerializer interface, and then created various implementations based on common concrete serializers like Json.NET, System.Text.Json, Protobuf, MemoryPack and so on (you can see them listed here).

HybridCache of course has the same need, and the team come up with IHybridCacheSerializer which is essentially the same thing as above. Since this will be in the BCL, something I'll probably do is add support in FusionCache for these new abstraction (and so, all the related implementations) to widen the available supported components. Their interface is also potentially more performant by being based on IBufferWriter<byte> and friends, so that would be an extra bonus, which is also something I can add in FusionCache itself.

An alternative impl of HybridCache

As I stated above it's important to note that HybridCache will not just be an implementation, but an abstract class which acts as a new abstraction: think of it like the IDistributedCache interface, but for multi-level caches, so much so that the actual concrete class is called DefaultHybridCache if I remember correctly.

So there's an abstract HybridCache class that can be used as a dependency and anyone can implement, and a concrete DefaultHybridCache class which is the default implementation provided by Microsoft.

So the main idea for FusionCache is to also be available as an implementation of the abstract HybridCache class, for people that has a dependency on that.

FusionCache itself will keep being usable directly on its own as it is today, and by doing that people will get 100% of the features, both the ones in common with HybridCache (cache stampede protection, multi-level) and the extra ones only available in FusionCache (fail-safe, soft/hard timeouts, conditional refresh, auto-recovery, etc), and by doing this they'll have total control, both globally and granularly per-call.

But people will also be able to use FusionCache as an implementation of HybridCache (via an adapter class), so they can depend on a shared abstraction instead of FusionCache directly itself, if they like: the only FusionCache specific code will be at setup time (think Startup.cs or similar) where they will setup FusionCache and tell it to "make yourself available as an HybridCache", you can see a sneak peek of this here and some example results here.
On the other hand, by going down this path, they'll loose some granular control on some extra feature: a concrete example is that they will still be able to enable fail-safe or soft/hard timeout or auto-recovery globally at setup time, but they'll loose the ability to control these features per-call, since the methods signature of HybridCache is different/more limited and would not allow to do that.

Microsoft and OSS

There's a whole discussion going on from years now about the relation of Microsoft with the OSS community.

In regard to FusionCache, I can see MS coming out with this new higher level cache thing and people ignoring other 3rd party solutions, or maybe stifling innovation with a 100 tons common denominator abstraction (like in the DI/logging world) which... yeah, I mean that's what happened there, so there's definitely a risk here.
I can therefore see a potential future where any new feature I would add to FusionCache would need to be hidden behind this new abstraction because people would not like to depend on FusionCache as the abstraction, but only as an implementation on the HybridCache abstraction.

Is this a potential risk? Yes, we have to be honest about it.

But I can also see a value in having a shared abstraction that other components can depend on. I don't have a ready made answer for this that would solve this situation.

The team behing HybridCache (and Marc in particular) has been quite open in discussing it, took notes on what me and others are saying and even made some changes based on it. They took time off their schedule to read my uber long mega-comments (sorry again guys 😅) and sometimes answer those. I truly believe that they care about making things right this time regarding the relationship with the OSS community, and I know Marc in particular cares about this a lot even just because he's been there for years when he was at StackExchange, so he knows the potential struggle.

I personally think we (meaning the team, me and other OSS contributors/implementers) should talk a little bit more, to better define some details, smooth out the process and in general be ready for v1 in November (if that is still the case), but apart from that I honestly see they care so kudos for that.

So maybe, just maybe, here's what may happen:

• shared components like output cache, efcore 2nd level cache, etc that need to run inside a wide variety of people's apps and services will depend directly on the MS abstraction (abstract HybridCache class) so that different implementations like FusionCache can just be configured differently at startup and used in a less granular way, but would also work seamlessly
• people's apps and services can choose instead to depend directly on FusionCache or other specific impl, to take full advantage of their features, even granularly per-call

I don't know, maybe I'm dreaming, let's see 🤷

Let me know!

Does all of this make sense?

I'd like to know from the community, so please tell me!

Oh, and particularly I'd like @mgravell and the team to chip in and add any detail they'd like, or tell me if I said something wrong.

[giulianob]

But people will also be able to use FusionCache as an implementation of HybridCache (via an adapter class), so they can depend on a shared abstraction instead of FusionCache directly itself, if they like: the only FusionCache specific code will be at setup time (think Startup.cs or similar) where they will setup FusionCache and tell it to "make yourself available as an HybridCache", you can see a sneak peek of this here and some example results here.
On the other hand, by going down this path, they'll loose some granular control on some extra feature: a concrete example is that they will still be able to enable fail-safe or soft/hard timeout or auto-recovery globally at setup time, but they'll loose the ability to control these features per-call, since the methods signature of HybridCache is different/more limited and would not allow to do that.

I think this is a great option and makes me very happy. Hopefully the design of HybridCache is making alternative implementations of caches a goal. FusionCache will still be able to be used if some other libraries rely on HybridCache but in my app I can always use FusionCache directly for features missing in HybridCache.

[jodydonetti]

I think this is a great option and makes me very happy. Hopefully the design of HybridCache is making alternative implementations of caches a goal.

Yes, currently it seems so.
There were a couple of potential pitfalls, not necessarily in the api surface area but mostly in the default implementation, for example related to the way inheritance would work for DefaultEntryOptions, but probably that's not the case anymore.
In general I'd like to adhere to the expected behaviour (in this case of the default HybridCache implementation by MS), to avoid surprises for users.
We'll see more when we'll approach RC2 and GA 🤞

FusionCache will still be able to be used if some other libraries rely on HybridCache but in my app I can always use FusionCache directly for features missing in HybridCache.

Exactly, that's the goal!

[mgravell]

I genuinely appreciate the frank and open conversations here. We absolutely don't want to be "bad MSFT, stamping over all the 3rd party tools" - I think I have enough libraries behind my name for it to be obvious that I genuine care about 3rd party being strong. We also need to acknowledge that there exists a subset of users who for whatever reasons, misguided or not (corporate policies etc): want the "in box thing". So yes, there are similarities and overlaps - and differences - between the two, but I hope we are in the healthy region where we're simply offering additional choices.

A minor correction here re default interface methods:

but it felt not stable enough yet.

Since we have taken the decision to actively target down-level TFMs, default interface methods are not an option (they require runtime support).

[jodydonetti]

I genuinely appreciate the frank and open conversations here. We absolutely don't want to be "bad MSFT, stamping over all the 3rd party tools" - I think I have enough libraries behind my name for it to be obvious that I genuine care about 3rd party being strong. We also need to acknowledge that there exists a subset of users who for whatever reasons, misguided or not (corporate policies etc): want the "in box thing". So yes, there are similarities and overlaps - and differences - between the two, but I hope we are in the healthy region where we're simply offering additional choices.

Absolutely, and that is my sentiment too based on what I saw up until now: fingers crossed for the potential italian-english "lost in translation" thing here, I hope I've been able to communicate the same feeling.

A minor correction here re default interface methods:

but it felt not stable enough yet.

Since we have taken the decision to actively target down-level TFMs, default interface methods are not an option (they require runtime support).

Ah, good point! I was thinking mostly about the inability for a default interface method implementation to access private/protected members, which makes it less powerful and probably more suitable for new methods/overloads that calls existing ones.

[vanna798756]

Great discussion. When I found HybridCache my eye's lit up, it was something I had been wanting for a number of years. But the deal breaker for me was async only support. I initially attempted to move all the required bits to async, but after a massive job converting my system to .NET Core, this was not really feasible. That's when I found FusionCache and the sync support is a god send and killer feature.

[jodydonetti]

Hi @vanna798756 , sorry for the delay but I just came back from (very late) summer vacations 😅

I'm happy you are liking FusionCache, that's great!

[jodydonetti]

Hi all, v2.0.0-preview-3 is out 🥳

It includes the HybridCache integration I mentioned here: if you try it, please let me know!

[jodydonetti]

Hi all, I release preview-4, which most probably will be the last preview before going GA with FusionCache V2 🥳

If you try it, please let me know!

[jodydonetti]

Hi all, v2.0.0 is finally out 🎉