[GSoC] Develop a caching library for etcd

[serathius]

Submitted as project as part of Google Summer of Code with @MadhavJivrajani as second mentor.

While etcd is a powerful distributed key-value store, building scalable infrastructure management systems directly on top of it can be challenging. Kubernetes has demonstrated the effectiveness of the reconciliation pattern for managing complex deployments, and its watch cache plays a crucial role in achieving scalability. However, this crucial caching mechanism is tightly coupled with Kubernetes and not readily available for general etcd usage. Projects like Cilium and Calico Typha, while successfully using etcd for control planes, have had to implement custom solutions to address this gap.

This project addresses the need for a standardized, performant caching solution for etcd, enabling easier adoption of the reconciliation pattern and simplifying the development of scalable etcd-based systems. By providing a generic watch cache implementation, we aim to lower the barrier to entry for building robust and efficient infrastructure management tools on etcd.

Goals:

• Develop a generic proxy that provides feature parity to K8s watch cache, allowing in future possibility of replacing K8s builtin library.

Milestones:

• Cache for watch requests, stores history of watch events and demultiplexes requests.
• Cache for non-consistent list requests, stores latest state of etcd in Btree cache. Cache is fed by Range response, that is later updated by subscribing to updates from Watch Cache.
• Handling requests during cache initialization and re-intialization.
• Testing, including e2e and robustness tests
• Metrics for cache size, cache latency etc
• Benchmarks for watch and read throughput.
• Support for custom encoder/decoder
• Support for custom indexing
• Support for consistent reads
• Support for exact stale reads, by storing snapshots of btree.

I'm proposing to locate the project within the etcd mono repo, but as a separate package, that will not be released/tagged until it's ready. Proposed package name: go.etcd.io/cache. Client library would be developed under go.etcd.io/cache/client.

/cc @fuweid @MadhavJivrajani @ahrtr @henrybear327

[ahrtr]

High level I agree with the improvement & direction, as performance should be one of the key areas that we should spend more effort on. It will definitely ensure the long-term success of etcd.

[fuweid]

We need to develop a generic cache for etcd, that allows users to easily addopt multi layered caching architecture similar to K8s. Having an official library would allow us to properly test it ensuring it's correctness and performance.

Sounds great. It could be more efficient to make and evaluate changes as an official library. +1 for help, if need.

[serathius]

cc @ahrtr @ivanvc any preference where development should happen. My proposal:

I'm proposing to locate the project within the etcd mono repo, but as a separate package, that will not be released/tagged until it's ready. Proposed package name: go.etcd.io/cache. Client library would be developed under go.etcd.io/cache/client.

[ahrtr]

I'm proposing to locate the project within the etcd mono repo, but as a separate package

It should be OK.

go.etcd.io/cache/client

I think all packages in the etcd mono repo should have the same prefix go.etcd.io/etcd/. Also is the cache dedicated for the watch scenario, or potentially be for other cases as well? Could you provide more context or details before we make any detailed decision?

[serathius]

I think all packages in the etcd mono repo should have the same prefix go.etcd.io/etcd/

Ok, don't think it should be a problem.

I expect that on top level of hierarchy we will want client cache, and standalone cache server (like a grpc proxy but based on new cache library with configurable caching, covering all Range types and with proper guarantees). Within the client cache we will have separate watch de-multiplexer and cache for range requests.

[ahrtr]

I expect that on top level of hierarchy we will want client cache, and standalone cache server (like a grpc proxy but based on new cache library with configurable caching, covering all Range types and with proper guarantees). Within the client cache we will have separate watch de-multiplexer and cache for range requests.

Can we have a spec & design doc for these?

[serathius]

No, I was just providing more context. Using go.etcd.io/etcd/cache package should be ok.

[abdurrehman107]

This sounds exciting, and I’d love to take it up as part of Google Summer of Code. The idea of a standardized caching solution for etcd is impactful and I'd love to implement this as my project.

I'm currently exploring how we've implemented caching in k8s and I look forward to mirroring something similar for etcd in this project. Looking forward to the opportunity to contribute and collaborate with everyone on this.

[mutokrm]

Hi, I have a question!

It seems like the b-tree structure in api-server has recently been introduced. Can I ask what's been encouraging the community to strengthen the etcd caching logic?

My intention is to know if there were specific team goals behind the recent activities on api-server and this proposal :)