-
Notifications
You must be signed in to change notification settings - Fork 108
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
redis driver for blob cache information and metadb (#2865)
* feat: add redis cache support #2005 Fixes #2004 * feat: add redis cache support Currently, we have dynamoDB as the remote shared cache but ideal only for the cloud use case. For on-prem use case, add support for redis. Signed-off-by: Ramkumar Chinchani <[email protected]> * feat(redis): added blackbox tests for redis Signed-off-by: Petu Eusebiu <[email protected]> * feat(redis): dummy implementation of MetaDB interface for redis cache Signed-off-by: Alexei Dodon <[email protected]> * feat: check validity of driver configuration on metadb instantiation Signed-off-by: Andrei Aaron <[email protected]> * feat: multiple fixes for redis cache driver implementation - add missing method GetAllBlobs - add redis cache tests, with and without mocking Signed-off-by: Andrei Aaron <[email protected]> * feat(redis): redis implementation for MetaDB Signed-off-by: Andrei Aaron <[email protected]> * feat(redis): use redsync to block concurrent write access to the redis DB Signed-off-by: Andrei Aaron <[email protected]> * feat(redis): update .github/workflows/cluster.yaml to also test redis Signed-off-by: Andrei Aaron <[email protected]> * feat(metadb): add keyPrefix parameter for redis and remove unneeded method meta.Crate() Signed-off-by: Andrei Aaron <[email protected]> * feat(redis): support RedisCluster configuration and add unit tests Signed-off-by: Andrei Aaron <[email protected]> * feat(redis): more tests for redis metadb implementation Signed-off-by: Andrei Aaron <[email protected]> * feat(redis): add more examples and update examples/README.md Signed-off-by: Andrei Aaron <[email protected]> * feat(redis): move option parsing and redis client initialization under pkg/api/config/redis Signed-off-by: Andrei Aaron <[email protected]> * chore(cachedb): move Cache interface to pkg/storage/types Signed-off-by: Andrei Aaron <[email protected]> * feat(redis): reorganize code in pkg/storage/cache.go Signed-off-by: Andrei Aaron <[email protected]> * feat(redis): call redis.SetLogger() with the zot logger as parameter Signed-off-by: Andrei Aaron <[email protected]> * feat(redis): rename pkg/meta/redisdb to pkg/meta/redis Signed-off-by: Andrei Aaron <[email protected]> --------- Signed-off-by: Ramkumar Chinchani <[email protected]> Signed-off-by: Petu Eusebiu <[email protected]> Signed-off-by: Alexei Dodon <[email protected]> Signed-off-by: Andrei Aaron <[email protected]> Co-authored-by: a <[email protected]> Co-authored-by: Ramkumar Chinchani <[email protected]> Co-authored-by: Petu Eusebiu <[email protected]> Co-authored-by: Alexei Dodon <[email protected]>
- Loading branch information
5 people
authored
Jan 30, 2025
1 parent
90e1393
commit 05823cd
Showing
43 changed files
with
7,878 additions
and
434 deletions.
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| { | ||
| "distSpecVersion": "1.1.0", | ||
| "storage": { | ||
| "dedupe": true, | ||
| "gc": true, | ||
| "rootDirectory": "/tmp/zot", | ||
| "cacheDriver": { | ||
| "name": "redis", | ||
| "keyprefix": "zot", | ||
| "addr": ["host1:6379", "host2:6379", "host3:6379"], | ||
| "client_name": "client", | ||
| "db": 1, | ||
| "protocol": 3, | ||
| "username": "user1", | ||
| "password": "pass1", | ||
| "sentinel_username": "user2", | ||
| "sentinel_password": "pass2", | ||
| "max_retries": 3, | ||
| "min_retry_backoff": "5s", | ||
| "max_retry_backoff": "5s", | ||
| "dial_timeout": "5s", | ||
| "read_timeout": "5s", | ||
| "write_timeout": "5s", | ||
| "context_timeout_enabled": false, | ||
| "pool_fifo": false, | ||
| "pool_size": 3, | ||
| "pool_timeout": "5s", | ||
| "min_idle_conns": 1, | ||
| "max_idle_conns": 3, | ||
| "max_active_conns": 3, | ||
| "conn_max_idle_time": "5s", | ||
| "conn_max_lifetime": "5s", | ||
| "max_redirects": 3, | ||
| "read_only": false, | ||
| "route_by_latency": false, | ||
| "route_randomly": false, | ||
| "master_name": "zotmeta", | ||
| "disable_identity": false, | ||
| "identity_suffix": "zotmeta", | ||
| "unstable_resp3": false | ||
| }, | ||
| "storageDriver": { | ||
| "name": "s3", | ||
| "rootdirectory": "/zot", | ||
| "region": "us-east-2", | ||
| "regionendpoint": "localhost:4566", | ||
| "bucket": "zot-storage", | ||
| "forcepathstyle": true, | ||
| "secure": false, | ||
| "skipverify": false | ||
| } | ||
| }, | ||
| "http": { | ||
| "address": "0.0.0.0", | ||
| "port": "8484" | ||
| }, | ||
| "log": { | ||
| "level": "debug" | ||
| }, | ||
| "extensions": { | ||
| "ui": { | ||
| "enable": true | ||
| }, | ||
| "search": { | ||
| "enable": true | ||
| } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| { | ||
| "distSpecVersion": "1.1.0", | ||
| "storage": { | ||
| "dedupe": true, | ||
| "gc": true, | ||
| "rootDirectory": "/tmp/zot", | ||
| "cacheDriver": { | ||
| "name": "redis", | ||
| "url": "redis://user:password@host1:6379?dial_timeout=3&read_timeout=6s&addr=host2:6379&addr=host3:6379", | ||
| "keyprefix": "zot" | ||
| }, | ||
| "storageDriver": { | ||
| "name": "s3", | ||
| "rootdirectory": "/zot", | ||
| "region": "us-east-2", | ||
| "regionendpoint": "localhost:4566", | ||
| "bucket": "zot-storage", | ||
| "forcepathstyle": true, | ||
| "secure": false, | ||
| "skipverify": false | ||
| } | ||
| }, | ||
| "http": { | ||
| "address": "0.0.0.0", | ||
| "port": "8484" | ||
| }, | ||
| "log": { | ||
| "level": "debug" | ||
| }, | ||
| "extensions": { | ||
| "ui": { | ||
| "enable": true | ||
| }, | ||
| "search": { | ||
| "enable": true | ||
| } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| { | ||
| "distSpecVersion": "1.1.0", | ||
| "storage": { | ||
| "dedupe": true, | ||
| "gc": true, | ||
| "rootDirectory": "/tmp/zot", | ||
| "cacheDriver": { | ||
| "name": "redis", | ||
| "url": "redis://localhost:6379", | ||
| "keyprefix": "zot" | ||
| }, | ||
| "storageDriver": { | ||
| "name": "s3", | ||
| "rootdirectory": "/zot", | ||
| "region": "us-east-2", | ||
| "regionendpoint": "localhost:4566", | ||
| "bucket": "zot-storage", | ||
| "forcepathstyle": true, | ||
| "secure": false, | ||
| "skipverify": false | ||
| } | ||
| }, | ||
| "http": { | ||
| "address": "0.0.0.0", | ||
| "port": "8484" | ||
| }, | ||
| "log": { | ||
| "level": "debug" | ||
| }, | ||
| "extensions": { | ||
| "ui": { | ||
| "enable": true | ||
| }, | ||
| "search": { | ||
| "enable": true | ||
| } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.