Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TT-11223]ERROR: Tyk PUMP not able to connect to Redis Sentinels #6002

Open
arshavardan opened this issue Jan 31, 2024 · 1 comment
Open
Labels

Comments

@arshavardan
Copy link

arshavardan commented Jan 31, 2024

Environment:
Kubernetes

Using Tyk Pro-helm/tyk-pro --version 0.14.0
using Bitnami/Redis-18.6.0 (Sentinels Enabled)
PUMP Configuration:
env:
# Legacy support for Redis Cluster driver. Driver dropped in v3.0.0.
- name: REDIGOCLUSTER_SHARDCOUNT
value: "128"
- name: TYK_PMP_ANALYTICSSTORAGECONFIG_DATABASE
value: "0"
- name: TYK_PMP_ANALYTICSSTORAGECONFIG_REDISUSESSL
value: "true"
- name: TYK_PMP_ANALYTICSSTORAGECONFIG_ADDRS
value: "redis-sentinels-headless.namespace.svc.cluster.local:26379"
- name: TYK_PMP_ANALYTICSSTORAGECONFIG_ENABLECLUSTER
value: "false"
- name: TYK_PMP_ANALYTICSSTORAGECONFIG_REDISSSLINSECURESKIPVERIFY
value: "true"
- name: TYK_PMP_ANALYTICSSTORAGECONFIG_PASSWORD
valueFrom:
secretKeyRef:
name: secrets-tyk-pro
key: redisPass
and pump.conf: | is default as per the charts
before we are using standalone redis, where we have given direct master pod DNS address, now we are giving redis-sentinels-headless service with port 26379.

Pump logs:
time="Jan 31 05:44:23" level=debug msg="Unpacked vals: 0" prefix=redis
time="Jan 31 05:44:23" level=debug msg="Getting raw key set: tyk-system-analytics_3_protobuf" prefix=redis
time="Jan 31 05:44:23" level=debug msg="keyName is: tyk-system-analytics_3_protobuf" prefix=redis
time="Jan 31 05:44:23" level=debug msg="Input key was: analytics-tyk-system-analytics_3_protobuf" prefix=redis
time="Jan 31 05:44:23" level=debug msg="Fixed keyname is: analytics-tyk-system-analytics_3_protobuf" prefix=redis
time="Jan 31 05:44:23" level=error msg="Multi command failed: read: connection reset by peer" prefix=redis
time="Jan 31 05:44:23" level=debug msg="Storage Engine already initialized..." prefix=redis
time="Jan 31 05:44:23" level=debug msg="Unpacked vals: 0" prefix=redis
time="Jan 31 05:44:23" level=debug msg="Getting raw key set: tyk-system-analytics_4" prefix=redis
time="Jan 31 05:44:23" level=debug msg="keyName is: tyk-system-analytics_4" prefix=redis
time="Jan 31 05:44:23" level=debug msg="Input key was: analytics-tyk-system-analytics_4" prefix=redis
time="Jan 31 05:44:23" level=debug msg="Fixed keyname is: analytics-tyk-system-analytics_4" prefix=redis
time="Jan 31 05:44:23" level=error msg="Multi command failed: read: connection reset by peer" prefix=redis

These are the recurring logs in the tyk_pump:
redis-master pod logs:

1:M 31 Jan 2024 05:12:31.526 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number (addr=:44116 laddr=:6379)
1:M 31 Jan 2024 05:12:33.139 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number (addr=:44122 laddr=:6379)
1:M 31 Jan 2024 05:12:33.159 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number (addr=:44128 laddr=:6379)
1:M 31 Jan 2024 05:12:33.183 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number (addr=:44142 laddr=:6379)
1:M 31 Jan 2024 05:12:33.200 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number (addr=:44158 laddr=:6379)
1:M 31 Jan 2024 05:12:33.211 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number (addr=:44160 laddr=:6379)
1:M 31 Jan 2024 05:12:33.222 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number (addr=:44166 laddr=:6379)
1:M 31 Jan 2024 05:12:33.233 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number (addr=:44168 laddr=:6379)
1:M 31 Jan 2024 05:12:33.251 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number (addr=:44184 laddr=:6379)
1:M 31 Jan 2024 05:12:33.262 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number (addr=:44186 laddr=:6379)
1:M 31 Jan 2024 05:12:33.287 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number (addr=:44196 laddr=:6379)

if anyone faced this issue please provide appropriate solution!
Thanks

@arshavardan
Copy link
Author

Could this be causing by different openssl Versions between Redis and the Tyk Pump, below are the listed Openssl versions of both tyk-components and Redis:
Dashboard: openssl 1.1.1w

Gateway: openssl 1.1.1w

mdcb: openssl 1.1.1w

pump: Openssl 1.1.1n

Redis: openssl 1.1.1w

You can see that pump openssl version is different from the others!

@MichaelTyk MichaelTyk changed the title ERROR: Tyk PUMP not able to connect to Redis Sentinels [TT-11223]ERROR: Tyk PUMP not able to connect to Redis Sentinels Feb 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant