-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Vertx eventbus not working even after cluster formation #5194
Labels
Comments
Hi @harisht87 This is the GH repository of the Vert.x core library, please send future reports to vertx-hazelcast In the Java code, it seems the event bus host address is forced to Likely that's why the nodes can't establish a connection to send messages. You should let the cluster manager pickup the right address. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Questions
We have two services running in OpenShift/K8s platform developed using Vertx framework, the cluster.xml is configure to use Kubernetes API and the cluster is getting formed correctly(logs below).
Vertx eventbus is set as in below code
Version
Hazelcast version : 5.3.6
vert.x version : 4.5.7
vert.x-hazecast : 4.5.7
K8s version : v1.25.11+c43ddea Red Hat Enterprise Linux 8.9 (Ootpa)
Istio version : 1.17.5-solo
java - jdk 8
Context
Sending message from one service to other service is not working getting below error.
Not connected to server c7ee11dc-98d4-4217-969c-e2ee2ac85ce9 - starting queuing
Draining the queue for server c7ee11dc-98d4-4217-969c-e2ee2ac85ce9
Cluster connection closed for server c7ee11dc-98d4-4217-969c-e2ee2ac85ce9
Do you have a reproducer?
No
Steps to reproduce
NA
Extra
Service 1 : [After 2nd member in cluster is deployed]
———————
Service 1 logs
————————
INFO: [192.168.54.48]:5701 [demo] [5.3.6] Initialized new cluster connection between /192.168.54.48:5701 and /127.0.0.6:52235
202Apr 23, 2024 1:09:26 AM com.hazelcast.internal.cluster.ClusterService
203INFO: [192.168.54.48]:5701 [demo] [5.3.6]
204
205Members {size:2, ver:2} [
206Member [192.168.54.48]:5701 - 3ee30ad6-4788-4aec-8050-7ed5603abdfa this
207Member [192.168.58.91]:5701 - 096a8f32-9212-4323-8fcf-5f505c11587f
208]
Service 2 :
———————
INFO: [192.168.58.91]:5701 [demo] [5.3.6] Kubernetes plugin discovered node name: cld-paas-d-eusw1b-3-k659z-worker-ds04-ckrsv
58Apr 23, 2024 1:09:20 AM com.hazelcast.kubernetes.KubernetesClient
59WARNING: Cannot fetch public IPs of Hazelcast Member PODs, you won't be able to use Hazelcast Smart Client from outside of the Kubernetes network
60Apr 23, 2024 1:09:21 AM com.hazelcast.internal.server.tcp.TcpServerConnection
61INFO: [192.168.58.91]:5701 [demo] [5.3.6] Initialized new cluster connection between /192.168.58.91:38785 and /192.168.54.48:5701
62Apr 23, 2024 1:09:26 AM com.hazelcast.internal.cluster.ClusterService
63INFO: [192.168.58.91]:5701 [demo] [5.3.6]
64
65Members {size:2, ver:2} [
66Member [192.168.54.48]:5701 - 3ee30ad6-4788-4aec-8050-7ed5603abdfa
67Member [192.168.58.91]:5701 - 096a8f32-9212-4323-8fcf-5f505c11587f this
68]
Issue 1:
————————————
01:13:42.674 [vert.x-eventloop-thread-2] DEBUG io.vertx.core.eventbus.impl.clustered.ConnectionHolder - tx.id=0e24b487-2840-40a5-9103-4f56d31d74ea Not connected to server c7ee11dc-98d4-4217-969c-e2ee2ac85ce9 - starting queuing
01:13:42.774 [vert.x-eventloop-thread-2] DEBUG io.vertx.core.eventbus.impl.clustered.ConnectionHolder - tx.id=0e24b487-2840-40a5-9103-4f56d31d74ea Draining the queue for server c7ee11dc-98d4-4217-969c-e2ee2ac85ce9
01:13:42.780 [vert.x-eventloop-thread-2] DEBUG io.vertx.core.eventbus.impl.clustered.ConnectionHolder - tx.id=0e24b487-2840-40a5-9103-4f56d31d74ea Cluster connection closed for server c7ee11dc-98d4-4217-969c-e2ee2ac85ce9
(NO_HANDLERS,-1) No handlers for address child
Issue 2 :
—————
When service 2 is deployed and service 1 is trying to initialize a cluster connection, Instead of the connecting to the cluster ip, it is trying to connect an unknown ip but cluster is formed with the correct service 2 ip.
Service 1 ip : 192.168.54.48
Service 2 ip : 192.168.58.91
Unknown ip : 127.0.0.6:52235
Log snippet
———————
Service 1 logs
————————
INFO: [192.168.54.48]:5701 [demo] [5.3.6] Initialized new cluster connection between /192.168.54.48:5701 and /127.0.0.6:52235
202Apr 23, 2024 1:09:26 AM com.hazelcast.internal.cluster.ClusterService
203INFO: [192.168.54.48]:5701 [demo] [5.3.6]
204
205Members {size:2, ver:2} [
206Member [192.168.54.48]:5701 - 3ee30ad6-4788-4aec-8050-7ed5603abdfa this
207Member [192.168.58.91]:5701 - 096a8f32-9212-4323-8fcf-5f505c11587f
208]
Application hazelcast cluster.xml file
————————————————
The text was updated successfully, but these errors were encountered: