Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RobustConnection is not restored after "Server connection was stuck" #588

Open
tsdaemon opened this issue Oct 13, 2023 · 2 comments
Open

Comments

@tsdaemon
Copy link

aio-pika==9.2.3
aiormq==6.7.7
RabbitMQ: 3.10.10, remote cluster

Related issues: #577 #563

After period of inactivity we observed an error in logs:

Server connection was stuck. No frames were received in 183 seconds.

This error was logged but it didn't stopped the process: it would help if consumer just fail, then K8s would restart it. But it was left hanging.

I was not able to reproduce it locally. We will be looking into options how to self-heal a connection on a callback, but any advice is appreciated.

@dimbler
Copy link

dimbler commented Nov 27, 2023

+1
aio-pika=9.3.1
aiormq=6.7.7
No reconnection attempt after
Server connection <Connection: "amqp://******@******:5672/apsv?heartbeat=60" at 0x7f042d374040> was stuck. No frames were received in 3 seconds. but in brocker side is closing AMQP connection <0.7020.2515> (*******:62649 -> *******:5672, vhost: '******', user: 'test'): client unexpectedly closed TCP connection
You can reproduce this issuer by blocking all connection between server and client for time more that heartbeat timeout.

@guishake
Copy link

guishake commented Nov 28, 2023

I get this issue too, but it's not bound to the 9.3 version, I encountered it also with 9.0 and thought the 9.1.3 would help unfortunately not. Is there any progress on this ?
It would help if we could get back the control over the connection to close it properly and re-open it for instance, would it be possible ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants