Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bootstrap node can't add the connected node into peers using examples/distributed-key-value-store in public network #5445

Closed
ShuochengWang opened this issue Jun 6, 2024 · 5 comments

Comments

@ShuochengWang
Copy link

ShuochengWang commented Jun 6, 2024

I use two Azure servers in public network environments,

  • one for bootstrap node, called node A
  • one for new-joined node, called node B

The bootstrap node runs examples/distributed-key-value-store example, listening on 3000 port
And the new-joined node runs a modified examples/distributed-key-value-store example, add the address of the bootstrap node, and listening on 3000 port too.

    ......
    swarm.behaviour_mut().kademlia.set_mode(Some(Mode::Server));


    let bootstrap_peer: libp2p::Multiaddr =
        "/ip4/here is my bootstrap_node ip/tcp/3000".parse().unwrap();
    println!("Bootstrap Peer: {:?}", bootstrap_peer);

    use std::str::FromStr;
    // here is my peer_id of my bootstrap node
    let bootstrap_peer_id = libp2p::PeerId::from_str("12D3KooWT1BYJpwNrQpAtF5kYw439PJmabBx8QnhVdUqCTGnhy7p").unwrap();
    println!("PeerId: {:?}", bootstrap_peer_id);

    swarm
        .behaviour_mut()
        .kademlia
        .add_address(&bootstrap_peer_id, bootstrap_peer);


    // Read full lines from stdin
    let mut stdin = io::BufReader::new(io::stdin()).lines().fuse();

    // Listen on all interfaces and whatever port the OS assigns.
    swarm.listen_on("/ip4/0.0.0.0/tcp/3000".parse()?)?;
    ......

And I found that, the new-joined node B can add the bootstrap node A into peers, But the bootstrap node A can't add the new-joined node B into peers. Why? How to fix it?

bootstrap node output

Listening in "/ip4/127.0.0.1/tcp/3000"
Listening in "/ip4/10.0.0.15/tcp/3000"

GET key
Failed to get record: NotFound { key: Key(b"key"), closest_peers: [] }

GET key
Got record "key" "value"

PUT key2 value2
Failed to put record: QuorumFailed { key: Key(b"key2"), success: [], quorum: 1 }

new-joined node output

Listening in "/ip4/127.0.0.1/tcp/3000"
Listening in "/ip4/10.0.0.16/tcp/3000"

GET key
Failed to get record: NotFound { key: Key(b"key"), closest_peers: [PeerId("12D3KooWT1BYJpwNrQpAtF5kYw439PJmabBx8QnhVdUqCTGnhy7p")] }

PUT key value
Successfully put record "key"

GET key2
Got record "key2" "value2"

And if I print all events, I found BehaviourEvent: UnroutablePeer event

peer_id: 12D3KooWMiHQnCFsmoCeeGvYdJ3pmBxq9jdTPqtE8Hwsk4XxBF3y
2024-06-06T10:00:17.612692Z  INFO libp2p_swarm: local_peer_id=12D3KooWMiHQnCFsmoCeeGvYdJ3pmBxq9jdTPqtE8Hwsk4XxBF3y
2024-06-06T10:00:17.614029Z  INFO libp2p_mdns::behaviour::iface: creating instance on iface address address=10.0.0.15
Listening in "/ip4/127.0.0.1/tcp/3000"
Listening in "/ip4/10.0.0.15/tcp/3000"
2024-06-06T10:00:50.027521Z  INFO distributed_key_value_store_example: event: IncomingConnection { connection_id: ConnectionId(1), local_addr: "/ip4/10.0.0.15/tcp/3000", send_back_addr: "/ip4/here is my new-joined node ip/tcp/44470" }
2024-06-06T10:00:50.074444Z  INFO distributed_key_value_store_example: event: ConnectionEstablished { peer_id: PeerId("12D3KooWGqcDokRpdUJgFj8sqysVm6V3UY8KjFBpZWDX6Knz9bMx"), connection_id: ConnectionId(1), endpoint: Listener { local_addr: "/ip4/10.0.0.15/tcp/3000", send_back_addr: "/ip4/here is my new-joined node ip/tcp/44470" }, num_established: 1, concurrent_dial_errors: None, established_in: 46.779269ms }
2024-06-06T10:00:50.074808Z  INFO distributed_key_value_store_example: event: Behaviour(BehaviourEvent: UnroutablePeer { peer: PeerId("12D3KooWGqcDokRpdUJgFj8sqysVm6V3UY8KjFBpZWDX6Knz9bMx") })

2024-06-06T10:00:50.156606Z  INFO distributed_key_value_store_example: event: Behaviour(BehaviourEvent: InboundRequest { request: FindNode { num_closer_peers: 0 } })
2024-06-06T10:00:50.200464Z  INFO distributed_key_value_store_example: event: Behaviour(BehaviourEvent: InboundRequest { request: FindNode { num_closer_peers: 0 } })
2024-06-06T10:00:50.245503Z  INFO distributed_key_value_store_example: event: Behaviour(BehaviourEvent: InboundRequest { request: FindNode { num_closer_peers: 0 } })
2024-06-06T10:00:50.289643Z  INFO distributed_key_value_store_example: event: Behaviour(BehaviourEvent: InboundRequest { request: FindNode { num_closer_peers: 0 } })
@ShuochengWang
Copy link
Author

And if I use server B as bootstrap node, server A as new-joined node, the problem is the same
(I have opened the corresponding ports 3000 in my server setting)

@guillaumemichel
Copy link
Contributor

It may be linked to #5349

@dariusc93
Copy link
Member

You might want to try adding an external address with Swarm::add_external_address so the nodes would be set into server mode automatically (or maybe change the server mode), assuming that is the actual issue you are having. It might be related to what @guillaumemichel mentioned as well. Any additional logs would be helpful

@Fabijoey
Copy link

It may be linked to #5349

I do not think that it is linked to the issue above. I was able to replicate it with my the newest merge.
I think that is the same problem as mentioned here #2673 .

@ShuochengWang
Copy link
Author

I solved this problem by using both Swarm::add_external_address (add the real ip and port according to the config file) and identify protocal, and update the event loop

                        SwarmEvent::NewExternalAddrCandidate { address } => {
                            swarm.add_external_address(address);
                        }
                        SwarmEvent::NewExternalAddrOfPeer { peer_id, address } => {
                            swarm.behaviour_mut().kademlia.add_address(&peer_id, address);
                        }
                        SwarmEvent::ConnectionEstablished { peer_id, endpoint, .. } => {
                            if let libp2p::core::ConnectedPoint::Dialer { address, .. } = endpoint {
                                tracing::info!("Add address {:?}", address);
                                swarm.behaviour_mut().kademlia.add_address(&peer_id, address);
                            }
                        }
                        SwarmEvent::Behaviour(BehaviourEvent::Identify(identify::Event::Received {
                            peer_id,
                            info: identify::Info { listen_addrs, observed_addr, protocols, .. },
                        })) => {
                            if protocols.iter().any(|p| *p == kad::PROTOCOL_NAME) {
                                for addr in listen_addrs {
                                    swarm
                                        .behaviour_mut()
                                        .kademlia
                                        .add_address(&peer_id, addr);
                                }
                            }
                        }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants