Got Device or resource busy for a new interface

[ffourcot]

Hello,

We found a weird issue with recent pyroute2. On master, we ran a code very close of this script:

import os
from pyroute2 import IPRoute

os.system("ip netns add netnsb")

def reproducer():
    ipra = IPRoute()
    iprb = IPRoute(netns="netnsb")
    ifname = "toto"
    # get one interface index in namespace A
    index = ipra.link("get", ifname="eth0")[0].get("index")
    # be sure that index does not exists in namespace b
    try:
        iprb.link("get", index=index)
    except Exception as err:
        # should be does not exists. That is expected behaviour
        print(err)
    iprb.link("add", ifname=ifname, kind="veth", peer="tata", index=index)

reproducer()

At first run, response is pyroute2.netlink.exceptions.NetlinkError: (16, 'Device or resource busy') when adding the veth.

At second run, interface is created.

We reproduced it on several computer (with this script). It looks like setting index is critical here, but I do not understand why. The same code (with adaptation to open iprb is another namespace) was working with a pyroute2 from year 2023.

[ffourcot]

I just noticed that the "working" version of pyroute2 print this log:

log.warning('Error 16, retry {}.'.format(retry_count))

so perhaps that this retry should be reintroduced?

[svinota]

Investigating, thanks!

[ffourcot]

Actually, the reproducer can be simplified with:

import os
from pyroute2 import IPRoute


os.system("ip netns add netnsb")

def reproducer():
    iprb = IPRoute(netns="netnsb")
    ifname = "toto"
    iprb.link("add", ifname=ifname, kind="veth", peer="tata", index=2)


reproducer()

It looks that forcing index=2 will fail for some unknown reason, but only at the first call, probably not pyroute2 related. The question is then if the "retry" on EBUSY error should be restored on main branch.

[ffourcot]

For the record, I found what happens inside the kernel.

Each namespace has an XArray to store currents ifindex in use. To find a "free" index, the net structure stores internal counter of last attributed ifindex, used as minimal value to attribute something free.

So:

1. Namespace is created. lo get ifindex 1, and internal counter is now 2
2. We try to insert a veth interface. The peer interface is created first, with auto ifindex attribution. The peer interface takes ifindex 2 (the first free index), and namespace internal counter is incremented to 3.
3. When creating the main veth interface, since we gave ifindex 2 as parameter, XArray structure returns EBUSY => ifindex already in use

The second call is working, since internal is now 3, and the peer interface will take ifindex 3. ifindex 2 stay free for our call. We are in a corner case of interface creation, since we are adding a pair of interfaces.