Reuse and reallocate TCP/UDP port range

[DavidePrincipi]

The current implementation of TCP and UDP ports allocation has two limitations:

1. Modules like NethVoice allocates a wide range of port numbers, leading to possible node port exhaustion. When an instance is removed, used ports must be reclaimed by the node agent.
2. By time, the number of ports required by a module may change. For instance, Mattermost received an additional TCP port for the Calls plugin. As the ports are allocated by the add-module action, the only way to require more ports is with backup/restore or instance clone.

Proposed solution

• Reclaim and reuse a port range after an instance is removed.
• Implement a node action that a module can invoke to receive a new range of ports (both TCP and UDP).
• Migrate existing range allocations from module/ID/environment keys to the new (trusted) allocation storage

Alternative solutions

This feature is required. Keeping things as they are makes difficult to install NethVoice and manage the update of applications that increase their port requirements.

See also

Discussion https://mattermost.nethesis.it/nethesis/pl/5s131bmj7jdwzj7gaogukb5z6a

[DavidePrincipi]

Acceptance tests

• Works on new installation
• Upgrade an existing system. Existing TCP and UDP ports allocation are migrated to /var/lib/nethserver/node/ports.sqlite DB for each cluster node.
• Redis keys *_ports_sequence are removed.
• Modules with role portsadm can run allocate-ports and deallocate-ports actions on the local node agent.
• Port ranges are allocated without conflicts.
• Port ranges are deallocated and can be reallocated later.
• Port allocation and deallocation is workable with clone-module
• Port allocation and deallocation is workable with import-module (migration) NethServer/ns8-core@9afcb17

Useful commands

Dump the ports.sqlite DB with

podman run -i --rm --volume /var/lib/nethserver/node/state/ports.sqlite:/srv/ports.sqlite:z alpine ash -c 'apk add sqlite ; sqlite3 /srv/ports.sqlite' <<<"SELECT *, 'tcp' FROM TCP_PORTS UNION SELECT *, 'udp' FROM UDP_PORTS;"

[DavidePrincipi]

Testing in core 3.1.0-dev.1

[DavidePrincipi]

Testing with core 3.1.0-dev.2

Test case

• Add a label to module image, for portsadm role authorization https://nethserver.github.io/ns8-core/modules/port_allocation/
• Test new module installation and existing module instance update
• The module instance must be authorized to run agent.allocate_ports() and agent.deallocate_ports() as documented

[gsanchietti]

Current implementation has one main problem: it always reallocates the entire range
Reconfiguring modules that utilize numerous ports can be challenging. Additionally, these ports might be publicly accessible and used by external clients. Changing them could lead to issues, as clients would be unaware of the new connection points (e.g., VPN endpoint of NethSecurity Controller). This issue can also arise in modules that do not publicly expose ports but use them for internal communication with other modules (e.g., NethVoice with NethVoice Proxy).

[gsanchietti]

Testing with core 3.1.0-dev.3

[DavidePrincipi]

Released in https://github.com/NethServer/ns8-core/releases/tag/3.1.0