Node metadata lost after long-running: ros2 node list shows no nodes, but processes are alive and topics exist！

[cocaston]

We are running a large-scale ROS 2 application launched via ros2 launch service call. After approximately 8–9 hours of continuous operation, the following symptoms occur:

• ros2 node list shows no active nodes.
• However, all corresponding node processes are still alive (confirmed via ps, top, or systemd status).
• ros2 topic list shows that all topics are still present.
• Running ros2 topic info -v /battery_state shows the associated node as:
  NODE_NAME_UNKNOWN
• All logs suddenly stop at a certain point in time, including rosout.
  This suggests that while the nodes are still running and publishing data, their ROS graph metadata has been lost or not properly maintained, which may be related to RMW/DDS internal state or lifecycle management.

My Environment:

• ROS Version: HUMBLE;
• RMW IMPLEMENTATION: rmw_fastrtps_cpp;
• FastRtps Version: 6.2.7;
• OS: Ubuntu22.04;
• Kernel Version: 5.15.136-tegra;
• CPU Architecture: aarch(Orin Nano).

Has anyone seen this before? Your help would be appreciated.

[fujitatomoya]

Can you not get any topic list without using daemon?

ros2 topic list --no-daemon

how about restarting daemon, does it fix the problem?

ros2 daemon stop
ros2 daemon start

Assuming this is not gonna work, but can you echo the topic data?

ros2 topic echo <TOPIC>

Any error/warning in the logging data in the file system? (not rosout)

[cocaston]

Thanks for your reply!

1. This command also does not display any nodes.

ros2 topic list --no-daemon

2. Operating the daemon has made no improvement to the current situation.
3. There is no output from ros2 topic echo.
4. Some nodes that use gRPC are still publishing and processing data.
5. I have some nodes that print logs when they haven't received data for a long time, and this is reflected in launch.log.In addition to this, there are no other warning messages.

Since the restart, it has not recurred so far; some specific logs will be added next time it occurs.
Not sure if this information is helpful.

[wjwwood]

(from the bug triage meeting) @MiguelCompany any ideas on this?