UnchangedFieldsFilter (WIP naming) Filter to drop messages that do not contain changed fields

[EsipovPA]

Description

disclamer1
The proposed class name is not the final one. Better naming required.

disclamer2
This is just an idea, that appeared to me today. I am not not sure if it should be implemented as a part of this library, or if it is just an overengineering of sorts for such a trivial problem. Wanted to discuss it.

(Sort of a)Problem to solve

In my work I oftentimes encounter a need to drop the messages, that do not contain changed data in a specific fields. Like the messages from automobile CAN-bus, that contain data about vehicle body. These messages are published with the frequency ~100Hz, but contain all the same data. It is only an example

The idea

The idea is to implement a filter, that drops messages, that do not contain changes in the specified fields, or all the fields, excluding headers.

How it is solved now

This is purely for demonstration.
Say there is a node with a subscription.

from rclpy_message_converter.message_converter import convert_ros_message_to_dictionary

class ExampleNode:
    def __init__(self, node, *args, **kwargs):
        self.node = node
        self.subscription = node.create_subscription(...., self.sub_callback)
        self.last_data: dict | None = None

    def sub_callback(self, message):
        # Suppose they may be compared this way
        # Suppose the `message` contains only data used further
        # This comparison may be much more complicated and take more operations
        # Gets more and more complicated with more messages and subscriptions
        new_data = convert_ros_message_to_dictionary(message)
        if new_data != self.last_data:
            # If fields changed, do some work here

How it may look in my opinion

For example, the python implementation may have the following interface

class UnchangedFieldsFilter(SimpleFilter):
    def __init__(
        self,
        other_filter,       # Supposedly a Subscriber filter
        include_fields_regex: list[str]=[],         # In this case, the empty list means all fields except a Header
        exclude_fields_regex: list[str]=[],        # Same thing, but for exclusion
    ):
        # The main implementation and all other stuff

So, with the use of this filter the previous code example may be converted to

class ExampleNode:
    def __init__(self, node, *args, **kwargs):
        self.node = node
        self.subscriber_filter = Subscriber(# Initialize here #)
        self.unchanged_fields_filter = UnchangedFieldsFilter(
            self.subscriber_filter
            ['fields', 'expected', 'to', 'change'],
            ['fields', 'to', 'ignore', 'changes', 'in'],
        )
        self.unchanged_fields_filter.registerCallback(self.sub_callback)

    def sub_callback(self, message):
        # Do the work that requires updated data straight away

So for any subscription, added to the node, that needs only changed data, it will only require to add a Subscriber filter, an UnchangedFieldsFilter and a callback for the latter.

Motivation

• Clearer code for nodes with multiple subscriptions that have the requirements, described in the (Sort of a)Problem to solve section of the description.
• Simpler code support for such nodes as a result.

Design / Implementation Considerations

• Needs a better name, definetely.
• Interfaces for Python and C++ may be better. What is proposed is just a concept.
• Does someone see this as a useful thing in general? I have my own implementations for such scenarios, as most of other developers, using ROS2, I think. But it seems like there should be something that may appear in a message_filters, doing this king of stuff.

Additional Information

[EsipovPA]

If it seems like something to add to this library, I may take a shot at it. I just think I need to finish with tutorials section and one other issue on filters inconsistency between C++ and Python, that I proposed to deal with.

[mjcarroll]

Does someone see this as a useful thing in general? I have my own implementations for such scenarios, as most of other developers, using ROS2, I think. But it seems like there should be something that may appear in a message_filters, doing this king of stuff.

I think this is a very good idea and a common pattern, even if people have their own implementations. Centralizing this to message_filters is a good move in my mind.

As to name: DeltaFilter or ChangedFieldFilter both seem reasonable to me.

[mjcarroll]

As to implementation, I think that extracting the entire C++ message as a dictionary is pretty costly and we may want to think of a better way to do the comparison there. Perhaps the user could provide a lambda function that implements the comparison function that the filter will use? Then we can side-step all of the introspection performance hit.

There are some other considerations around comparing things like floats, arrays, and compound data types that we may want to think about in the implementation here as well.

[EsipovPA]

Just saving some useful links for future issue research.

Concerning C++ implementation.

I've done some googling, and came across this post at ROS dicourse by @christophebedard on converting messages into YAML-objects. Also there is this repo with the implementation.

Maybe I can utilize the proposed approach not to convert meassages into yamls, but rather to iterate over messages contents recursively.

Python implementation seems to be quite straightforward in comparison. For reference may address this implementation. Not to convert to dicts, but to iterarte the contents as well.

Did not dive deep into the proposed implementations. Saved here for the future context and not to loose the links.

Will dive deeper into the matter, when will start working on the implementation.