RFC: Add basic support for local and remote CAN

[stoupa-cz]

Description

This is a draft of basic CAN support. It is in RFC, as there are open questions, missing docs and tests, and moreover, it needs more testing. We plan to use it to remotely access the satellite's telemetry buses during development. It is tested on our local branch to remotely access the satellite's telemetry buses during development.

Local and remote CAN resources are added. Additionally, a simple CAN driver is added. The driver has very limited capabilities. For local use, it can set up the interface and tell you the interface name. For remote use, it establishes a transparent tunnel using cannelloni. With it, you can use the local virtual CAN interface. All the traffic is routed to the real interface on the other end.

Be aware, as cannelloni states in its doc:

cannelloni is not suited for production deployments. Use it only in environments where packet loss is tolerable. There is no guarantee that CAN frames will reach their destination at all and/or in the right order.

I tested in the lab only. We will test it in production during September/October.

Draft version (without labgrid-can-interface) is being tested in production for ~two months.

I see the main blocker: it requires root privileges on both sides (exporter and client; even for local use). What is the preferred way to handle privileged access? May we use labgrid-raw-interface? Helper labgrid-can-interface has been added.

Checklist

• Solve the need for root privileges
• Test initial implementation
• More robust handling of cannelloni processes
• CAN configuration via labgrid or e.g., systemd-networkd, see the discission
• Fix remaining TODOs in the code
• Rebase on master
• Documentation
  • The arguments and description in doc/configuration.rst have been updated
  • Add a section on how to use the feature to doc/usage.rst
  • Man pages have been regenerated
• Write tests
• Test in production

Reference issues: #1287 and #1289

This is a part of my work at VZLU AEROSPACE.

[hundeboll]

Hi @stoupa-cz

Thanks for working on this. I've been experimenting with [socketcand](https://github.com/linux-can/socketcand). It has the same limitation as cannelloni (root on both ends for vcan setup), but the protocol is pretty simple, so the client side can be implemented as python functions in the labgrid driver. This should be enough for tests with limited CAN traffic.

[stoupa-cz]

Hi @stoupa-cz

Thanks for working on this. I've been experimenting with [socketcand](https://github.com/linux-can/socketcand). It has the same limitation as cannelloni (root on both ends for vcan setup), but the protocol is pretty simple, so the client side can be implemented as python functions in the labgrid driver. This should be enough for tests with limited CAN traffic.

Hi @hundeboll ,

Thanks a lot for the information. This is a clever way. We are using the cannelloni. Root privileges are manageable on exporters; however, handling them on clients (e.g., developers' computers) is cumbersome.

I'm currently trying a helper like labgrid-raw-interface. I will push it when ready.

Anyway, as pointed out in #1289, we may support both backends (you can see that issue for more information about SW selection).

BTW, is your experiment available anywhere?

[hundeboll]

Hi @stoupa-cz
Thanks for working on this. I've been experimenting with [socketcand](https://github.com/linux-can/socketcand). It has the same limitation as cannelloni (root on both ends for vcan setup), but the protocol is pretty simple, so the client side can be implemented as python functions in the labgrid driver. This should be enough for tests with limited CAN traffic.

Hi @hundeboll ,

Thanks a lot for the information. This is a clever way. We are using the cannelloni. Root privileges are manageable on exporters; however, handling them on clients (e.g., developers' computers) is cumbersome.

I'm currently trying a helper like labgrid-raw-interface. I will push it when ready.

I see. I think the cannelloni protocol would be just as easy to implement in python though.

Just like you've done, my plan was to create a resource similar to SerialPortExport, which would start socketcand on a NetworkInterface resource (i.e. can0).

The driver could then connect to the resource similar to SerialDriver. Or do regular socket operations on local interfaces instead of socketcand procotol communication. This still doesn't solve the root requirement for configuration of local interfaces, but as a start we could leave that to the administrator (e.g. using systemd.network(5) configuration).

Anyway, as pointed out in #1289, we may support both backends (you can see that issue for more information about SW selection).

BTW, is your experiment available anywhere?

It still resides in our internal repository, but I've attached it here. It uses SSH forward to talk with socketcand on the target though, as it is just used for communicating with a CAN based service in our bootstrap rootfs, but that's easy to change.

[hundeboll]

Hi @stoupa-cz

I tried running an exporter with your PR, where a USBCANPort is exported as an NetworkCANPort. On the client, I implemented the cannelloni protocol in the driver (can.py).

It works quite well for our purpose.

One thing I miss from socketcand is the ability to subscribe to certain CAN IDs, so I added (very) simple filtering in the driver.

And again: It might make sense to remove the interface configuration, and leave that to e.g. systemd-networkd?

[stoupa-cz]

I tried running an exporter with your PR, where a USBCANPort is exported as an NetworkCANPort. On the client, I implemented the cannelloni protocol in the driver (can.py).

It works quite well for our purpose.

One thing I miss from socketcand is the ability to subscribe to certain CAN IDs, so I added (very) simple filtering in the driver.

It seems we have different needs. For us, a client-side "real" vcan interface is necessary. Fortunately, we have a configurable driver, so any client (e.g., your cannelloni protocol reimplementation) can be selected. For the sake of simplicity, I’d prefer to continue using the "native" cannelloni approach.

BTW, what is your use case? Do you want to replay some traffic?

And again: It might make sense to remove the interface configuration, and leave that to e.g. systemd-networkd?

On exporter side, the interface is controlled (e.g., down, set-bitrate, up) when the place is acquired. The bitrate configuration comes from the expoter configuration. It is similar on client when the driver is activated, although the bitrate is loaded from the exporter's configuration. Which interface configuration managed by systemd-networkd are you referring to?

[hundeboll]

It seems we have different needs. For us, a client-side "real" vcan interface is necessary. Fortunately, we have a configurable driver, so any client (e.g., your cannelloni protocol reimplementation) can be selected. For the sake of simplicity, I’d prefer to continue using the "native" cannelloni approach.

BTW, what is your use case? Do you want to replay some traffic?

Our use case is to instrument a service on target that talks CAN only. It is low volume traffic, so receiving and sending frames directly from (pytest) python code is just fine; preferably actually. I can see other cases, where "external" test software requires a read (virtual) can interface is necessary also.

But yes, different drivers on the client solves this nicely.

[hundeboll]

On exporter side, the interface is controlled (e.g., down, set-bitrate, up) when the place is acquired. The bitrate configuration comes from the expoter configuration. It is similar on client when the driver is activated, although the bitrate is loaded from the exporter's configuration. Which interface configuration managed by systemd-networkd are you referring to?

CAN bus configuration is, in my view, a hardware property. So not something that should change from one test invocation to another. Thus I believe that leaving configuration of the CAN bus to the OS hosting the exporter (for remote) or the client (for local) is the least surprising approach.

When I mention systemd-networkd, it is because it can manage/configure CAN interface by use of systemd.network(5) files:
https://man.archlinux.org/man/systemd.network.5.en#%5BCAN%5D_SECTION_OPTIONS

So, to let the OS manage CAN bus properties, simply let the administrator drop a can.network file in /etc/systemd/network/ and that's it. No need for privileges in labgrid.

[stoupa-cz]

It seems we have different needs. For us, a client-side "real" vcan interface is necessary. Fortunately, we have a configurable driver, so any client (e.g., your cannelloni protocol reimplementation) can be selected. For the sake of simplicity, I’d prefer to continue using the "native" cannelloni approach.
BTW, what is your use case? Do you want to replay some traffic?

Our use case is to instrument a service on target that talks CAN only. It is low volume traffic, so receiving and sending frames directly from (pytest) python code is just fine; preferably actually. I can see other cases, where "external" test software requires a read (virtual) can interface is necessary also.

But yes, different drivers on the client solves this nicely.

Ok, understand. The CANProtocol might be beneficial in this case. For us, there is an application like microcom connecting to vcan, which handles the communication via a pytest fixture. The communication is quite complex (although of low volume as well).

On exporter side, the interface is controlled (e.g., down, set-bitrate, up) when the place is acquired. The bitrate configuration comes from the expoter configuration. It is similar on client when the driver is activated, although the bitrate is loaded from the exporter's configuration. Which interface configuration managed by systemd-networkd are you referring to?

CAN bus configuration is, in my view, a hardware property. So not something that should change from one test invocation to another. Thus I believe that leaving configuration of the CAN bus to the OS hosting the exporter (for remote) or the client (for local) is the least surprising approach.

When I mention systemd-networkd, it is because it can manage/configure CAN interface by use of systemd.network(5) files: https://man.archlinux.org/man/systemd.network.5.en#%5BCAN%5D_SECTION_OPTIONS

So, to let the OS manage CAN bus properties, simply let the administrator drop a can.network file in /etc/systemd/network/ and that's it. No need for privileges in labgrid.

Speaking of bitrate (which is the only property handled now), I followed the approach of a serial line. The configuration is stored close to labgrid (in the exporter.yaml) and connected with the expected udev match. In my opinion, the most important part is handling the interface state (up/down), which should be part of the acquire/release process to enter the defined state. For this, we need the elevated privileges, thus the sudo helper.

However, I see an option: we may have conditional handling of the CAN interface configuration in labgrid. This way, you can choose: Either control the interface state by labgrid and let the system configure the rest, or give it fully into the hands of labgrid.

Anyway, I have no strong opinion on this point, so I've added a link to your comment to the checklist of things to solve. I'd like to hear the maintainer's opinion.

Thanks for your ideas and testing. I appreciate them.

[hundeboll]

I agree. Just let the CANPort resource default to no bitrate setting, and only run ip link ... commands if a bitrate is configured. Having a default is probably bad anyways, as we cannot guess the bitrate reliably.

I think we should focus on getting the resource merged, as it's easy to write custom drivers in the client context. Then we can discuss the details of the driver later.

[hundeboll]

@jluebbe @Bastian-Krause What do you think?