Unified print library

[Avamander]

Due to all the differences between different micro-controllers this library works on the current method of printing debug, warning and error messages is rather clunky and ugly (relatively speaking), often causing confusion. Because of that I propose that a universal serial printing wrapping library is created. (Also see nRF24/RF24#189 (comment))

I opened identical issue also in nRF24/RF24Network#103 nRF24/RF24Mesh#110

(This would also prove really useful for other RF24* libraries, maybe even for some other libraries too)

[wmarkow]

@Avamander, if I understand correctly that need to be some kind of abstraction which exposes some common/unified API methods. That common API could be used for logging purposes in general C/C++ code (so it would be portable). During the compilation it will "somehow" detect the target hardware and use the correct implementation to print to serial. Of course the unified library needs to provide the proper printing implementation for every supported hardware, right?

[Avamander]

Indeed. You are interested in implementing something of that sort?

[wmarkow]

I was just thinking that this library could be used for a logging purposes. Let me just one more question: would it be something similar to slf4j project for Java? slf4j is some kind of facade on different logging frameworks. The library could do the same (but with a lot of limitations - we are working with embedded devices). I imagine that with this library you could:

• provide a standard abstraction API that could be portable
• provide some methods to log messages at different level, like info, warn, error, debug
• configure what global logging level you want in your application; one of info, warn, error, debug
• choose what log appenders you can use:
  • DevNullLogAppender which will trash your logs. This could be a default implementation for platforms/devices that do not have any display, file system, or serial port (the code will compile but logging will not be available)
  • SerialPortLogAppender wich will output the logs to the serial port. This could be a default implementation for Arduino (for instance)
  • ConsoleLogAppender which will print the logs to the console. This could be useful for RaspberyPi platform.
  • FileLogAppender which will output the logs to file. This could be useful for RaspberryPi platform.
• when you implement a library you just use some standard API methods (info, warn, error, debug) to log
• when you implement the final application the default log appender would be info, warn, error, debug but you could register some other if you want

The library must provide a proper implementations for various platforms. I feel like the DevNullLogAppender would a common for all of supported platforms (even for those which are not yet supported). This appender will just do nothing. On unsupported platforms/devices the code will compile but logging will not be available.

The question is: would it be so simple as I described above or maybe I'm missing something that I'm not aware of (and the thing would not be so easy then)?
Second question: are we talking about C or C++ (or both)?

[Avamander]

It never is as simple, but it won't be hard either.
C and C++ wrapper seems reasonable?

Though I've been thinking, what about @TMRh20 making a GH RF24 organization and creating both this library's repository there and moving other RF24* under that organization too?

[wmarkow]

I have found something similar for C++. It is called plog. Its readme document is very nice especially this image.

@Avamander, are you talking about something similar? This plog may be to heavy for Arduino and embedded systems but at least we could learn something.
C and C++ wrapper sounds reasonable so somebody could choose what he need. On github repo for C version and another one for C++ wrapper?

[Avamander]

Indeed. Though I feel like the wrapper and library can be put into the same repository.

[TMRh20]

I'm not able to keep up with these discussions lately, but sounds like some good ideas. If anyone wants to take charge with RF24 org etc, feel free. I'll still contribute what I can. Thanks for keeping things going!

[Avamander]

@TMRh20 I made an organization, but your help is needed for migrating the repositories, sites.

[TMRh20]

Ok, I'll check that out soon. Not very familiar with it, so let me know if you need something specific.

[Avamander]

@TMRh20 To migrate a repository to an organization go to repository Settings->Transfer->Type in the name and the organization.

[2bndy5]

I'm picturing a StringStream type library to get things rolling. There are so many issues that get snagged on the printf() problem. I love the idea of adding logging levels, but people I've collaborated with in the past tend to inadvertently kill a project by setting goal(s) way too high before the code-base even gets a skeleton to build from. With that said, you can probably guess I'm a "don't re-invent the wheel" kinda guy.

@Avamander I assume you want to take the lead and start a new nRF24 repo in this regard. We could always merge the stable product into RF24 repo (maybe using submodule feature), but this idea clearly needs a home for incubation (evident by duplicate issues accross multiple repos).

@TMRh20 @Avamander @Oitzu @wmarkow Anyone opposed to starting a nRF24 github "project" so we can expedite efforts via Kanban board? I'd say let's form a Github "team", but without $$, this github org is limited to having only teams of 7 people (can't remember if multiple teams is also limited).

p.s I wish the org name wasn't the same name of the RF24 (core) repo. Can we address this now before it gets too confusing? It would require a owner's consent.

[Avamander]

I assume you want to take the lead and start a new nRF24 repo in this regard.

I can create one, yes.

Anyone opposed to starting a nRF24 github "project" so we can expedite efforts via Kanban board?

An issue would suffice.

Can we address this now before it gets too confusing?

It's nRF24 not RF24, and it's been like this for years. Changing would cause a lot of problems with little benefit.

[2bndy5]

years?! nevermind. oops I didn't see how old this thread was

[Avamander]

Oh you mean the display name, I changed that, it doesn't affect any URLs or anything.

[wmarkow]

I have found this:
https://github.com/thijse/Arduino-Log

I just took a quick look at this project.

[2bndy5]

@wmarkow that ArduinoLog lib looks very promising! Tried it out with the Feather M0 Express (a board that's experiencing printf() trouble with RF24::printDetails()), and it works! I have a question though: What log level would printDetails() fall under?

0 - LOG_LEVEL_SILENT no output
1 - LOG_LEVEL_FATAL fatal errors
2 - LOG_LEVEL_ERROR all errors
3 - LOG_LEVEL_WARNING errors and warnings
4 - LOG_LEVEL_NOTICE errors, warnings and notices
5 - LOG_LEVEL_TRACE errors, warnings, notices, traces
6 - LOG_LEVEL_VERBOSE all

That list is directly from the header (which also seems to be doxygen compatible 👍 ). I need to re-read the printf.h file to understand how to keep the current x86 platform implementation for printf().

[wmarkow]

@2bndy5 I think it should go under LOG_LEVEL_TRACE as the doc around this method says:

Print a giant block of debugging information to stdout

Sorry but it will be a long comment.

I just took a look at ArduinoLog. It may be promising - as the basic feature of logging is provided - however it misses a few nice things that I would like to have in my projects. From my point of view - let's give an theoretic example: I just want to have some project which will use the follwoing Arduino libraries:

• library with name RF24 to be able to send data with nRF24L01 chip
• library with name RDA5807M to use FM tuner chip
• library with name LCD to display some data on the display

I'm assuming that all of these libraries will use some unified pront library or some kind of unified Arduino logger. My project will have a name like HiFi and it will just put some music from FM tuner to the speakers, also will display some info (like current volume and radio station or even RDS data) on the lcd. RF24 will be used just to provide remote controll feature.
In this project it would be nice for me to se the following log in the serial console (when attached to my computer):

2  INFO  HiFi     main.cpp         application start
4  DEBUG HiFi     RF24Handler.cpp  configure RF24 chip
5  DEBUG RF24     RF24.cpp         begin() called
6  DEBUG HiFi     RDAHandler.cpp   configure RDA5807M chip
8  DEBUG RDA5807M                  begin() called             
10 DEBUG LCD                       init lcd with defaults
15 TRACE HiFi     Button.cpp       button pressed
20 DEBUG HiFi     SeekUpButton.cpp seek up button pressed
25 DEBUG RDA5807M                  Seeking up...
30 DEBUG RDA5807M                  ...found next station
35 INFO  HiFi     main.cpp         found new station 103.4 MHz

The log information is organised in columns. Here is the column meaning:

• arduino system timestamp in milliseconds
• log level (ERROR, WARN, INFO, DEBUG or TRACE)
• ID of library or application - this is mandatory field
• additional tag (not mandatory) which may help to identify where the message comes from (like file name or something similar)
• log message - this is mandatory field

Log level, ID of library and log message would be mandatory, so for example the log library API should offer at least (or something similar) the methods like:

Log.error(char* id, char* message);
Log.warn(char* id, char* message);
Log.info(char* id, char* message);
Log.debug(char* id, char* message);
Log.trace(char* id, char* message);

Then it would be nice to configure the logger and for example say:

• I want to totally disable logging for LCD library
• for RF24 I want to set INFO log level
• for HiFi I would like to have TRACE
• for RDA5807M I leave a default value (which is INFO)

At work I'm a Java developer where we use slf4j, which allows a powerfull configuration possibilities. But in Arduino we do not have such a resources like huge amount of RAM or "sophisticated" data collections like lists or hashmaps (we could have them but they will consume RAM). By default the log library will have INFO set as a log level, for all usages. I just came with the idea that the library cound have an entry point to the software developer's method where he can accept the log before being printed, so in my HiFi project I can implement a method like:

bool acceptLog(char* id, char* logLevel)
{
    // disable log for LCD
    if(id is "LCD")
    {
        return false;
    }

    // accept INFO for RF24
    if(id is "RF24" and logLevel in (ERROR, WARN, INFO))
    {
        return true;
    }

    // accept TRACE for HiFi
    if(id is "HiFi")
    {
        return true;
    }

    // accept only INFO for other messages
    if(logLevel in (ERROR, WARN, INFO))
    {
        return true;
    }

    // everything else is disabled
    return false;
}

If the developer's method is registered in the log engine, then it will be called any time, before something will be logged. If the method is not registered, then logger will pass only INFO messages (a default behaviour).

I'm not quite sure about resources needed to provide such a mechanism. Will it consume too much of RAM or FLASH memories? Will it consume a lot of processor time (the acceptLog method compares strings)? Does it make even sense or I would like to have too much?

Besides, I was inspired by slf4j logger.

[wmarkow]

Heh, a cherry on the cake would be to support ANSI escape colors and display ERROR in red, WARNING in yellow, INFO in GREEN and others in BLUE. Who would use that feature?

[2bndy5]

Great notes! Clearly you have a grander picture in mind than I do. My main drive is to address the lack of printf() on certain platforms.

Who would use that feature?

Linux people (in relation to only RF24* libs).

I hail from a python background, and I have used a similar logging library for a flask webapp. I've never seen custom colors (or even emojis to that extent) used in the Arduino IDE's Serial monitor.

About resources

The AVR platform does have significant limitations on memory, which is probably why the ArduinoLog library disables ALL logging instead of certain levels. If the readme mentions saving memory when explaining how to use the DISABLE_LOGGING macro, then its a utilitarian solution. Also that ArduinoLog library seems specifically geared towards the development of AVR-based libraries and not-so-much for end-user applications. Still, its an excellent lead.

[Avamander]

I've never seen custom colors (or even emojis to that extent) used in the Arduino IDE's Serial monitor.

That's actually really rather sad. It's not like escape sequences are expensive to send...

Will it consume too much of RAM or FLASH memories? Will it consume a lot of processor time (the acceptLog method compares strings)? Does it make even sense or I would like to have too much?

Doing this compile-time seems to be the only reasonable way e.g. #define RF24_LOG_LEVEL WARN and all calls under that get excluded.

however it misses a few nice things that I would like to have in my projects.

I have to agree. I think we still have to whip our own library up, I'll make the repository as promised.

[Avamander]

@2bndy5 @wmarkow I'll name the repository as "RF24Log" but I'd change it when someone has any better name ideas.

[2bndy5]

Would printing to the serial GPIO RX/TX pins be a viable option? Can't think of a good use case for this (maybe hc-05 Bluetooth modules or compatibles like 3d printers sometimes offer), but figured I'd ask anyway.

[wmarkow]

@2bndy5, that should be possible, just need to implement a correct log handler (a big discussion here). If you want to output data in the format of UART then a SoftwareSerial library may be used for this purpose. Of course the logger just writes to serial, so actually RX would be not used.