should we talk about reserving levels?

[2bndy5]

The thought occurred to me that we could skip the timestamp if a negative version of the log level were passed to log(). What if the user just wants an indented line (with vendorID) for multi-line messages?

We could of course not reserve certain levels and just patch in flags to control the header (kinda like a printf() specifier string).

[wmarkow]

I can imagine that this feature could be useful in some cases, but I do not recommend to use vendorId to indent the lines. The indentation should be done in the log handler; like for example the log handler will display only 10 first characters of vendor id. With this solution the lines could be always nice aligned because each field (timestamp, log level and vendor id) have always fixed length.

Can we use log level 1 for this purpose instead of negative values? A value 1 could have a special meaning: this log message is a multi-line message and is a part of previously logged message. The log handler could support this and in this specific case the timestamp (or even log level and vendor id?) will be not appended.
To log this kind of messages we could use:

RF24LOGGER_info("vendor", "The parameters are below:");
RF24LOGGER_log(1, "Weather", "temperature %f", 23.5);
RF24LOGGER_log(1, "Weather", "wind speed %f", 10);
RF24LOGGER_log(1, "Weather", "wind direction %s", "SE");

and the output could be:

   123;INFO   ;Weather;The parameters are below:
                       temperature 23.5
                       wind speed 10
                       wind direction SE

Of course it will look nice on terminals (file viewers) which have always the same character width (so the letter 'w' has the same width as the letter 'i').

[2bndy5]

I do not recommend to use vendorID to indent the lines

that's not what I meant, but close. I meant the vendorID be embedded in the margin of the indent. Imagine 2 Loggers writing a multi-line prompt to the same stream... Having an origin of reference might help that.

A value 1 could have a special meaning: this log message is a multi-line message and is a part of previously logged message.

Actually I was thinking of dedicating a whole octal level to special punctuation where at least 4 levels are devoted to different quantity indentations

RF24LOGGER_info("Weather", "The parameters are below:");
RF24LOGGER_log(1, "Weather", "temperature %f", 23.5);
RF24LOGGER_log(2, "Weather", "wind speed %f", 10);
RF24LOGGER_log(3, "Weather", "wind direction %s", "SE");

would output like:

   123; INFO  ;Weather;The parameters are below:
Weather;       temperature 23.5
Weather;              wind speed 10
Weather;                     wind direction SE

above demo is using 8-space tab characters.

[wmarkow]

Imagine 2 Loggers writing a multi-line prompt to the same stream...

Ok, I see. Good point. That may happen. On multithread operating system multi-line log messages may be a pain because they may mix together and appear in an "undefined" order. They should be logged under the same vendor id - in my opinion. How do you distinguish which multiline message belong to which master message? With this punctuation? Then every single thread should use a different punctuation (log level). That may be hard to manage, because some piece of code may be called from different threads.

Keeping log messages small to one line eliminates those problems. You could also try to construct log messages with \n character inside, which will actually move the caret to the next line:

RF24LOGGER_info("weather", "The parameters are below:\n temperature %f \n wind speed %f \n wind direction %s", 23.5, 10, "SE");

should something like:

   123; INFO  ;Weather;The parameters are below:
 temperature 23.5
 wind speed 10
 wind direction SE

From the logging perspective this is just a "one-line" log message and will be displayed in one call. But if you want to display 200 elements in a loop then we have a nut to crack. Normally in Java I would use:

• a StringBuilder object to construct a very long multiline string and pass it to the logger as a one string. However on simple AVR microcontrollers we do not have such a big RAM to construct long texts on the fly (so I do not recommend this solution)
• log everything in a loop as a simple one line messages (this could go even on AVR):

for(q=0; q<200;q++)
{
    RF24LOGGER_info("weather", "temp value %f", <put value here>);
}

[2bndy5]

Since we're already parsing through the format c-str one char at a time, we could also replace any \n char with a new header instead of just \n. That would be a bit more intense though.

I also have a RF24LOG_NO_EOL macro to disable new line ending at the end of a message (for LCDs). This behavior could also be used to filter out any \n chars from the message.

[2bndy5]

I just refactored the printf parsing a bit more, so now all platforms enjoy the same specifier support... I was able to combine appendFormatedMessage() into write(). I say this here because now all \n characters are automatically triggering a new header (timestamp, logLevel, vendorID).

from the modified gettingStarted.ino

  RF24Log_log(077, vendorID, "This\n\tis a multiline\n\t\tmessage that\n\tends with a\nblank line\n\n");
  RF24Log_log(75, vendorID, "%%%%This is level 0x%02x (0b%08b or%4lu)%3c", 75, 75, 75, '!');

Output from above code

        43;Lvl  77;RF24LogExample;This
        46;Lvl  77;RF24LogExample;	is a multiline
        51;Lvl  77;RF24LogExample;		message that
        55;Lvl  77;RF24LogExample;	ends with a
        59;Lvl  77;RF24LogExample;blank line
        62;Lvl  77;RF24LogExample;
        65;Lvl 113;RF24LogExample;%%This is level 0x4B (0b01001011 or  75)  !

This tactic also extends the macro RF24LOG_NO_EOL to filter out all \n chars (for scrolling LED matrices and stuff). We could also use the same tactic to replace \t chars with a literal 8 space padding 😜

[2bndy5]

Maybe the level 0 should be reserved as the "broadcasting" channel in which the header's log level description is skipped. With the ability to disable timestamps and the user can pass a zero-length string as the vendorID, its the closest thing we can offer to a truly blank line in the logs. I could also be able to rf24Log_log(0, "", ""); to print a blank line in the examples for all platforms - no more

#ifdef ARDUINO
Serial.println("");
#else
std::cout << std::endl;
#endif

[wmarkow]

@2bndy5 I see that you did quite a good job. It fixes some issues and introduces new features. Could you make a PR so we can make a review?

The idea with \n and \t formats seems to be ok. So \n moves the log messages to a new line (and the header - timestamp, log level and vendor id - is here also printed) and \t make a tabulator? This looks ok to me.

To print blank lines or disable timestamps - can we use some kind of new format - instead of reusing log levels for this:
like %m (this is just an example) which stands for "multiline" - then the part of the messages after %m will be moved to a new line without header. make it sense?
We could log then in this way:

RF24Log_log(RF24LogLevel::INFO, vendorID, "This is the beginning of multiline log");
RF24Log_log(RF24LogLevel::INFO, vendorID, "%mAnd this will be displayed in a new line without log header.");

With this idea the log level will be preserved and some developer may implement a very custom log handler where multiline feature will be just ignored and printed as a standard log message.

[2bndy5]

To print blank lines or disable timestamps - can we use some kind of new format - instead of reusing log levels for this

Instead of having the parser look for a %m specifier, I would just use the parser's escape mechanism: %\n (I have tested this idea - it works).

The only thing we would need to turn off is the log level and timestamp. The vendorId can be disabled with a simple

const char PROGMEM DisableVendor[] = ""; // I'm currently detecting empty strings and skipping the delimiter
RF24Log_log(0, DisableVendor, ""); // this is now a do while loop, so empty messages still get a header

The only other way to disable a timestamp is through a macro called RF24LOG_NO_TIMESTAMP (this is done to save program space).

The only use case I can think of needing the entire textual real-estate of a log line is for dumping a formatted file (like JSON for example). This is why its easier to just add an extra if condition to the appendLogLevel() instead of a bool (locally scoped in write()) to toggle \n manipulation when the parser finds a %m (assuming it can be anywhere in the line to toggle \n manipulation). Remember that RF24_NO_EOL is done using macros also to cut down on code space. The more supported specifiers we add to the parser, the more the program memory goes up... Basically right now, all numbers are either signed or unsigned long (even though the parser can/is handle/ignore specifier's modifiers like h)

[2bndy5]

Could you make a PR so we can make a review?

@wmarkow you can comment on commits, open a PR to the refactor branch to add your changes... I originally set out to get this building on PC and PicoSDK (using CMake). So I'm not yet ready to submit a PR for review (sorry).

and \t make a tabulator

This is also disabled when the macro RF24LOG_TAB_SIZE (which expects an integer value) is not defined for space reasons.

[wmarkow]

The vendorId can be disabled with a simple

const char PROGMEM DisableVendor[] = ""; // I'm currently detecting empty strings and skipping the delimiter 
RF24Log_log(0, DisableVendor, ""); // this is now a do while loop, so empty messages still get a header

I think providing (for any purpose - including disabling vendor id) an empty string as vendor id should be not allowed. Vendor id should be always provided as not empty so the log handler may use it to provide more sophisticated logic related to handling log messages. Disabling of vendor id must be done in a different way.

@2bndy5 , what is the status of your #18 pull request against this issue? I meant, does this PR reserve some log levels for special purposes or you did everything configurable by some define switches?

[2bndy5]

what is the status of your #18 pull request against this issue?

please checkout the examples. I am using the quoted snippet to prompt the user for input about filtering verbosity. AllLogLevels.ino would be a good example to run.

does this PR reserve some log levels for special purposes or you did everything configurable by some define switches?

There are a few new macros but nothing that conditionally reserves a logging level for a certain purpose. Level 0 is hard-configured to skip the timestamp and level description though.

[2bndy5]

Vendor id should be always provided as not empty so the log handler may use it to provide more sophisticated logic related to handling log messages. Disabling of vendor id must be done in a different way.

How would you go about preventing an empty vendorId from being accepted? I've been criticized about documenting something that should be unsupported but not restricting unsupported behavior (mostly concerning RF24 clones/counterfeits).

[wmarkow]

Disabling of vendor id must be done in a different way.

Just to clarify: by "disabling" I mean the way that vendor id is not written to the destination stream, however a correct value of vendor should be passed in the API macros.

@2bndy5, I have nothing against disabling vendor id, timestamp or other parts of the log message, but it should be done directly in a correct implementation of RF24LogBaseHandler provided by the final application.

Preventing empty vendorId may be not easy to do. Normally I would check the argument and throw exception but on Arduino (AVR) we do not have such u thing. Maybe it should be written somewhere in the docs that a good practice is to provide a valid non empty vendorId. The last way would be to modify RF24Logging::log(...) so it will not pass the log message to the handler when vendorId is null or empty (other way log messages with empty vendorId go to trash).

[2bndy5]

Maybe it should be written somewhere in the docs that a good practice is to provide a valid non empty vendorId

I would like to avoid relying on this. It has become painfully obvious to me that people just don't RTFM (or they don't care to read anything because it isn't a 10 minute youtube video). There's also the language barrier for those that can't fluently read English writing...

The last way would be to modify RF24Logging::log(...) so it will not pass the log message to the handler when vendorId is null or empty

This can't be done on 1 line of code because the vendorId must be a variable stored in flash (according to current implementation). I have started an entry point to adding custom logic about parsing/filtering the vendorId. This is an excerpt from AVR's RF24LogPrintfParser::write():

    PGM_P id = reinterpret_cast<PGM_P>(vendorId);
    char v = pgm_read_byte(id++);
    if (v) {
        appendStr(vendorId);
        appendChar(RF24LOG_DELIMITER);
    }

other way log messages with empty vendorId go to trash

This would seem counter-productive to those that aren't aware of our lib's best practices. I can already foresee the issues complaining about this tactic.

---

I really don't understand the problem with using an empty c-string as a way to disable vendorId (on a per-message basis). I'm only using the null terminating char as a sentinel if there is no char data preceding it. In that sense, it is actually like reserving a special phrase (and position) in the vendorId to trigger a certain behavior (like skipping the vendorId output for that message only). And, this accounts for your undesired behavior to accept/output empty vendorId values.

We definitely cannot just use a conditionally defined macro to disable the vendorId because it isn't dynamic enough for runtime.

a correct value of vendor should be passed in the API macros

Technically, an empty c-string is a correct value because it isn't a int, string, double, etc.

[wmarkow]

@2bndy5 thank you for your detailed comment.

I would like to avoid relying on this. It has become painfully obvious...
This can't be done on 1 line of code because...
This would seem counter-productive to those...

You have absolute right here. My intention with vendorId is to offer the possibility to catch log messages coming from a specific library and process them in some special way. Example: my project uses two libraries RF24 and RF24Network. I want to pass all DEBUG messages from RF24Network to Serial but I'm not interested in all DEBUG from RF24. With the correct (something significant and not empty, like "RF24" for RF24 lib and "RF24Network" for RF24Network library) vendorId in both libraries I can implement the desired logic. If you put an empty vendorId in some places inside of RF24 library then I will not be able to find out where the specific message comes from (in sucha case I would probably redirect it to Serial).

@2bndy5, give me some time - I just want to read the whole issue again to refresh my memory about your scenario. It was about multiline messages, right?

[2bndy5]

I just want to read the whole issue again to refresh my memory about your scenario. It was about multiline messages, right?

😆 It was supposed to be a general discussion about reserving log levels for different purposes. I used multi-line messages as an example, but that led to manipulating the header info when a \n char is reached. And that led to how we could temporarily disable the vendorId because I started hinting at outputting formatted data to a file without using any of the header info...

TBH, the only use case that disabling the vendorId should be used is when the user wants to output formatted data to a file (like JSON) or a general user prompt on the Serial terminal. EDIT: this should be @noted in the docs under the log() function's docs.

Example: my project uses two libraries RF24 and RF24Network. I want to pass all DEBUG messages from RF24Network to Serial but I'm not interested in all DEBUG from RF24.

YES! This is a very common application for python's logging lib.

I have this kind of implemented with the DualStreams example on the refactor branch (please please please checkout those changes). However its not geared towards multiple libraries... hmm. It sounds like manipulating 2 internally configured log levels for the same output stream would be easier than customizing the vendorId parsing. I think #19 would be a step in that direction.

[2bndy5]

Example: my project uses two libraries RF24 and RF24Network. I want to pass all DEBUG messages from RF24Network to Serial but I'm not interested in all DEBUG from RF24.

I'm assuming that this is the last goal that needs resolution for this lib. I'll be exploring some options (locally), but I don't like the idea of comparing vendorIDs. I think the MSBit in a handler's stored log_level could be used as propagation flag. I'll report more when I have a more concrete idea.