Extend RFC3164 parser to support Cisco iOS logs #22
Conversation
These devices seem to send non-standard logs that include a sequence
number, and a couple of extra characters (for good luck, I assume).
For example:
```
10.51.101.233.50433 > 10.51.101.164.514: SYSLOG, length: 112 Facility local7 (23), Severity notice (5)
Msg: 485: *Jan 8 21:50:03.406: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback100, changed state to up
```
Note the `485: *` here. This is the sequence number (and a `*` -- no
idea).
It also includes a ms-precision timestamp.
There was a problem hiding this comment.
Thank you for your contribution!
Aside from the test errors (we'll focus on those later),
I think these sequence changes (not in the RFC3164 AFAIK) should be hidden behind a machine option.
Meaning, I would like to have a WithSequence() or WithCiscoSequence() machine option to expressly change the behavior of the parsing machine.
Can you please iterate on this? 🙏
|
Another thing I'm noticing now is... Where is the priority part in the example message you reported? I never saw (not that I remember) a Syslog message like this one... Shouldn't it be something like the following one? |
|
That seems reasonable. 4b1f2a2 makes this sequence parsing optional. I had to update the test structure a bit to allow options to be passed in -- let me know if you think there's a better way. As for the missing priority, I think that's because I pasted output from tcpdump which itself parses the priority into facility and severity: |
| @@ -131,14 +143,19 @@ action err_content { | |||
|
|
|||
| pri = ('<' prival >mark %from(set_prival) $err(err_prival) '>') @err(err_pri); | |||
|
|
|||
| timestamp = (datemmm sp datemday sp hhmmss) >mark %set_timestamp @err(err_timestamp); | |||
| timestamp = (datemmm sp datemday sp partialtime) >mark %set_timestamp @err(err_timestamp); | |||
There was a problem hiding this comment.
hmmm, where partialtime is coming from?
| @@ -161,7 +178,7 @@ fail := (any - [\n\r])* @err{ fgoto main; }; | |||
|
|
|||
| # note > some BSD syslog implementations insert extra spaces between "PRI", "Timestamp", and "Hostname": although these strictly violate RFC3164, it is useful to be able to parse them | |||
| # note > OpenBSD like many other hardware sends syslog messages without hostname | |||
| main := pri sp* (timestamp | (rfc3339 when { m.rfc3339 })) sp+ (hostname sp+)? msg '\n'?; | |||
| main := pri sp* sequence? (timestamp | (rfc3339 when { m.rfc3339 })) ':'? sp+ (hostname sp+)? msg '\n'?; | |||
There was a problem hiding this comment.
I think that ':'? should be evaluated only when m.sequence is true, too.
Like you did for the sequence rule.
| priority uint8 | ||
| sequence int |
There was a problem hiding this comment.
Why int and not an unsigned type?
These devices seem to send non-standard logs that include a sequence number, and a couple of extra characters (for good luck, I assume).
For example:
Note the
485: *here. This is the sequence number (and a*-- no idea).It also includes a ms-precision timestamp.
This is the first time I've ever written ragel code (first time I've even heard of it), so there's a pretty good chance that I got something wrong, or sub-optimal. I'm happy to make changes as you see fit.