Is your feature request related to a problem? Please describe.
I want to monitor the "status" file which is written in regular intervals by apcupsd, which is a plain text file containing dynamic data like power consumption and battery status. This application behaves in a special (but reasonable) way: it keeps the status file open all the time, for each run it seeks to 0 and rewrites the complete file in one go without either closing or truncating the file. Usually the file size does not change. So the Tail input notices that the file content has changed, but since the file size does not go down, this does not trigger the "copytruncate-detection" logic. Net effect is that the file is read only once.
Describe the solution you'd like
This could be solved without affecting other usecases for Tail by implementing a new mode of operation which works in the following way: whenever the file content changes (as determined by inotify or mtime), rewind and re-read the complete file. I think Tail is the right place to implement this because it is just a simple combination of existing logic and the read_from_head mode. This would be configured with a new flag, e.g. "reread_on_change: true"
This may be useful for any application which behaves similarly by regularly rewriting a log/status file, or perhaps even something like a circular log file.
Describe alternatives you've considered
The obvious solution to move the file away by an external process like logrotate does is not effective, because apcupsd keeps the file open and so would write the next status into the moved file. A way I have tried successfully is an external process which regularly truncates the file, but this is ugly and disturbs other processes which also may read that file.
I have not been able to get any useful result trying the Blob input.
An external process which watches the file and on any changes sends it over to a UDP input works. Nonetheless I think the above solution would be more elegant (and consume less resources).
Additional context
This issue can easily reproduced with a program which does the following: open a file, keep it open, in regular intervals seek to 0 and write a constant-length line of text with changing content. Try to monitor that file with Tail.
Is your feature request related to a problem? Please describe.
I want to monitor the "status" file which is written in regular intervals by apcupsd, which is a plain text file containing dynamic data like power consumption and battery status. This application behaves in a special (but reasonable) way: it keeps the status file open all the time, for each run it seeks to 0 and rewrites the complete file in one go without either closing or truncating the file. Usually the file size does not change. So the Tail input notices that the file content has changed, but since the file size does not go down, this does not trigger the "copytruncate-detection" logic. Net effect is that the file is read only once.
Describe the solution you'd like
This could be solved without affecting other usecases for Tail by implementing a new mode of operation which works in the following way: whenever the file content changes (as determined by inotify or mtime), rewind and re-read the complete file. I think Tail is the right place to implement this because it is just a simple combination of existing logic and the read_from_head mode. This would be configured with a new flag, e.g. "reread_on_change: true"
This may be useful for any application which behaves similarly by regularly rewriting a log/status file, or perhaps even something like a circular log file.
Describe alternatives you've considered
The obvious solution to move the file away by an external process like logrotate does is not effective, because apcupsd keeps the file open and so would write the next status into the moved file. A way I have tried successfully is an external process which regularly truncates the file, but this is ugly and disturbs other processes which also may read that file.
I have not been able to get any useful result trying the Blob input.
An external process which watches the file and on any changes sends it over to a UDP input works. Nonetheless I think the above solution would be more elegant (and consume less resources).
Additional context
This issue can easily reproduced with a program which does the following: open a file, keep it open, in regular intervals seek to 0 and write a constant-length line of text with changing content. Try to monitor that file with Tail.