Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(input.tail): Add initial_read_offset config for controlling read behavior #16342

Open
wants to merge 19 commits into
base: master
Choose a base branch
from

Conversation

wen-templari
Copy link

@wen-templari wen-templari commented Dec 22, 2024

Summary

This PR add a initial_read_offset config for tail plugin. This option gives user more control over the read behavior.

  • It can take following options:

    • beginning -- start reading from the beginning of the file ignoring any persisted offset
    • end -- start reading from the end of the file ignoring any persisted offset
    • saved-or-beginning -- use the persisted offset of the file or, if no offset persisted, start from the beginning of the file
    • saved-or-end -- use the persisted offset of the file or, if no offset persisted, start from the end of the file
  • Use saved-or-beginning as default because it matches the previous default from_beginning = false behavior.

  • After setting initial_read_offset , the from_beginning config will be ignored.

  • Change store offset logic. Now the plugin will always store offset ignoring the from_beginning config.

Checklist

  • No AI generated code was used in this PR

Related issues

resolves #16237
replaces #16240

@telegraf-tiger telegraf-tiger bot added area/tail feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin plugin/input 1. Request for new input plugins 2. Issues/PRs that are related to input plugins labels Dec 22, 2024
Copy link
Member

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @wen-templari for your contribution! I do have some comments in the code. Please remember to run make docs after changing the sample.conf file...

Additionally I want to ask you to deprecate the from_beginning option as it is replaced by the new one...

plugins/inputs/tail/sample.conf Outdated Show resolved Hide resolved
plugins/inputs/tail/tail.go Outdated Show resolved Hide resolved
plugins/inputs/tail/tail.go Outdated Show resolved Hide resolved
plugins/inputs/tail/tail_test.go Show resolved Hide resolved
@srebhan srebhan self-assigned this Jan 17, 2025
plugins/inputs/tail/tail.go Outdated Show resolved Hide resolved
plugins/inputs/tail/tail.go Outdated Show resolved Hide resolved
plugins/inputs/tail/tail.go Outdated Show resolved Hide resolved
@wen-templari wen-templari requested a review from srebhan January 21, 2025 03:49
Copy link
Member

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot @wen-templari!

plugins/inputs/tail/README.md Outdated Show resolved Hide resolved
plugins/inputs/tail/sample.conf Outdated Show resolved Hide resolved
plugins/inputs/tail/tail.go Outdated Show resolved Hide resolved
@srebhan
Copy link
Member

srebhan commented Jan 22, 2025

@wen-templari I fixed some small formatting issues. Could you please solve the merge conflict?!?

@wen-templari
Copy link
Author

wen-templari commented Jan 23, 2025

Hi @srebhan, I have resolved the merge conflicts.

@wen-templari wen-templari changed the title feat(input.tail): Add read_start config for controlling read behavior feat(input.tail): Add initial_read_offset config for controlling read behavior Jan 23, 2025
@srebhan srebhan added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label Jan 23, 2025
@srebhan srebhan assigned DStrand1 and unassigned srebhan Jan 23, 2025
@telegraf-tiger
Copy link
Contributor

@wen-templari
Copy link
Author

Hi @srebhan, I updated the code to fix initialization on the InitialReadOffset field. Could you please review it again?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/tail feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin plugin/input 1. Request for new input plugins 2. Issues/PRs that are related to input plugins ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[inputs.tail] Behaviour when from_beginning and statefile coexist is not as expected.
4 participants