Skip to content

Commit

Permalink
Update pipeline readme
Browse files Browse the repository at this point in the history
  • Loading branch information
vadimalekseev committed Feb 24, 2025
1 parent 39c2fa6 commit 40ac06f
Show file tree
Hide file tree
Showing 2 changed files with 32 additions and 2 deletions.
17 changes: 16 additions & 1 deletion pipeline/README.idoc.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,22 @@ The amount of time the metric can be idle until it is deleted. Used for deleting

**`pool`** *`string`* *`options=std|low_memory`*

Type of `EventPool`. `std` pool is an original pool with the slice of `Event` pointers and slices of free events indicators. `low_memory` pool is a leveled pool based on multiple `sync.Pool` for the events of different size. The latter one is experimental.
Type of `EventPool` that file.d uses to reuse memory.
`std` pool is an original event pool with pre-allocated events at the start of the application.
This pool only frees up memory if event exceeds `avg_log_size`.

`low_memory` event pool based on Go's [sync.Pool](https://pkg.go.dev/sync#Pool) with lazy memory allocation.
It frees up memory depending on the application load - if file.d processes a lot of events, then a lot of memory will be allocated.
If the application load decreases, then the extra events will be freed up in background.

Note that `low_memory` pool increases the load on the garbage collector.
If you are confident in what you are doing, you can change the [GOGC](https://tip.golang.org/doc/gc-guide) (file.d uses GOGC=30 as default value)
environment variable to adjust the frequency of garbage collection – this can reduce the load on the CPU.

Both pools support the `capacity` setting, which both pools use to ensure that they do not exceed the number of allocated events.
This parameter is useful for avoiding OOM.

Default pool is `low_memory`.

<br>

Expand Down
17 changes: 16 additions & 1 deletion pipeline/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,22 @@ The amount of time the metric can be idle until it is deleted. Used for deleting

**`pool`** *`string`* *`options=std|low_memory`*

Type of `EventPool`. `std` pool is an original pool with the slice of `Event` pointers and slices of free events indicators. `low_memory` pool is a leveled pool based on multiple `sync.Pool` for the events of different size. The latter one is experimental.
Type of `EventPool` that file.d uses to reuse memory.
`std` pool is an original event pool with pre-allocated events at the start of the application.
This pool only frees up memory if event exceeds `avg_log_size`.

`low_memory` event pool based on Go's [sync.Pool](https://pkg.go.dev/sync#Pool) with lazy memory allocation.
It frees up memory depending on the application load - if file.d processes a lot of events, then a lot of memory will be allocated.
If the application load decreases, then the extra events will be freed up in background.

Note that `low_memory` pool increases the load on the garbage collector.
If you are confident in what you are doing, you can change the [GOGC](https://tip.golang.org/doc/gc-guide) (file.d uses GOGC=30 as default value)
environment variable to adjust the frequency of garbage collection – this can reduce the load on the CPU.

Both pools support the `capacity` setting, which both pools use to ensure that they do not exceed the number of allocated events.
This parameter is useful for avoiding OOM.

Default pool is `low_memory`.

<br>

Expand Down

0 comments on commit 40ac06f

Please sign in to comment.