Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update LogRecord batching processor behavior description #4409

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
4 changes: 4 additions & 0 deletions specification/configuration/sdk-environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,8 @@ Depending on the value of `OTEL_TRACES_SAMPLER`, `OTEL_TRACES_SAMPLER_ARG` may b

## Batch Span Processor

Refer to the Trace SDK [Batching processor specification](../trace/sdk/#batching-processor) for more information on these configuration parameters.

| Name | Description | Default | Type | Notes |
|--------------------------------|------------------------------------------------------------------|---------|--------------|-----------------------------------------------------------------------------------|
| OTEL_BSP_SCHEDULE_DELAY | Delay interval (in milliseconds) between two consecutive exports | 5000 | [Duration][] | |
Expand All @@ -217,6 +219,8 @@ Depending on the value of `OTEL_TRACES_SAMPLER`, `OTEL_TRACES_SAMPLER_ARG` may b

## Batch LogRecord Processor

Refer to the Logs SDK [Batching processor specification](../logs/sdk/#batching-processor) for more information on these configuration parameters.

| Name | Description | Default | Type | Notes |
|---------------------------------|------------------------------------------------------------------|---------|--------------|------------------------------------------------------------------------------------|
| OTEL_BLRP_SCHEDULE_DELAY | Delay interval (in milliseconds) between two consecutive exports | 1000 | [Duration][] | |
Expand Down
18 changes: 17 additions & 1 deletion specification/logs/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -421,6 +421,20 @@ representations to the configured `LogRecordExporter`.
The processor MUST synchronize calls to `LogRecordExporter`'s `Export`
to make sure that they are not invoked concurrently.

The processor SHOULD export a batch when any of the following happens AND the
previous export call has returned:

- `scheduledDelayMillis` after the processor is constructed OR the first `LogRecord`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the first LogRecord
is received by the log record processor.

Why export when first record is received 🤔 ?

Copy link
Member

@pellared pellared Feb 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the original author meant "delay after first record'. Probably this could be clarified, but I do not feel it has to be done in this PR.

is received by the log record processor.
Comment on lines +427 to +428
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would we want to have the 2nd part? Asking this question more coming from "why do we want to have complex rules instead of simple rules".

  • the first LogRecord is received by the log record processor.

- `scheduledDelayMillis` after the previous export timer ends, OR the previous
export completes, OR the first `LogRecord` is added to the queue after the previous
export timer ends or previous batch completes.
- The queue contains `maxExportBatchSize` or more `LogRecord`s.
- `ForceFlush` is called.

If the queue is empty when an export is triggered, the processor MAY export
an empty batch OR skip the export and consider it to be completed immediately.

**Configurable parameters:**

* `exporter` - the exporter where the `LogRecord`s are pushed.
Expand All @@ -431,7 +445,9 @@ to make sure that they are not invoked concurrently.
* `exportTimeoutMillis` - how long the export can run before it is cancelled.
The default value is `30000`.
* `maxExportBatchSize` - the maximum batch size of every export. It must be
smaller or equal to `maxQueueSize`. The default value is `512`.
smaller or equal to `maxQueueSize`. If the queue reaches `maxExportBatchSize`
a batch will be exported even if `scheduledDelayMillis` milliseconds have not
elapsed. The default value is `512`.

## LogRecordExporter

Expand Down
Loading