Skip to content

pipeline: inputs: forward: style #1725

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 11, 2025
Merged

pipeline: inputs: forward: style #1725

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1c072c2
pipeline: inputs: forward: style
esmerel Jun 11, 2025
37593d0
Apply suggestions from code review
esmerel Jun 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 48 additions & 35 deletions pipeline/inputs/forward.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,52 +3,53 @@
_Forward_ is the protocol used by [Fluent Bit](http://fluentbit.io) and [Fluentd](http://www.fluentd.org) to route messages between peers.
This plugin implements the input service to listen for Forward messages.

## Configuration Parameters
## Configuration parameters

The plugin supports the following configuration parameters:

| Key | Description | Default |
|:--------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| :--- |
| Listen | Listener network interface. | 0.0.0.0 |
| Port | TCP port to listen for incoming connections. | 24224 |
| Unix_Path | Specify the path to unix socket to receive a Forward message. If set, `Listen` and `Port` are ignored. | |
| Unix_Perm | Set the permission of the unix socket file. If `Unix_Path` is not set, this parameter is ignored. | |
| Buffer\_Max\_Size | Specify the maximum buffer memory size used to receive a Forward message. The value must be according to the [Unit Size](../../administration/configuring-fluent-bit/unit-sizes.md) specification. | 6144000 |
| Buffer\_Chunk\_Size | By default the buffer to store the incoming Forward messages, do not allocate the maximum memory allowed, instead it allocate memory when is required. The rounds of allocations are set by _Buffer\_Chunk\_Size_. The value must be according to the [Unit Size ](../../administration/configuring-fluent-bit/unit-sizes.md)specification. | 1024000 |
| Tag_Prefix | Prefix incoming tag with the defined value.| |
| Tag | Override the tag of the forwarded events with the defined value.| |
| Shared\_Key | Shared key for secure forward authentication. | |
| Empty\_Shared\_Key | Use this option to connect to Fluentd with a zero-length shared key. | `false` |
| Self\_Hostname | Hostname for secure forward authentication. | |
| Security.Users | Specify the username and password pairs for secure forward authentication. | |
| Threaded | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |
| Key | Description | Default |
|:----|:------------| :------ |
| `Listen` | Listener network interface. | `0.0.0.0` |
| `Port` | TCP port to listen for incoming connections. | `24224` |
| `Unix_Path` | Specify the path to Unix socket to receive a Forward message. If set, `Listen` and `Port` are ignored. | _none_ |
| `Unix_Perm` | Set the permission of the Unix socket file. If `Unix_Path` isn't set, this parameter is ignored. | _none_ |
| `Buffer_Max_Size` | Specify the maximum buffer memory size used to receive a Forward message. The value must be according to the [Unit Size](../../administration/configuring-fluent-bit/unit-sizes.md) specification. | `6144000` |
| `Buffer_Chunk_Size` | By default the buffer to store the incoming Forward messages, don't allocate the maximum memory allowed, instead it allocate memory when it's required. The rounds of allocations are set by `Buffer_Chunk_Size`. The value must be according to the [Unit Size ](../../administration/configuring-fluent-bit/unit-sizes.md)specification. | `1024000` |
| `Tag_Prefix` | Prefix incoming tag with the defined value.| _none_ |
| `Tag` | Override the tag of the forwarded events with the defined value.| _none_ |
| `Shared_Key` | Shared key for secure forward authentication. | _none_ |
| `Empty_Shared_Key` | Use this option to connect to Fluentd with a zero-length shared key. | `false` |
| `Self_Hostname` | Hostname for secure forward authentication. | _none_ |
| `Security.Users` | Specify the username and password pairs for secure forward authentication. | |
| `Threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |

## Getting Started
## Get started

In order to receive Forward messages, you can run the plugin from the command line or through the configuration file as shown in the following examples.
To receive Forward messages, you can run the plugin from the command line or through the configuration file as shown in the following examples.

### Command Line
### Command line

From the command line you can let Fluent Bit listen for _Forward_ messages with the following options:
From the command line you can let Fluent Bit listen for Forward messages with the following options:

```bash
$ fluent-bit -i forward -o stdout
fluent-bit -i forward -o stdout
```

By default the service will listen an all interfaces \(0.0.0.0\) through TCP port 24224, optionally you can change this directly, e.g:
By default, the service listens on all interfaces (`0.0.0.0`) through TCP port `24224`. You can change this by passing parameters to the command:

```bash
$ fluent-bit -i forward -p listen="192.168.3.2" -p port=9090 -o stdout
fluent-bit -i forward -p listen="192.168.3.2" -p port=9090 -o stdout
```

In the example the Forward messages will only arrive through network interface under 192.168.3.2 address and TCP Port 9090.
In the example, the Forward messages arrive only through network interface `192.168.3.2` address and TCP Port `9090`.

### Configuration File
### Configuration file

In your main configuration file append the following _Input_ & _Output_ sections:
In your main configuration file append the following `Input` and `Output` sections:

{% tabs %}
{% tab title="fluent-bit.conf" %}

```python
[INPUT]
Name forward
Expand All @@ -61,9 +62,11 @@ In your main configuration file append the following _Input_ & _Output_ sections
Name stdout
Match *
```

{% endtab %}

{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
inputs:
Expand All @@ -76,19 +79,21 @@ pipeline:
- name: stdout
match: '*'
```

{% endtab %}
{% endtabs %}

## Fluent Bit + Secure Forward Setup
## Fluent Bit and Secure Forward Setup

Since Fluent Bit v3, in\_forward can handle secure forward protocol.
In Fluent Bit v3 or later, `in_forward` can handle secure forward protocol.

For using user-password authentication, it needs to specify `security.users` at least an one-pair.
For using shared key, it needs to specify `shared_key` in both of forward output and forward input.
`self_hostname` is not able to specify with the same hostname between fluent servers and clients.
For using user-password authentication, specify `security.users` at least an one-pair.
For using shared key, specify `shared_key` in both of forward output and forward input.
`self_hostname` isn't able to specify with the same hostname between fluent servers and clients.

{% tabs %}
{% tab title="fluent-bit-secure-forward.conf" %}

```python
[INPUT]
Name forward
Expand All @@ -104,9 +109,11 @@ For using shared key, it needs to specify `shared_key` in both of forward output
Name stdout
Match *
```

{% endtab %}

{% tab title="fluent-bit-secure-forward.yaml" %}

```yaml
pipeline:
inputs:
Expand All @@ -122,21 +129,27 @@ pipeline:
- name: stdout
match: '*'
```

{% endtab %}
{% endtabs %}

## Testing

Once Fluent Bit is running, you can send some messages using the _fluent-cat_ tool \(this tool is provided by [Fluentd](http://www.fluentd.org):
After Fluent Bit is running, you can send some messages using the `fluent-cat` tool, provided by [Fluentd](http://www.fluentd.org):

```bash
$ echo '{"key 1": 123456789, "key 2": "abcdefg"}' | fluent-cat my_tag
echo '{"key 1": 123456789, "key 2": "abcdefg"}' | fluent-cat my_tag
```

In [Fluent Bit](http://fluentbit.io) we should see the following output:
When you run the plugin with the following command:

```bash
$ bin/fluent-bit -i forward -o stdout
bin/fluent-bit -i forward -o stdout
```

In [Fluent Bit](http://fluentbit.io) you should see the following output:

```text
Fluent-Bit v0.9.0
Copyright (C) Treasure Data

Expand Down