Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HTTP Output GZIP Compression Not Working #16431

Closed
arnelplim opened this issue Jan 25, 2025 · 1 comment
Closed

HTTP Output GZIP Compression Not Working #16431

arnelplim opened this issue Jan 25, 2025 · 1 comment
Labels
bug unexpected problem or unintended behavior

Comments

@arnelplim
Copy link

Relevant telegraf.conf

[agent]
  collection_jitter = "0s"
  debug = false
  flush_interval = "1s"
  flush_jitter = "1s"
  hostname = "EXOS-5520"
  interval = "4s"
  logfile = "./telegraf.log"
  metric_batch_size = 1000
  metric_buffer_limit = 1000
  omit_hostname = true
  precision = ""
  quiet = false
  round_interval = true

[[inputs.file]]
files = ["./inputs/input"]
interval = "2s"
data_format = "influx"
metric_batch_size = 50

[[outputs.http]]
   url = "http://10.68.9.12:8080/"
   method = "POST"
   data_format = "json"
   metric_batch_size = 5
   use_batch_format = true
   content_encoding = "gzip"
   [outputs.http.headers]
   Content-Type = "application/json; charset=utf-8"

Logs from Telegraf

2025-01-24T23:30:09Z I! Loading config: telegraf-gzip.conf
2025-01-24T23:30:09Z I! Starting Telegraf 1.33.1 brought to you by InfluxData the makers of InfluxDB
2025-01-24T23:30:09Z I! Available plugins: 236 inputs, 9 aggregators, 33 processors, 26 parsers, 63 outputs, 6 secret-stores
2025-01-24T23:30:09Z I! Loaded inputs: file
2025-01-24T23:30:09Z I! Loaded aggregators:
2025-01-24T23:30:09Z I! Loaded processors:
2025-01-24T23:30:09Z I! Loaded secretstores:
2025-01-24T23:30:09Z I! Loaded outputs: http
2025-01-24T23:30:09Z I! Tags enabled:
2025-01-24T23:30:09Z I! [agent] Config: Interval:4s, Quiet:false, Hostname:"EXOS-5520", Flush Interval:1s
2025-01-24T23:30:09Z W! [agent] The default value of 'skip_processors_after_aggregators' will change to 'true' with Telegraf v1.40.0! If you need the current default behavior, please explicitly set the option to 'false'!
2025-01-24T23:30:09Z D! [agent] Initializing plugins
2025-01-24T23:30:09Z D! [agent] Connecting outputs
2025-01-24T23:30:09Z D! [agent] Attempting connection to [outputs.http]
2025-01-24T23:30:09Z D! [agent] Successfully connected to outputs.http
2025-01-24T23:30:09Z D! [agent] Starting service inputs
2025-01-24T23:30:10Z D! [outputs.http] Wrote batch of 4 metrics in 2.193124ms
2025-01-24T23:30:10Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:12Z D! [outputs.http] Wrote batch of 4 metrics in 1.867239ms
2025-01-24T23:30:12Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:13Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:15Z D! [outputs.http] Wrote batch of 4 metrics in 1.743923ms
2025-01-24T23:30:15Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:16Z D! [outputs.http] Wrote batch of 4 metrics in 1.640537ms
2025-01-24T23:30:16Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:18Z D! [outputs.http] Wrote batch of 4 metrics in 1.647616ms
2025-01-24T23:30:18Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:20Z D! [outputs.http] Wrote batch of 4 metrics in 1.695444ms
2025-01-24T23:30:20Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:21Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:23Z D! [outputs.http] Wrote batch of 4 metrics in 1.820818ms
2025-01-24T23:30:23Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:25Z D! [outputs.http] Wrote batch of 4 metrics in 1.755888ms
2025-01-24T23:30:25Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:26Z D! [outputs.http] Wrote batch of 4 metrics in 1.540186ms
2025-01-24T23:30:26Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:27Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:29Z D! [outputs.http] Wrote batch of 4 metrics in 1.805931ms
2025-01-24T23:30:29Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:30Z D! [outputs.http] Wrote batch of 4 metrics in 1.650959ms
2025-01-24T23:30:30Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:31Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:31Z D! [agent] Stopping service inputs
2025-01-24T23:30:31Z D! [agent] Input channel closed
2025-01-24T23:30:31Z I! [agent] Hang on, flushing any cached metrics before shutdown
2025-01-24T23:30:31Z D! [outputs.http] Buffer fullness: 0 / 1000 metrics
2025-01-24T23:30:31Z I! [agent] Stopping running outputs
2025-01-24T23:30:31Z D! [agent] Stopped Successfully

System info

Telegraf 1.33.1 (git: HEAD@44f3a504) Linux calabash 6.5.0-35-generic #35~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue May 7 09:00:52 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

Docker

No response

Steps to reproduce

  1. Configure http output plugin with gzip content_encoding
  2. Start debug webserver that prints encoding and dumps payload
  3. Start telegraf reading JSON input from file.
    NOTE: telegraf version downloaded directly via:
    wget https://dl.influxdata.com/telegraf/releases/telegraf-1.33.1_linux_amd64.tar.gz
./telegraf-1.33.1 --config telegraf-gzip.conf
  1. See that compressed payload at HTTP server is empty.
10.68.9.12 - - [24/Jan/2025 23:24:11] "POST / HTTP/1.1" 200 -
('content_length:\n', 0)
('content_encoding:\n', 'gzip')
('content_location:\n', '')
---------- gzip ---------

('Decompressed Data:\n', u'')
10.68.9.12 - - [24/Jan/2025 23:24:22] "POST / HTTP/1.1" 200 -
('content_length:\n', 0)
('content_encoding:\n', 'gzip')
('content_location:\n', '')
---------- gzip ---------

('Decompressed Data:\n', u'')
  1. Re-configure http output plug WITHOUT gzip content_encoding not set. (Default)
  2. Start telegraf reading same JSON input from file as above
  3. See that UNCOMPRESSED payload at HTTP server is correct.
('content_length:\n', 1097)
('content_encoding:\n', '')
('content_location:\n', '')
++++++ NOT COMPRESSED ----

('Received Data:\n', u'{"metrics":[{"fields":{"@1_portList_tags":"1:1","@2_portList_tags":"1:7","@3_portList_tags":"1:13","ipAddress":"10.1.2.3","macAddr":"00:73:04:05:06:07","vlanId_key":10},"name":"MyStats","tags":{"reporterSerialnum":"\\"XYZ-5678\\"","serialnum":"\\"ABCD-1234\\""},"timestamp":1737762308},{"fields":{"@1_portList_tags":"1:2","@2_portList_tags":"1:9","@3_portList_tags":"1:4","ipAddress":"10.33.33.3","macAddr":"00:73:55:55:06:07","vlanId_key":33},"name":"MyStats","tags":{"reporterSerialnum":"\\"XYZ-5678\\"","serialnum":"\\"ABCD-1234\\""},"timestamp":1737762308},{"fields":{"@1_portList_tags":"1:1","@2_portList_tags":"1:7","@3_portList_tags":"1:13","ipAddress":"10.1.2.99","macAddr":"00:73:04:33:44:55","vlanId_key":40},"name":"MyStats","tags":{"reporterSerialnum":"\\"XYZ-5678\\"","serialnum":"\\"WXYZ-5678\\""},"timestamp":1737762308},{"fields":{"@1_portList_tags":"1:2","@2_portList_tags":"1:9","@3_portList_tags":"1:4","ipAddress":"10.33.33.3","macAddr":"00:88:88:55:06:88","vlanId_key":88},"name":"MyStats","tags":{"reporterSerialnum":"\\"XYZ-5678\\"","serialnum":"\\"ABCD-1234\\""},"timestamp":1737762308}]}\n')
10.68.9.12 - - [24/Jan/2025 23:45:09] "POST / HTTP/1.1" 200 -

Test Input File Being Read

ElrpStats,serialnum="ABCD-1234",reporterSerialnum="XYZ-5678" vlanId_key=10,ipAddress="10.1.2.3",macAddr="00:73:04:05:06:07",@1_portList_tags="1:1",@2_portList_tags="1:7",@3_portList_tags="1:13"
ElrpStats,serialnum="ABCD-1234",reporterSerialnum="XYZ-5678" vlanId_key=33,ipAddress="10.33.33.3",macAddr="00:73:55:55:06:07",@1_portList_tags="1:2",@2_portList_tags="1:9",@3_portList_tags="1:4"
ElrpStats,serialnum="WXYZ-5678",reporterSerialnum="XYZ-5678" vlanId_key=40,ipAddress="10.1.2.99",macAddr="00:73:04:33:44:55",@1_portList_tags="1:1",@2_portList_tags="1:7",@3_portList_tags="1:13"
ElrpStats,serialnum="ABCD-1234",reporterSerialnum="XYZ-5678" vlanId_key=88,ipAddress="10.33.33.3",macAddr="00:88:88:55:06:88",@1_portList_tags="1:2",@2_portList_tags="1:9",@3_portList_tags="1:4"

Expected behavior

I would expect server to show content_length to be non-zero and conent_encoding seen as gzip

i.e.

('content_length:\n', 285)
('content_encoding:\n', 'gzip')


### Actual behavior

Looks like when content_encoding is set to "gzip" in telegraf config, the payload sent out is empty.

('content_length:\n', 0)
('content_encoding:\n', 'gzip')

Additional info

I built a version of telegraf (v1.21 branch) with the following changes to http.go, gzip and compression works fine.

('content_length:\n', 285)
('content_encoding:\n', 'gzip')
('content_location:\n', '')
---------- gzip ---------

('Decompressed Data:\n', u'{"metrics":[{"fields":{"@1_portList_tags":"1:1","@2_portList_tags":"1:7","@3_portList_tags":"1:13","ipAddress":"10.1.2.3","macAddr":"00:73:04:05:06:07","vlanId_key":10},"name":"MyStats","tags":{"reporterSerialnum":"\\"XYZ-5678\\"","serialnum":"\\"ABCD-1234\\""},"timestamp":1737762754},{"fields":{"@1_portList_tags":"1:2","@2_portList_tags":"1:9","@3_portList_tags":"1:4","ipAddress":"10.33.33.3","macAddr":"00:73:55:55:06:07","vlanId_key":33},"name":"MyStats","tags":{"reporterSerialnum":"\\"XYZ-5678\\"","serialnum":"\\"ABCD-1234\\""},"timestamp":1737762754},{"fields":{"@1_portList_tags":"1:1","@2_portList_tags":"1:7","@3_portList_tags":"1:13","ipAddress":"10.1.2.99","macAddr":"00:73:04:33:44:55","vlanId_key":40},"name":"MyStats","tags":{"reporterSerialnum":"\\"XYZ-5678\\"","serialnum":"\\"WXYZ-5678\\""},"timestamp":1737762754},{"fields":{"@1_portList_tags":"1:2","@2_portList_tags":"1:9","@3_portList_tags":"1:4","ipAddress":"10.33.33.3","macAddr":"00:88:88:55:06:88","vlanId_key":88},"name":"MyStats","tags":{"reporterSerialnum":"\\"XYZ-5678\\"","serialnum":"\\"ABCD-1234\\""},"timestamp":1737762754}]}')
10.68.9.12 - - [24/Jan/2025 23:52:35] "POST / HTTP/1.1" 200 -

Changes I made to to http.go (telegraf.1.21). I created a local version of CompressWithGzip() and I compress reqBody first, before doing reqBodyBuffer io.Reader = bytes.NewBuffer(reqBody).

diff --git a/plugins/outputs/http/http.go b/plugins/outputs/http/http.go
index d61796fba..808c7e29e 100644
--- a/plugins/outputs/http/http.go
+++ b/plugins/outputs/http/http.go
@@ -9,6 +9,7 @@ import (
        "net/http"
        "strings"
        "sync/atomic"
+       "compress/gzip"
 
        "github.com/influxdata/telegraf"
        "github.com/influxdata/telegraf/internal"
@@ -187,45 +188,40 @@ func (h *HTTP) Write(metrics []telegraf.Metric) error {
        return nil
 }
 
+func CompressWithGzip(data []byte) ([]byte, error) {
+        var buf bytes.Buffer
+        gz := gzip.NewWriter(&buf)
+        _, err := gz.Write(data)
+        if err != nil {
+                return nil, err
+        }
+        if err := gz.Close(); err != nil {
+                return nil, err
+        }
+        return buf.Bytes(), nil
 }
 
 func (h *HTTP) writeMetric(reqBody []byte) error {
-       var reqBodyBuffer io.Reader = bytes.NewBuffer(reqBody)
 
        var err error
 
        if h.ContentEncoding == "gzip" {
-               rc, err := internal.CompressWithGzip(reqBodyBuffer)
+               rc, err := CompressWithGzip(reqBody)
                if err != nil {
                        return err
                }
-               defer rc.Close()
-               reqBodyBuffer = rc
+
+               reqBody = rc
        }
+       var reqBodyBuffer io.Reader = bytes.NewBuffer(reqBody)
        
        req, err := http.NewRequest(h.Method, h.URL, reqBodyBuffer)
        if err != nil {
@@ -247,7 +243,9 @@ func (h *HTTP) writeMetric(reqBody []byte) error {
                }
                req.Header.Set(k, v)
        }
-       
+                       
+
        resp, err := h.client.Do(req)
        if err != nil {
@arnelplim arnelplim added the bug unexpected problem or unintended behavior label Jan 25, 2025
@arnelplim
Copy link
Author

My bad. User error. My test webserver was looking at Content-Length to decompress which was 0. Disregard this reported issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug unexpected problem or unintended behavior
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@arnelplim