Cannot establish connection to Kafka with SSL protection on output

[jori-be]

Bug Report

Describe the bug

When using the kafka output on fluent-bit, I cannot make the it work to output the logs to kafka.
This kafka is protected with an self signed certificate.

I know that the certificates that I provided are working, as I tried an another log agent before.

[2025/02/20 15:48:05] [error] [output:kafka:kafka.0] fluent-bit#producer-1: [thrd:ssl://kafka01.example.com:9093/bootstrap]: ssl://kafka01.example.com:9093/bootstrap: SSL handshake failed: error:0A000086:SSL routines::certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (install ca-certificates package) (after 22ms in state SSL_HANDSHAKE, 13 identical error(s) suppressed)

Config:
[SERVICE]
    HTTP_Server  On
    HTTP_Listen  0.0.0.0
    HTTP_PORT    2020
    Health_Check On 
    HC_Errors_Count 5 
    HC_Retry_Failure_Count 5 
    HC_Period 5
    Log_Level debug
    
    Parsers_File /fluent-bit/parsers/parsers.conf
[INPUT]
    Name              tail
    Tag               someapp
    Path              /var/log/pods/*/*/*.log
    DB                /var/log/flb_kube.db
    Parser            json
    multiline.parser  docker, cri
    Mem_Buf_Limit     5MB
    Skip_Long_Lines   On
    Refresh_Interval  10
[OUTPUT]
    Name kafka
    Match *
    Brokers kafka01.example.com:9093,kafka02.example.com:9093,kafka03.example.com:9093
    Topics eks-fluent-bit
    rdkafka.security.protocol ssl
    rdkafka.ssl.ca.location /etc/secrets/ca.crt
    rdkafka.ssl.certificate.location /etc/secrets/tls.crt
    rdkafka.ssl.key.location /etc/secrets/tls.key

To Reproduce

1. Install config
2. Create self signed certificates
3. Install them on fluent-bit and kafka
4. Start fluent-bit and notice the error in the logs

• Example log message:

`[2025/02/20 15:48:05] [error] [output:kafka:kafka.0] fluent-bit#producer-1: [thrd:ssl://kafka01.example.com:9093/bootstrap]: ssl://kafka01.example.com:9093/bootstrap: SSL handshake failed: error:0A000086:SSL routines::certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (install ca-certificates package) (after 22ms in state SSL_HANDSHAKE, 13 identical error(s) suppressed)`

• Steps to reproduce the problem:

Expected behavior

Fluent-bit should be able to connect to kafka over ssl.

Your Environment

• Version used: I used 1.9.10 & also latest version 3.2.6
• Running in docker container and deployed via helm in EKS

Additional context

My goal is to harvest logs of pods in EKS and push the logs to Kafka on prem which is protected via ssl certificate.