email configuration when using postfix on docker host #6822

ahmgithubahm · 2024-03-21T15:40:32Z

ahmgithubahm
Mar 21, 2024

I am setting up redash on a Debian 12 server, manually running the equivalent of the setup.sh file. The redash UI itself is working fine, but I now need to get outbound email working. We have postfix running on the docker host, so I want redash to talk to that. So, the default settings, which include:
REDASH_MAIL_SERVER=localhost
would appear, naively, to be fine. However, I suspect that localhost won't mean what I think when referenced inside the redash server container. Certainly, this doesn't work:

$ docker compose run --rm server manage send_test_mail
[+] Creating 2/0
 ✔ Container redash-postgres-1  Running                                                                                                                                               0.0s
 ✔ Container redash-redis-1     Running                                                                                                                                               0.0s
Traceback (most recent call last):
  File "/app/manage.py", line 9, in <module>
    manager()
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/flask/cli.py", line 380, in main
    return AppGroup.main(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/flask/cli.py", line 257, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/app/redash/cli/__init__.py", line 72, in send_test_mail
    body="Test message."))
  File "/usr/local/lib/python2.7/site-packages/flask_mail.py", line 491, in send
    with self.connect() as connection:
  File "/usr/local/lib/python2.7/site-packages/flask_mail.py", line 144, in __enter__
    self.host = self.configure_host()
  File "/usr/local/lib/python2.7/site-packages/flask_mail.py", line 158, in configure_host
    host = smtplib.SMTP(self.mail.server, self.mail.port)
  File "/usr/local/lib/python2.7/smtplib.py", line 256, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/local/lib/python2.7/smtplib.py", line 317, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/local/lib/python2.7/smtplib.py", line 292, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "/usr/local/lib/python2.7/socket.py", line 575, in create_connection
    raise err
socket.error: [Errno 99] Cannot assign requested address

Google-fu has lead me to try (in docker-compose.yml):

services:
  server:
    .....
    extra_hosts:
      - "host.docker.internal:host-gateway"

and in the redash env file:

REDASH_MAIL_SERVER=host.docker.internal

and this gives a different error:

$ docker compose run --rm server manage send_test_mail
[+] Creating 2/0
 ✔ Container redash-postgres-1  Running                                                                                                                                               0.0s
 ✔ Container redash-redis-1     Running                                                                                                                                               0.0s
Traceback (most recent call last):
  File "/app/manage.py", line 9, in <module>
    manager()
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/flask/cli.py", line 380, in main
    return AppGroup.main(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/flask/cli.py", line 257, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/app/redash/cli/__init__.py", line 72, in send_test_mail
    body="Test message."))
  File "/usr/local/lib/python2.7/site-packages/flask_mail.py", line 491, in send
    with self.connect() as connection:
  File "/usr/local/lib/python2.7/site-packages/flask_mail.py", line 144, in __enter__
    self.host = self.configure_host()
  File "/usr/local/lib/python2.7/site-packages/flask_mail.py", line 158, in configure_host
    host = smtplib.SMTP(self.mail.server, self.mail.port)
  File "/usr/local/lib/python2.7/smtplib.py", line 256, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/local/lib/python2.7/smtplib.py", line 317, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/local/lib/python2.7/smtplib.py", line 292, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "/usr/local/lib/python2.7/socket.py", line 575, in create_connection
    raise err
socket.error: [Errno 111] Connection refused

Any ideas?

Answered by ahmgithubahm

Mar 21, 2024

Ah, I think, instead of deleting this question, I'll leave the answer here, since I found it almost straight away.

My host's postfix config needed some changes too. Firstly, I needed:

inet_interfaces = all

instead of the default (on Debian 12 at least) of:

inet_interfaces = loopback-only

I also needed to add:

172.16.0.0/12

to the mynetworks line, since the SMTP connections from docker containers will be coming from that CIDR.

With these changes, sending a test mail from the server container worked. I then had to add the same 'extra_hosts' config to the 'scheduler', 'scheduled_worker' and 'adhoc_worker' container config, so that sending invitation and password reset emails worked too. I…

View full answer

ahmgithubahm · 2024-03-21T16:42:50Z

ahmgithubahm
Mar 21, 2024
Author

Ah, I think, instead of deleting this question, I'll leave the answer here, since I found it almost straight away.

My host's postfix config needed some changes too. Firstly, I needed:

inet_interfaces = all

instead of the default (on Debian 12 at least) of:

inet_interfaces = loopback-only

I also needed to add:

172.16.0.0/12

to the mynetworks line, since the SMTP connections from docker containers will be coming from that CIDR.

With these changes, sending a test mail from the server container worked. I then had to add the same 'extra_hosts' config to the 'scheduler', 'scheduled_worker' and 'adhoc_worker' container config, so that sending invitation and password reset emails worked too. I'm not sure whether all three needed it, but, well, why not, for covering any eventualities. 😄

0 replies

justinclift · 2024-03-21T20:15:26Z

justinclift
Mar 21, 2024
Maintainer

Cool, that sounds a like useful way to do it.

One thought that pops to mind is that the inet_interfaces = all line is probably telling postfix to allow connections from every network interface.

So it might be good to add some firewall rules (or security group settings in AWS) to make sure nothing external to the host can reach it.

0 replies

ahmgithubahm · 2024-03-21T21:56:41Z

ahmgithubahm
Mar 21, 2024
Author

Thanks for the reply, @justinclift . Indeed, I forgot to mention, this is on AWS, and the security groups we have in force do not allow inbound port 25 to the host server.

0 replies

justinclift · 2024-03-21T22:24:01Z

justinclift
Mar 21, 2024
Maintainer

As a general thought, another way of getting email working is to use an external provider and send the mail through them. I've personally been using SMTP2Go, and they've been super reliable. 😄

These are the env file settings needed for that to work:

REDASH_MAIL_SERVER=mail.smtp2go.com
[email protected]
REDASH_MAIL_PORT=2525
REDASH_MAIL_USE_TLS=yes
REDASH_MAIL_USERNAME=your.smtp2go.username
REDASH_MAIL_PASSWORD=your.smtp2go.password

As a special note, don't bother using SendGrid (sendgrid.com). First place I tried, but they're blocked by a lot of places for being a source of spam, so they're completely unreliable. ☹️

2 replies

ahmgithubahm Mar 22, 2024
Author

Thanks for this too, Justin. I also omitted to mention that our standard Postfix config on hosts such as this uses the mail relaying service we use (Mimecast) rather than directly emailing out. For personal stuff I use AuthSMTP, who I also find to be good.

I prefer to have a standard host-based Postfix, and then apps can all just splurge to localhost:25 without having to worry about any other config.

justinclift Mar 22, 2024
Maintainer

No worries at all. 😄

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

email configuration when using postfix on docker host #6822

{{title}}

Replies: 4 comments 2 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

email configuration when using postfix on docker host #6822

ahmgithubahm Mar 21, 2024

Replies: 4 comments · 2 replies

ahmgithubahm Mar 21, 2024 Author

justinclift Mar 21, 2024 Maintainer

ahmgithubahm Mar 21, 2024 Author

justinclift Mar 21, 2024 Maintainer

ahmgithubahm Mar 22, 2024 Author

justinclift Mar 22, 2024 Maintainer

ahmgithubahm
Mar 21, 2024

Replies: 4 comments 2 replies

ahmgithubahm
Mar 21, 2024
Author

justinclift
Mar 21, 2024
Maintainer

ahmgithubahm
Mar 21, 2024
Author

justinclift
Mar 21, 2024
Maintainer

ahmgithubahm Mar 22, 2024
Author

justinclift Mar 22, 2024
Maintainer