Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blank /debug page after upgrade to latest version #5928

Closed
5 tasks done
delendum opened this issue Jun 29, 2024 · 49 comments · Fixed by #5951
Closed
5 tasks done

Blank /debug page after upgrade to latest version #5928

delendum opened this issue Jun 29, 2024 · 49 comments · Fixed by #5951

Comments

@delendum
Copy link

Contribution guidelines

I've found a bug and checked that ...

  • ... I understand that not following the below instructions will result in immediate closure and/or deletion of my issue.
  • ... I have understood that this bug report is dedicated for bugs, and not for support-related inquiries.
  • ... I have understood that answers are voluntary and community-driven, and not commercial support.
  • ... I have verified that my issue has not been already answered in the past. I also checked previous issues.

Description

After upgrading from 2024-04 to 2024-06a, the /debug page is now blank. It appears to hang for a few seconds before returning a blank page.

Logs:

While waiting for /debug to load, changing to /admin sometimes catches a glimpse of this error:

An unknown error occured: TypeError Object( [message:protected] => Cannot access offset of type string on string [string:Error:private] => [code:protected] => 0 [file:protected] => /web/debug.php [line:protected] => 32 [trace:Error:private] => Array ( ) [previous:Error:private] => )

Logs are otherwise uneventful:
php-fpm-mailcow-1    | 172.22.1.8 -  29/Jun/2024:21:07:56 +1200 "GET /debug.php" 200
nginx-mailcow-1      | 192.168.1.5 - - [29/Jun/2024:21:08:07 +1200] "GET /debug HTTP/1.1" 200 31 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0"

Steps to reproduce:

1. Upgrade to 2024-06a
2. Log in with the admin username
3. Blank /debug page is returned

I have tried looking after some of the things that had caused similar issues in the past:
1. My TZ is: TZ=Pacific/Auckland
2. The cache folder exists:
drwxr-xr-x 22   82   82  4096 Jun 29 21:07 cache

Which branch are you using?

master

Which architecture are you using?

x86

Operating System:

Debian 12

Server/VM specifications:

VM: 32GB RAM, 4 Cores

Is Apparmor, SELinux or similar active?

Yes

Virtualization technology:

Virtualbox

Docker version:

27.0.2, build 912c1dd

docker-compose version or docker compose version:

v2.28.1

mailcow version:

2024-06a

Reverse proxy:

HAProxy

Logs of git diff:

diff --git a/data/conf/postfix/main.cf b/data/conf/postfix/main.cf
index 6a87f2ec..c9ede2d1 100644
--- a/data/conf/postfix/main.cf
+++ b/data/conf/postfix/main.cf
@@ -173,3 +173,33 @@ parent_domain_matches_subdomains = debug_peer_list,fast_flush_domains,mynetworks
 
 # DO NOT EDIT ANYTHING BELOW #
 # Overrides #
+
+postscreen_dnsbl_sites = wl.mailspike.net=127.0.0.[18;19;20]*-2
+  hostkarma.junkemailfilter.com=127.0.0.1*-2
+  list.dnswl.org=127.0.[0..255].0*-2
+  list.dnswl.org=127.0.[0..255].1*-4
+  list.dnswl.org=127.0.[0..255].2*-6
+  list.dnswl.org=127.0.[0..255].3*-8
+  ix.dnsbl.manitu.net*2
+  bl.spamcop.net*2
+  bl.suomispam.net*2
+  hostkarma.junkemailfilter.com=127.0.0.2*3
+  hostkarma.junkemailfilter.com=127.0.0.4*2
+  hostkarma.junkemailfilter.com=127.0.1.2*1
+  backscatter.spameatingmonkey.net*2
+  bl.ipv6.spameatingmonkey.net*2
+  bl.spameatingmonkey.net*2
+  b.barracudacentral.org=127.0.0.2*7
+  bl.mailspike.net=127.0.0.2*5
+  bl.mailspike.net=127.0.0.[10;11;12]*4
+  dnsbl.sorbs.net=127.0.0.10*8
+  dnsbl.sorbs.net=127.0.0.5*6
+  dnsbl.sorbs.net=127.0.0.7*3
+  dnsbl.sorbs.net=127.0.0.8*2
+  dnsbl.sorbs.net=127.0.0.6*2
+  dnsbl.sorbs.net=127.0.0.9*2
+
+
+# User Overrides
+myhostname = mail.<redacted>.com
+
diff --git a/docker-compose.yml b/docker-compose.yml
index b0f50dd6..563144af 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -611,36 +611,6 @@ services:
           aliases:
             - ofelia
 
-    ipv6nat-mailcow:
-      depends_on:
-        - unbound-mailcow
-        - mysql-mailcow
-        - redis-mailcow
-        - clamd-mailcow
-        - rspamd-mailcow
-        - php-fpm-mailcow
-        - sogo-mailcow
-        - dovecot-mailcow
-        - postfix-mailcow
-        - memcached-mailcow
-        - nginx-mailcow
-        - acme-mailcow
-        - netfilter-mailcow
-        - watchdog-mailcow
-        - dockerapi-mailcow
-        - solr-mailcow
-      environment:
-        - TZ=${TZ}
-      image: robbertkl/ipv6nat
-      security_opt:
-        - label=disable
-      restart: always
-      privileged: true
-      network_mode: "host"
-      volumes:
-        - /var/run/docker.sock:/var/run/docker.sock:ro
-        - /lib/modules:/lib/modules:ro
-
 networks:
   mailcow-network:
     driver: bridge

Logs of iptables -L -vn:

Chain INPUT (policy DROP 70 packets, 4086 bytes)
 pkts bytes target     prot opt in     out     source               destination         
19126 4188K MAILCOW    0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* mailcow */
27978 6512K ufw-before-logging-input  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
27978 6512K ufw-before-input  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
  278 33518 ufw-after-input  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
   70  4086 ufw-after-logging-input  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
   70  4086 ufw-reject-input  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
   70  4086 ufw-track-input  0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 148K   35M MAILCOW    0    --  *      *       0.0.0.0/0            0.0.0.0/0            /* mailcow */
 147K   35M DOCKER-USER  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
 147K   35M DOCKER-ISOLATION-STAGE-1  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
 110K   31M ACCEPT     0    --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
13094  832K DOCKER     0    --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0           
23785 2964K ACCEPT     0    --  br-mailcow !br-mailcow  0.0.0.0/0            0.0.0.0/0           
10343  667K ACCEPT     0    --  br-mailcow br-mailcow  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     0    --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     0    --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     0    --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     0    --  docker0 docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ufw-before-logging-forward  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ufw-before-forward  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ufw-after-forward  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ufw-after-logging-forward  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ufw-reject-forward  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ufw-track-forward  0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 36 packets, 1704 bytes)
 pkts bytes target     prot opt in     out     source               destination         
28703   15M ufw-before-logging-output  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
28703   15M ufw-before-output  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
  213 12759 ufw-after-output  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
  213 12759 ufw-after-logging-output  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
  213 12759 ufw-reject-output  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
  213 12759 ufw-track-output  0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.249         tcp dpt:6379
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.5           tcp dpt:8983
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.6           tcp dpt:3306
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:110
   22  1296 ACCEPT     6    --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:143
    5   386 ACCEPT     6    --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:993
   14   836 ACCEPT     6    --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:995
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:4190
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:12345
 2599  156K ACCEPT     6    --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.8           tcp dpt:80
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.8           tcp dpt:443
   39  2016 ACCEPT     6    --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:25
   67  3984 ACCEPT     6    --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:465
    5   300 ACCEPT     6    --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:587

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
23785 2964K DOCKER-ISOLATION-STAGE-2  0    --  br-mailcow !br-mailcow  0.0.0.0/0            0.0.0.0/0           
    0     0 DOCKER-ISOLATION-STAGE-2  0    --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
 191K   55M RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       0    --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0           
    0     0 DROP       0    --  *      docker0  0.0.0.0/0            0.0.0.0/0           
30966 7666K RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 191K   55M RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain MAILCOW (2 references)
 pkts bytes target     prot opt in     out     source               destination         
   93  7483 DROP       0    --  *      *       194.169.175.66       0.0.0.0/0           
  185 11040 DROP       0    --  *      *       194.169.175.20       0.0.0.0/0           
    0     0 DROP       6    --  !br-mailcow br-mailcow  0.0.0.0/0            0.0.0.0/0            /* mailcow isolation */

Chain ufw-after-forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw-after-input (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ufw-skip-to-policy-input  17   --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:137
    0     0 ufw-skip-to-policy-input  17   --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:138
    0     0 ufw-skip-to-policy-input  6    --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:139
    0     0 ufw-skip-to-policy-input  6    --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:445
    0     0 ufw-skip-to-policy-input  17   --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:67
    0     0 ufw-skip-to-policy-input  17   --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:68
  208 29432 ufw-skip-to-policy-input  0    --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type BROADCAST

Chain ufw-after-logging-forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 LOG        0    --  *      *       0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW BLOCK] "

Chain ufw-after-logging-input (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   23  1380 LOG        0    --  *      *       0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW BLOCK] "

Chain ufw-after-logging-output (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw-after-output (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw-before-forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     0    --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 ACCEPT     1    --  *      *       0.0.0.0/0            0.0.0.0/0            icmptype 3
    0     0 ACCEPT     1    --  *      *       0.0.0.0/0            0.0.0.0/0            icmptype 11
    0     0 ACCEPT     1    --  *      *       0.0.0.0/0            0.0.0.0/0            icmptype 12
    0     0 ACCEPT     1    --  *      *       0.0.0.0/0            0.0.0.0/0            icmptype 8
    0     0 ufw-user-forward  0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain ufw-before-input (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   23  2335 ACCEPT     0    --  lo     *       0.0.0.0/0            0.0.0.0/0           
27648 6474K ACCEPT     0    --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    4   160 ufw-logging-deny  0    --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate INVALID
    4   160 DROP       0    --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate INVALID
    0     0 ACCEPT     1    --  *      *       0.0.0.0/0            0.0.0.0/0            icmptype 3
    0     0 ACCEPT     1    --  *      *       0.0.0.0/0            0.0.0.0/0            icmptype 11
    0     0 ACCEPT     1    --  *      *       0.0.0.0/0            0.0.0.0/0            icmptype 12
    0     0 ACCEPT     1    --  *      *       0.0.0.0/0            0.0.0.0/0            icmptype 8
    0     0 ACCEPT     17   --  *      *       0.0.0.0/0            0.0.0.0/0            udp spt:67 dpt:68
  303 35018 ufw-not-local  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     17   --  *      *       0.0.0.0/0            224.0.0.251          udp dpt:5353
    0     0 ACCEPT     17   --  *      *       0.0.0.0/0            239.255.255.250      udp dpt:1900
  303 35018 ufw-user-input  0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain ufw-before-logging-forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw-before-logging-input (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw-before-logging-output (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw-before-output (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   23  2335 ACCEPT     0    --  *      lo      0.0.0.0/0            0.0.0.0/0           
28467   14M ACCEPT     0    --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
  213 12759 ufw-user-output  0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain ufw-logging-allow (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 LOG        0    --  *      *       0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW ALLOW] "

Chain ufw-logging-deny (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    4   160 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate INVALID limit: avg 3/min burst 10
    0     0 LOG        0    --  *      *       0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW BLOCK] "

Chain ufw-not-local (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   86  5136 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL
    9   450 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type MULTICAST
  208 29432 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type BROADCAST
    0     0 ufw-logging-deny  0    --  *      *       0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 10
    0     0 DROP       0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain ufw-reject-forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw-reject-input (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw-reject-output (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw-skip-to-policy-forward (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain ufw-skip-to-policy-input (7 references)
 pkts bytes target     prot opt in     out     source               destination         
  208 29432 DROP       0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain ufw-skip-to-policy-output (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain ufw-track-forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw-track-input (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw-track-output (1 references)
 pkts bytes target     prot opt in     out     source               destination         
  134  8048 ACCEPT     6    --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate NEW
   43  3007 ACCEPT     17   --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate NEW

Chain ufw-user-forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw-user-limit (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 LOG        0    --  *      *       0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 5 LOG flags 0 level 4 prefix "[UFW LIMIT BLOCK] "
    0     0 REJECT     0    --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable

Chain ufw-user-limit-accept (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    4   240 ACCEPT     0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain ufw-user-logging-forward (0 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw-user-logging-input (0 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw-user-logging-output (0 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw-user-output (1 references)
 pkts bytes target     prot opt in     out     source               destination

Logs of ip6tables -L -vn:

Chain INPUT (policy DROP 111 packets, 7770 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 1072  997K MAILCOW    0    --  *      *       ::/0                 ::/0                 /* mailcow */
 1474 1241K ufw6-before-logging-input  0    --  *      *       ::/0                 ::/0                
 1474 1241K ufw6-before-input  0    --  *      *       ::/0                 ::/0                
  111  7770 ufw6-after-input  0    --  *      *       ::/0                 ::/0                
  111  7770 ufw6-after-logging-input  0    --  *      *       ::/0                 ::/0                
  111  7770 ufw6-reject-input  0    --  *      *       ::/0                 ::/0                
  111  7770 ufw6-track-input  0    --  *      *       ::/0                 ::/0                

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
25125   30M MAILCOW    0    --  *      *       ::/0                 ::/0                 /* mailcow */
25152   30M DOCKER-USER  0    --  *      *       ::/0                 ::/0                
25152   30M DOCKER-ISOLATION-STAGE-1  0    --  *      *       ::/0                 ::/0                
18184   30M ACCEPT     0    --  *      br-mailcow  ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
 6968  477K DOCKER     0    --  *      br-mailcow  ::/0                 ::/0                
    0     0 ACCEPT     0    --  br-mailcow !br-mailcow  ::/0                 ::/0                
 6968  477K ACCEPT     0    --  br-mailcow br-mailcow  ::/0                 ::/0                
    0     0 ACCEPT     0    --  *      docker0  ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
    0     0 DOCKER     0    --  *      docker0  ::/0                 ::/0                
    0     0 ACCEPT     0    --  docker0 !docker0  ::/0                 ::/0                
    0     0 ACCEPT     0    --  docker0 docker0  ::/0                 ::/0                
    0     0 ufw6-before-logging-forward  0    --  *      *       ::/0                 ::/0                
    0     0 ufw6-before-forward  0    --  *      *       ::/0                 ::/0                
    0     0 ufw6-after-forward  0    --  *      *       ::/0                 ::/0                
    0     0 ufw6-after-logging-forward  0    --  *      *       ::/0                 ::/0                
    0     0 ufw6-reject-forward  0    --  *      *       ::/0                 ::/0                
    0     0 ufw6-track-forward  0    --  *      *       ::/0                 ::/0                

Chain OUTPUT (policy ACCEPT 78 packets, 7768 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 2903  896K ufw6-before-logging-output  0    --  *      *       ::/0                 ::/0                
 2903  896K ufw6-before-output  0    --  *      *       ::/0                 ::/0                
  210 17218 ufw6-after-output  0    --  *      *       ::/0                 ::/0                
  210 17218 ufw6-after-logging-output  0    --  *      *       ::/0                 ::/0                
  210 17218 ufw6-reject-output  0    --  *      *       ::/0                 ::/0                
  210 17218 ufw6-track-output  0    --  *      *       ::/0                 ::/0                

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::b  tcp dpt:110
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::b  tcp dpt:143
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::b  tcp dpt:993
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::b  tcp dpt:995
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::b  tcp dpt:4190
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::c  tcp dpt:80
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::c  tcp dpt:443
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::f  tcp dpt:25
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::f  tcp dpt:465
    0     0 ACCEPT     6    --  !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::f  tcp dpt:587

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-ISOLATION-STAGE-2  0    --  br-mailcow !br-mailcow  ::/0                 ::/0                
    0     0 DOCKER-ISOLATION-STAGE-2  0    --  docker0 !docker0  ::/0                 ::/0                
35619   41M RETURN     0    --  *      *       ::/0                 ::/0                

Chain DOCKER-ISOLATION-STAGE-2 (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       0    --  *      br-mailcow  ::/0                 ::/0                
    0     0 DROP       0    --  *      docker0  ::/0                 ::/0                
    0     0 RETURN     0    --  *      *       ::/0                 ::/0                

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
35619   41M RETURN     0    --  *      *       ::/0                 ::/0                

Chain MAILCOW (2 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-after-forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-after-input (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ufw6-skip-to-policy-input  17   --  *      *       ::/0                 ::/0                 udp dpt:137
    0     0 ufw6-skip-to-policy-input  17   --  *      *       ::/0                 ::/0                 udp dpt:138
    0     0 ufw6-skip-to-policy-input  6    --  *      *       ::/0                 ::/0                 tcp dpt:139
    0     0 ufw6-skip-to-policy-input  6    --  *      *       ::/0                 ::/0                 tcp dpt:445
    0     0 ufw6-skip-to-policy-input  17   --  *      *       ::/0                 ::/0                 udp dpt:546
    0     0 ufw6-skip-to-policy-input  17   --  *      *       ::/0                 ::/0                 udp dpt:547

Chain ufw6-after-logging-forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 LOG        0    --  *      *       ::/0                 ::/0                 limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW BLOCK] "

Chain ufw6-after-logging-input (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   22  1540 LOG        0    --  *      *       ::/0                 ::/0                 limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW BLOCK] "

Chain ufw6-after-logging-output (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-after-output (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-before-forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       0    --  *      *       ::/0                 ::/0                 rt type:0
    0     0 ACCEPT     0    --  *      *       ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 1
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 2
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 3
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 4
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 128
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 129
    0     0 ufw6-user-forward  0    --  *      *       ::/0                 ::/0                

Chain ufw6-before-input (1 references)
 pkts bytes target     prot opt in     out     source               destination         
  305  595K ACCEPT     0    --  lo     *       ::/0                 ::/0                
    0     0 DROP       0    --  *      *       ::/0                 ::/0                 rt type:0
  191  583K ACCEPT     0    --  *      *       ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 129
    0     0 ufw6-logging-deny  0    --  *      *       ::/0                 ::/0                 ctstate INVALID
    0     0 DROP       0    --  *      *       ::/0                 ::/0                 ctstate INVALID
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 1
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 2
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 3
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 4
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 128
  355 19880 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 133 HL match HL == 255
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 134 HL match HL == 255
  252 18144 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 135 HL match HL == 255
  260 16656 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 136 HL match HL == 255
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 141 HL match HL == 255
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 142 HL match HL == 255
    0     0 ACCEPT     58   --  *      *       fe80::/10            ::/0                 ipv6-icmptype 130
    0     0 ACCEPT     58   --  *      *       fe80::/10            ::/0                 ipv6-icmptype 131
    0     0 ACCEPT     58   --  *      *       fe80::/10            ::/0                 ipv6-icmptype 132
    0     0 ACCEPT     58   --  *      *       fe80::/10            ::/0                 ipv6-icmptype 143
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 148 HL match HL == 255
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 149 HL match HL == 255
    0     0 ACCEPT     58   --  *      *       fe80::/10            ::/0                 ipv6-icmptype 151 HL match HL == 1
    0     0 ACCEPT     58   --  *      *       fe80::/10            ::/0                 ipv6-icmptype 152 HL match HL == 1
    0     0 ACCEPT     58   --  *      *       fe80::/10            ::/0                 ipv6-icmptype 153 HL match HL == 1
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 144
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 145
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 146
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 147
    0     0 ACCEPT     17   --  *      *       fe80::/10            fe80::/10            udp spt:547 dpt:546
    0     0 ACCEPT     17   --  *      *       ::/0                 ff02::fb             udp dpt:5353
    0     0 ACCEPT     17   --  *      *       ::/0                 ff02::f              udp dpt:1900
  111  7770 ufw6-user-input  0    --  *      *       ::/0                 ::/0                

Chain ufw6-before-logging-forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-before-logging-input (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-before-logging-output (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-before-output (1 references)
 pkts bytes target     prot opt in     out     source               destination         
  305  595K ACCEPT     0    --  *      lo      ::/0                 ::/0                
    0     0 DROP       0    --  *      *       ::/0                 ::/0                 rt type:0
 1665  227K ACCEPT     0    --  *      *       ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 1
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 2
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 3
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 4
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 128
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 129
    1    56 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 133 HL match HL == 255
  252 16176 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 136 HL match HL == 255
  299 21528 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 135 HL match HL == 255
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 134 HL match HL == 255
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 141 HL match HL == 255
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 142 HL match HL == 255
    0     0 ACCEPT     58   --  *      *       fe80::/10            ::/0                 ipv6-icmptype 130
    0     0 ACCEPT     58   --  *      *       fe80::/10            ::/0                 ipv6-icmptype 131
    0     0 ACCEPT     58   --  *      *       fe80::/10            ::/0                 ipv6-icmptype 132
  171 18556 ACCEPT     58   --  *      *       fe80::/10            ::/0                 ipv6-icmptype 143
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 148 HL match HL == 255
    0     0 ACCEPT     58   --  *      *       ::/0                 ::/0                 ipv6-icmptype 149 HL match HL == 255
    0     0 ACCEPT     58   --  *      *       fe80::/10            ::/0                 ipv6-icmptype 151 HL match HL == 1
    0     0 ACCEPT     58   --  *      *       fe80::/10            ::/0                 ipv6-icmptype 152 HL match HL == 1
    0     0 ACCEPT     58   --  *      *       fe80::/10            ::/0                 ipv6-icmptype 153 HL match HL == 1
  210 17218 ufw6-user-output  0    --  *      *       ::/0                 ::/0                

Chain ufw6-logging-allow (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 LOG        0    --  *      *       ::/0                 ::/0                 limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW ALLOW] "

Chain ufw6-logging-deny (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     0    --  *      *       ::/0                 ::/0                 ctstate INVALID limit: avg 3/min burst 10
    0     0 LOG        0    --  *      *       ::/0                 ::/0                 limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW BLOCK] "

Chain ufw6-reject-forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-reject-input (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-reject-output (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-skip-to-policy-forward (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       0    --  *      *       ::/0                 ::/0                

Chain ufw6-skip-to-policy-input (6 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       0    --  *      *       ::/0                 ::/0                

Chain ufw6-skip-to-policy-output (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     0    --  *      *       ::/0                 ::/0                

Chain ufw6-track-forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-track-input (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-track-output (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   21  1680 ACCEPT     6    --  *      *       ::/0                 ::/0                 ctstate NEW
  111  7770 ACCEPT     17   --  *      *       ::/0                 ::/0                 ctstate NEW

Chain ufw6-user-forward (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-user-input (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-user-limit (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 LOG        0    --  *      *       ::/0                 ::/0                 limit: avg 3/min burst 5 LOG flags 0 level 4 prefix "[UFW LIMIT BLOCK] "
    0     0 REJECT     0    --  *      *       ::/0                 ::/0                 reject-with icmp6-port-unreachable

Chain ufw6-user-limit-accept (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     0    --  *      *       ::/0                 ::/0                

Chain ufw6-user-logging-forward (0 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-user-logging-input (0 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-user-logging-output (0 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain ufw6-user-output (1 references)
 pkts bytes target     prot opt in     out     source               destination

Logs of iptables -L -vn -t nat:

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 4902  294K DOCKER     0    --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER     0    --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
15385 1158K MASQUERADE  0    --  *      !br-mailcow  172.22.1.0/24        0.0.0.0/0           
    0     0 MASQUERADE  0    --  *      !docker0  172.17.0.0/16        0.0.0.0/0           
    0     0 MASQUERADE  6    --  *      *       172.22.1.249         172.22.1.249         tcp dpt:6379
    0     0 MASQUERADE  6    --  *      *       172.22.1.5           172.22.1.5           tcp dpt:8983
    0     0 MASQUERADE  6    --  *      *       172.22.1.6           172.22.1.6           tcp dpt:3306
    0     0 MASQUERADE  6    --  *      *       172.22.1.250         172.22.1.250         tcp dpt:110
    0     0 MASQUERADE  6    --  *      *       172.22.1.250         172.22.1.250         tcp dpt:143
    0     0 MASQUERADE  6    --  *      *       172.22.1.250         172.22.1.250         tcp dpt:993
    0     0 MASQUERADE  6    --  *      *       172.22.1.250         172.22.1.250         tcp dpt:995
    0     0 MASQUERADE  6    --  *      *       172.22.1.250         172.22.1.250         tcp dpt:4190
    0     0 MASQUERADE  6    --  *      *       172.22.1.250         172.22.1.250         tcp dpt:12345
    0     0 MASQUERADE  6    --  *      *       172.22.1.8           172.22.1.8           tcp dpt:80
    0     0 MASQUERADE  6    --  *      *       172.22.1.8           172.22.1.8           tcp dpt:443
    0     0 MASQUERADE  6    --  *      *       172.22.1.253         172.22.1.253         tcp dpt:25
    0     0 MASQUERADE  6    --  *      *       172.22.1.253         172.22.1.253         tcp dpt:465
    0     0 MASQUERADE  6    --  *      *       172.22.1.253         172.22.1.253         tcp dpt:587

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     0    --  br-mailcow *       0.0.0.0/0            0.0.0.0/0           
    0     0 RETURN     0    --  docker0 *       0.0.0.0/0            0.0.0.0/0           
    0     0 DNAT       6    --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:7654 to:172.22.1.249:6379
    0     0 DNAT       6    --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:18983 to:172.22.1.5:8983
    0     0 DNAT       6    --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:13306 to:172.22.1.6:3306
    0     0 DNAT       6    --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:110 to:172.22.1.250:110
   22  1296 DNAT       6    --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:143 to:172.22.1.250:143
    5   386 DNAT       6    --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:993 to:172.22.1.250:993
   14   836 DNAT       6    --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:995 to:172.22.1.250:995
    0     0 DNAT       6    --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:4190 to:172.22.1.250:4190
    0     0 DNAT       6    --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:19991 to:172.22.1.250:12345
 2669  160K DNAT       6    --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80 to:172.22.1.8:80
    0     0 DNAT       6    --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:443 to:172.22.1.8:443
   40  2068 DNAT       6    --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:25 to:172.22.1.253:25
 1185 71064 DNAT       6    --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:465 to:172.22.1.253:465
    6   360 DNAT       6    --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:587 to:172.22.1.253:587

Logs of ip6tables -L -vn -t nat:

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER     0    --  *      *       ::/0                 ::/0                 ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER     0    --  *      *       ::/0                !::1                  ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MASQUERADE  0    --  *      !br-mailcow  fd4d:6169:6c63:6f77::/64  ::/0                
    0     0 MASQUERADE  0    --  *      !docker0  fd00:dead:beef:c0::/80  ::/0                
    0     0 MASQUERADE  6    --  *      *       fd4d:6169:6c63:6f77::b  fd4d:6169:6c63:6f77::b  tcp dpt:110
    0     0 MASQUERADE  6    --  *      *       fd4d:6169:6c63:6f77::b  fd4d:6169:6c63:6f77::b  tcp dpt:143
    0     0 MASQUERADE  6    --  *      *       fd4d:6169:6c63:6f77::b  fd4d:6169:6c63:6f77::b  tcp dpt:993
    0     0 MASQUERADE  6    --  *      *       fd4d:6169:6c63:6f77::b  fd4d:6169:6c63:6f77::b  tcp dpt:995
    0     0 MASQUERADE  6    --  *      *       fd4d:6169:6c63:6f77::b  fd4d:6169:6c63:6f77::b  tcp dpt:4190
    0     0 MASQUERADE  6    --  *      *       fd4d:6169:6c63:6f77::c  fd4d:6169:6c63:6f77::c  tcp dpt:80
    0     0 MASQUERADE  6    --  *      *       fd4d:6169:6c63:6f77::c  fd4d:6169:6c63:6f77::c  tcp dpt:443
    0     0 MASQUERADE  6    --  *      *       fd4d:6169:6c63:6f77::f  fd4d:6169:6c63:6f77::f  tcp dpt:25
    0     0 MASQUERADE  6    --  *      *       fd4d:6169:6c63:6f77::f  fd4d:6169:6c63:6f77::f  tcp dpt:465
    0     0 MASQUERADE  6    --  *      *       fd4d:6169:6c63:6f77::f  fd4d:6169:6c63:6f77::f  tcp dpt:587

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     0    --  br-mailcow *       ::/0                 ::/0                
    0     0 RETURN     0    --  docker0 *       ::/0                 ::/0                
    0     0 DNAT       6    --  !br-mailcow *       ::/0                 ::/0                 tcp dpt:110 to:[fd4d:6169:6c63:6f77::b]:110
    0     0 DNAT       6    --  !br-mailcow *       ::/0                 ::/0                 tcp dpt:143 to:[fd4d:6169:6c63:6f77::b]:143
    0     0 DNAT       6    --  !br-mailcow *       ::/0                 ::/0                 tcp dpt:993 to:[fd4d:6169:6c63:6f77::b]:993
    0     0 DNAT       6    --  !br-mailcow *       ::/0                 ::/0                 tcp dpt:995 to:[fd4d:6169:6c63:6f77::b]:995
    0     0 DNAT       6    --  !br-mailcow *       ::/0                 ::/0                 tcp dpt:4190 to:[fd4d:6169:6c63:6f77::b]:4190
    0     0 DNAT       6    --  !br-mailcow *       ::/0                 ::/0                 tcp dpt:80 to:[fd4d:6169:6c63:6f77::c]:80
    0     0 DNAT       6    --  !br-mailcow *       ::/0                 ::/0                 tcp dpt:443 to:[fd4d:6169:6c63:6f77::c]:443
    0     0 DNAT       6    --  !br-mailcow *       ::/0                 ::/0                 tcp dpt:25 to:[fd4d:6169:6c63:6f77::f]:25
    0     0 DNAT       6    --  !br-mailcow *       ::/0                 ::/0                 tcp dpt:465 to:[fd4d:6169:6c63:6f77::f]:465
    0     0 DNAT       6    --  !br-mailcow *       ::/0                 ::/0                 tcp dpt:587 to:[fd4d:6169:6c63:6f77::f]:587

DNS check:

172.64.155.249
104.18.32.7
@delendum delendum added the bug label Jun 29, 2024
@lukebsa
Copy link

lukebsa commented Jun 29, 2024

I am having this issue too

@Flinkpfote
Copy link

Flinkpfote commented Jun 29, 2024

I have the same problem. If I go straight to /admin, I get a red message saying "array" but I haven't changed/saved anything (Only for the first load). But all the pages work, except /debug. /SoGo/ works all the time. A lot of things are loaded for a very long time.

I dont have ARM but the newest version of Docker (Docker version 27.0.2, build 912c1dd)

#5927

@Aterfax
Copy link

Aterfax commented Jun 29, 2024

Also seeing this. (ARM 64 docker.)

@delendum
Copy link
Author

Getting some inspiration from #5924

I have a search domain specified in /etc/resolv.conf - commenting it out and restarting the docker containers fixes the issue, the /debug page now loads normally.

If you're experiencing this issue then please try this as a temp fix, hopefully a more permanent solution can be found.

@obakuhl
Copy link

obakuhl commented Jun 30, 2024

I had the same issue, but commenting out the search domain solved the problem. Thx @delendum

@Magelo
Copy link

Magelo commented Jul 1, 2024

In my resolv.conf I have :

domain somedomain.com
search somedomain.com
nameserver mydnsip

Removing both domain and search lines resolved the problem for me. Removing only search was doing nothing.

I don't know how, but when I reboot, the two lines are back. So I used :

chattr +i /etc/resolv.conf

It's now working, but those lines were there before with no issue. The last update broke something for sure.

@xxtensazenxx
Copy link

xxtensazenxx commented Jul 2, 2024

Having this same issue as well. I'm on a VM so should I just roll back the update and wait until this is fixed?

@squal0nero
Copy link

I have the same problem with the Debian 12. Deleting the search entry in resolv.conf works as a workaround.

@pedrostefanogv
Copy link

I have the same problem with the Debian 12. Deleting the search entry in resolv.conf works as a workaround.

I tried it here and it didn't work. I'm using ARM on Oracle Cloud

@camodebw
Copy link

camodebw commented Jul 3, 2024

I had the same issue after update from 2024-04 to 2024-06a, reverted back to 2024-04 until a fix is available.

Debian Buster with Kernel: 4.19.0-26-amd64

@pedrostefanogv
Copy link

I had the same issue after update from 2024-04 to 2024-06a, reverted back to 2024-04 until a fix is available.

Debian Buster with Kernel: 4.19.0-26-amd64

Is there a step-by-step guide explaining how to revert the update?

@camodebw
Copy link

camodebw commented Jul 3, 2024

I had the same issue after update from 2024-04 to 2024-06a, reverted back to 2024-04 until a fix is available.
Debian Buster with Kernel: 4.19.0-26-amd64

Is there a step-by-step guide explaining how to revert the update?

Hi Pedro, my Mailcow is running on a VM in Proxmox, that's why I did a restore to 2024-04.

@FreddleSpl0it
Copy link
Collaborator

If nothing helps, edit the docker-compose.yaml and set the phpfpm image tag from

image: mailcow/phpfpm:1.88

to

image: mailcow/phpfpm:1.87

and do a docker compose up -d

@totalinfra
Copy link

I had to revert the php-fpm version and then it started working again.

@mrclschstr
Copy link
Contributor

If nothing helps, edit the docker-compose.yaml and set the phpfpm image tag from

image: mailcow/phpfpm:1.88

to

image: mailcow/phpfpm:1.87

and do a docker compose up -d

Is there a chance of a quick permanent fix? The fix for #5924 seemed to be quite manageable. Are the problems related?

@DerLinkman
Copy link
Member

DerLinkman commented Jul 9, 2024

Can someone try the newly uploaded phpfpm:1.89 container?

Because it seems it is all related to the bind-utils package from alpine which has been installed in this image version.

To try out: simply change the image tag (as described from @mrclschstr) to 1.89 instead of 1.87/1.88 depending on what you currently using...

@psonntag1
Copy link

I gave it a try, with the new phpfpm:1.89 I receive the following errors:

user@machine:/opt/mailcow-dockerized# docker compose logs --tail=200 php-fpm-mailcow
php-fpm-mailcow-1 | Uptime: 0 Threads: 2 Questions: 1 Slow queries: 0 Opens: 17 Open tables: 10 Queries per second avg: 0.000
php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again
php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again
php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again
php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again
php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again
php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again
php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again
php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again
php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again
php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again
php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again
php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again

@DerLinkman
Copy link
Member

Ok still not resolved then...

@DerLinkman
Copy link
Member

I gave it a try, with the new phpfpm:1.89 I receive the following errors:

user@machine:/opt/mailcow-dockerized# docker compose logs --tail=200 php-fpm-mailcow php-fpm-mailcow-1 | Uptime: 0 Threads: 2 Questions: 1 Slow queries: 0 Opens: 17 Open tables: 10 Queries per second avg: 0.000 php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again

Did you have anything written in your resolv.conf e.g. search oprion?

@psonntag1
Copy link

I gave it a try, with the new phpfpm:1.89 I receive the following errors:
user@machine:/opt/mailcow-dockerized# docker compose logs --tail=200 php-fpm-mailcow php-fpm-mailcow-1 | Uptime: 0 Threads: 2 Questions: 1 Slow queries: 0 Opens: 17 Open tables: 10 Queries per second avg: 0.000 php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again

Did you have anything written in your resolv.conf e.g. search oprion?

Yes, I reverted my resolv.conf to a version with search option to test the fix.

@NekoPavel
Copy link

Same issue here, rollback to 07838ccb132259267738a93b5d14cbe82f433166 helped

@DerLinkman
Copy link
Member

Yeah that rolledback version is the version before Alpine 3.19 and upwards. But that did not help on the long run.

@vineethmn
Copy link

If nothing helps, edit the docker-compose.yaml and set the phpfpm image tag from

image: mailcow/phpfpm:1.88

to

image: mailcow/phpfpm:1.87

and do a docker compose up -d

This works for now

@DerLinkman
Copy link
Member

DerLinkman commented Jul 10, 2024

Can someone try out the image mailcow/phpfpm:1.89. This one features a rebuild php container.

Simply do what @vineethmn said but only with the different tag i mentioned.

@totalinfra
Copy link

Hello,

I had rolled back the mailcow/phpfpm:1.88 version to the mailcow/phpfpm:1.87 version.

This resolved the issue of a blank page when accessing the /debug URL

Testing now by changing the tag to version (mailcow/phpfpm:1.89) as suggested by @DerLinkman, there are no problems.

No failures so far using version 1.89.

@BNoiZe
Copy link

BNoiZe commented Jul 10, 2024

Can someone try out the image mailcow/phpfpm:1.89. This one features a rebuild php container.

Simply do what @vineethmn said but only with the different tag i mentioned.

1.89 does indeed fix the problem as @totalinfra also confirmed.

@psonntag1
Copy link

I can also confirm that the new build fixes the issue for me.

@DerLinkman
Copy link
Member

Thank you! Sounds good.

@DerLinkman
Copy link
Member

Sounds like it is solving it, so yeah probably, thats why i asked to test :)

@DerLinkman

This comment was marked as outdated.

@DerLinkman
Copy link
Member

Thank you! Sounds good.

Will there be another hotfix release, like 2024-06b?

Today...

@mrclschstr
Copy link
Contributor

mrclschstr commented Jul 30, 2024

It seems that the curl error/bug is also contained in the acme container:

So far only new installations seem to be affected, but I could imagine that the certificate renewal is also no longer working correctly. At the moment I have no proof of my claim...

@DerLinkman
Copy link
Member

Could be as i said i cannot recreate it anywhere but i believe there is something related to it.

Then we have to reroll all Alpine Packages which runs curl. I cannot do anything else as this is clearly a Alpine Package problem...

@DerLinkman
Copy link
Member

Reopening this issue as I want to help the Alpine Devs to track down the root cause. I still can't reproduce this issue. However I'm using Debian 12 always with the default Network provider "Interfaces" as the base os. I read that some of you are using netplan... maybe I can recreate the issue when I try a Ubuntu with netplan though...

@iamspido
Copy link

iamspido commented Aug 31, 2024

I recieve the php-fpm-mailcow-1 | Could not get mysql-mailcow container id... trying again as well when upgrading from 2024-07 to 2024-08a. Downgrading from mailcow/phpfpm:1.89 to mailcow/phpfpm:1.87 fixed that for me.

Running on default Ubuntu 24.04 Image from Hetzner Cloud Server

Docker Version: 27.2.0
IPv6 active

@maxinux
Copy link

maxinux commented Sep 1, 2024

Ran into this myself today, ipv6 was disabled worked until update, then failed. reverted php-fpm and working again. (Arch, docker 27.2)

@DerLinkman
Copy link
Member

That cannot be... at least not with the c-ares bug then. as these versions did not have c-ares bundles.

Can someone check which php-fpm os release file is located inside the container?....

@DerLinkman
Copy link
Member

DerLinkman commented Sep 2, 2024

Or more easier... can someone let me visit his machine to finally see this error with my own eyes? I was no where able to reproduce it... if not then it won't be fixed in the near future, i'm sorry but without reproduction i cannot fix the issues.

If someone allows me to visit his machine please pm me on Telegram: https://t.me/DerLinkman

@iamspido
Copy link

iamspido commented Sep 2, 2024

The output from cat /etc/os-release in mailcow/phpfpm:1.89 container.

NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.18.6
PRETTY_NAME="Alpine Linux v3.18"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues"

@iamspido
Copy link

iamspido commented Sep 4, 2024

I was able to fix the problem.
Special thanks to @DerLinkman.

The error occurred for me because I manually created the following network:

docker network create --driver bridge --opt com.docker.network.bridge.name=br-mailcow --ipv6 --subnet 172.22.1.0/24 --subnet fd4d:6169:6c63:6f77::/64 mailcow-network

and then added it via docker-compose.override:

networks:
  mailcow-network:
    external: true
    driver: bridge
    driver_opts:
      com.docker.network.bridge.name: br-mailcow
    enable_ipv6: true
    ipam:
      driver: default
      config:
        - subnet: ${IPV4_NETWORK:-172.22.1}.0/24
        - subnet: ${IPV6_NETWORK:-fd4d:6169:6c63:6f77::/64}

This created the network mailcow-network, which does not match the default Mailcow name, mailcowdockerized_mailcow-network.

After removing the manually created network, deleting the network from the docker-compose.override, and allowing the network to be automatically created, everything now works with mailcow/phpfpm:1.89.

@DerLinkman
Copy link
Member

@iamspido's case was not a bug regarding mailcow, instead a "issue" of the system like he described.

We run mailcow tests etc. always depending on the official documentation installation guide/standard in his case the origin of his issue was a standard difference...

Does someone have a different mailcow setup as described in our docs?

@henningsieh
Copy link

henningsieh commented Nov 11, 2024

I think, the bug is still not fixed: #4571 (comment)

I have absolutly no idea... :(

I have a default up-to-date Ubuntu in Hetzner Cloud and have a default install in "/opt/mailcow-dockerized".

What can I do to get my Mailcow's /debug entry page back?

@DerLinkman
Copy link
Member

We'll publish 2024-11a today which is patching the php container. Maybe it is resolving this.

@Reggae13
Copy link

Still getting the error with the latest version - 2024-11a -. I had to reset php-fpm-mailcow to version 1.87 to resolve the error.

@DerLinkman
Copy link
Member

2024-11a is not published yet... so.

@AkamBilly
Copy link

Same behavior in 2024-11a, php-fpm still cant find mysql..

@DerLinkman
Copy link
Member

Then unfortunately won't fix until analyse this any further. I (as i said 100000000 times) cannot reproduce this and so i won't anymore. I'm sorry :(

@DerLinkman DerLinkman closed this as not planned Won't fix, can't repro, duplicate, stale Nov 15, 2024
@The-Inamati
Copy link

Hello,
I just updated mailcow and am having the same issue. I can confirm that rolling back the image version of php-fpm to 1.87 gets it back up but I think it would be better to fix the issue without rolling back the image.

I can easily replicate this in a test VM.

Any help is appreciated. This is clearly an issue with the image version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.