Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MG-2126 - Update messaging with mProxy TLS #2206

Open
wants to merge 41 commits into
base: main
Choose a base branch
from

Conversation

nyagamunene
Copy link
Contributor

What type of PR is this?

This is a feature: It updates Magistrala messaging with mProxy with TLS.

What does this do?

It updates Magistrala messaging with mProxy with TLS and mTLS.

Which issue(s) does this PR fix/relate to?

Have you included tests for your changes?

Yes

Did you document any new/modified feature?

Yes

Notes

@nyagamunene nyagamunene self-assigned this Apr 26, 2024
@nyagamunene nyagamunene force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch 3 times, most recently from 547e1ad to 52d8828 Compare April 30, 2024 17:37
Copy link
Contributor

@arvindh123 arvindh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I start with make run getting to below error.

GRPC_MTLS=
GRPC_TLS=
docker compose -f docker/docker-compose.yml --env-file docker/.env -p arvindh123_magistrala_git_  up 
WARN[0000] The "MG_MQTT_WS_ADAPTER_KEY_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_WS_ADAPTER_CERT_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_WS_ADAPTER_CLIENT_CA_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_ADAPTER_CERT_VERIFICATION_METHODS" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_WS_ADAPTER_PREFIX_PATH" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_WS_ADAPTER_OCSP_RESPONDER_URL" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_ADAPTER_CLIENT_CA_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHOD" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_WS_ADAPTER_SERVER_CA_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_ADAPTER_OCSP_RESPONDER_URL" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_HTTP_ADAPTER_CLIENT_CA_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_HTTP_ADAPTER_CERT_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_HTTP_ADAPTER_KEY_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_HTTP_ADAPTER_CERT_VERIFICATION_METHODS" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_HTTP_ADAPTER_SERVER_CA_FILE" variable is not set. Defaulting to a blank string. 
validating /home/arvindh123/magistrala/docker/docker-compose.yml: services.mqtt-adapter.volumes array items[3,4] must be unique
make: *** [Makefile:240: run] Error 15

MG_MQTT_WS_ADAPTER_SERVER_CA_FILE: ${MG_MQTT_WS_ADAPTER_SERVER_CA_FILE}
MG_MQTT_WS_ADAPTER_PREFIX_PATH: ${MG_MQTT_WS_ADAPTER_PREFIX_PATH}
MG_MQTT_WS_ADAPTER_CLIENT_CA_FILE: ${MG_MQTT_WS_ADAPTER_CLIENT_CA_FILE}
MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHOD: ${MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHOD}
Copy link
Contributor

@arvindh123 arvindh123 May 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHOD: ${MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHOD}
MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHODS: ${MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHODS}

@@ -656,6 +694,12 @@ services:
networks:
- magistrala-base-net
volumes:
- ./ssl/certs/ca.key:/etc/ssl/certs/ca.key
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have this as an env variable. Sth like: - ./ssl/certs/ca.key:${MG_MQTT_CA_CRT_KEY}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do the same for the others.

cmd/mqtt/main.go Outdated
@@ -74,7 +77,7 @@ func main() {
if err := env.Parse(&cfg); err != nil {
log.Fatalf("failed to load %s configuration : %s", svcName, err)
}

log.Println(cfg)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the debug log

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove @nyagamunene

Copy link
Contributor

@arvindh123 arvindh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tried other way for loading certifcate, but I could not find better way than approaches which i have mentioned in the suggestions

# MG_MQTT_WS_ADAPTER_CLIENT_CA_FILE=etc/ssl/certs/ca.crt
# MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHODS=ocsp
# MG_MQTT_WS_ADAPTER_OCSP_RESPONDER_URL=http://localhost:8080/ocsp

Copy link
Contributor

@arvindh123 arvindh123 May 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets have env name for mqtt websocket as MG_MQTT_ADAPTER_WS to align with previous version of env naming.
Then In .env we need to provide the actual certificates location. not the docker location.

Suggested change
## MPROXY MQTT WS
MG_MQTT_ADAPTER_WS_ADDRESS=:8080
MG_MQTT_ADAPTER_WS_TARGET=ws://${MG_MQTT_BROKER_TYPE}:8080/mqtt
MG_MQTT_ADAPTER_WS_PREFIX_PATH=/mqtt
# MG_MQTT_ADAPTER_WS_CERT_FILE=./ssl/certs/magistrala-server.crt
# MG_MQTT_ADAPTER_WS_KEY_FILE=./ssl/certs/magistrala-server.key
# MG_MQTT_ADAPTER_WS_SERVER_CA_FILE=./ssl/certs/ca.crt
# MG_MQTT_ADAPTER_WS_CLIENT_CA_FILE=./ssl/certs/ca.crt
# MG_MQTT_ADAPTER_WS_CERT_VERIFICATION_METHODS=ocsp
# MG_MQTT_ADAPTER_WS_OCSP_RESPONDER_URL=http://localhost:8080/ocsp

# MG_MQTT_WS_ADAPTER_CLIENT_CA_FILE=etc/ssl/certs/ca.crt
# MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHODS=ocsp
# MG_MQTT_WS_ADAPTER_OCSP_RESPONDER_URL=http://localhost:8080/ocsp

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should provide the certificate location , not the docker location
Same for HTTP Adapter also. Please change in HTTP Adapter env values to certificates location

Suggested change
MG_MQTT_ADAPTER_CERT_FILE=./ssl/magistrala-server.crt
MG_MQTT_ADAPTER_KEY_FILE=./ssl/magistrala-server.key
MG_MQTT_ADAPTER_SERVER_CA_FILE=./ssl/certs/ca.crt
MG_MQTT_ADAPTER_CLIENT_CA_FILE=./ssl/certs/ca.crt

- ./ssl/certs/magistrala-server.crt:/etc/ssl/certs/magistrala-server.crt
- ./ssl/certs/magistrala-server.key:/etc/ssl/certs/magistrala-server.key
- ./ssl/certs/thing.crt:/etc/ssl/certs/thing.crt
- ./ssl/certs/thing.key:/etc/ssl/certs/thing.key
# Things gRPC mTLS client certificates
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We load env cert path to const path, because in .env the actual location of certificates will be given and here we will load the const path with conditionally.
If the path given is not given , a dummy path will be loaded. but the env varabile of contaienr is not set.

Suggested change
# Things gRPC mTLS client certificates
# MQTT Adapter certificates
- type: bind
source: ${MG_MQTT_ADAPTER_CERT_FILE:-./ssl/certs/dummy/server_cert}
target: /magistrala-mqtt-adapter${MG_MQTT_ADAPTER_CERT_FILE:+.crt}
read_only: true
bind:
create_host_path: true
- type: bind
source: ${MG_MQTT_ADAPTER_KEY_FILE:-./ssl/certs/dummy/server_key}
target: /magistrala-mqtt-adapter${MG_MQTT_ADAPTER_KEY_FILE:+.key}
read_only: true
bind:
create_host_path: true
- type: bind
source: ${MG_MQTT_ADAPTER_SERVER_CA_FILE:-./ssl/certs/dummy/server_ca}
target: /magistrala-ca${MG_MQTT_ADAPTER_SERVER_CA_FILE:+.crt}
read_only: true
bind:
create_host_path: true
- type: bind
source: ${MG_MQTT_ADAPTER_CLIENT_CA_FILE:-./ssl/certs/dummy/client_ca}
target: /magistrala-client-ca${MG_MQTT_ADAPTER_CLIENT_CA_FILE:+.key}
read_only: true
bind:
create_host_path: true

@nyagamunene nyagamunene marked this pull request as ready for review May 2, 2024 14:27
Copy link
Member

@rodneyosodo rodneyosodo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have a look at #2212 there are some concepts overlapping

cmd/http/main.go Outdated Show resolved Hide resolved
cmd/http/main.go Outdated Show resolved Hide resolved
cmd/http/main.go Outdated Show resolved Hide resolved
cmd/mqtt/main.go Outdated Show resolved Hide resolved
cmd/mqtt/main.go Outdated Show resolved Hide resolved
http/api/endpoint_test.go Outdated Show resolved Hide resolved
http/api/endpoint_test.go Outdated Show resolved Hide resolved
pkg/sdk/go/message_test.go Outdated Show resolved Hide resolved
cmd/http/main.go Show resolved Hide resolved
go.mod Outdated
@@ -5,11 +5,11 @@ go 1.22.2
require (
github.com/0x6flab/namegenerator v1.3.1
github.com/absmach/callhome v0.14.0
github.com/absmach/mproxy v0.4.2
github.com/absmach/mproxy v0.4.3-0.20240430090627-27dad4c91c6c
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update mproxy

cmd/http/main.go Outdated
"go.opentelemetry.io/otel/trace"
"golang.org/x/sync/errgroup"
)

const (
svcName = "http_adapter"
envPrefix = "MG_HTTP_ADAPTER_"
envPrefixHTTP = "MG_HTTP_ADAPTER_"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert to using envPrefix

Suggested change
envPrefixHTTP = "MG_HTTP_ADAPTER_"
envPrefix = "MG_HTTP_ADAPTER_"

docker/.env Outdated
@@ -222,6 +222,8 @@ MG_THINGS_AUTH_GRPC_PORT=7000
MG_THINGS_AUTH_GRPC_SERVER_CERT=${GRPC_MTLS:+./ssl/certs/things-grpc-server.crt}${GRPC_TLS:+./ssl/certs/things-grpc-server.crt}
MG_THINGS_AUTH_GRPC_SERVER_KEY=${GRPC_MTLS:+./ssl/certs/things-grpc-server.key}${GRPC_TLS:+./ssl/certs/things-grpc-server.key}
MG_THINGS_AUTH_GRPC_SERVER_CA_CERTS=${GRPC_MTLS:+./ssl/certs/ca.crt}${GRPC_TLS:+./ssl/certs/ca.crt}
MG_THINGS_AUTH_GRPC_ADDRESS=test:7000
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not the correct URL

@nyagamunene nyagamunene force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch 2 times, most recently from dec4dae to 8c479cc Compare May 9, 2024 08:36
@dborovcanin dborovcanin changed the title MG-2126-Update Magistrala messaging with mproxy with TLS MG-2126 - Update Magistrala messaging with mProxy with TLS May 13, 2024
@nyagamunene nyagamunene changed the title MG-2126 - Update Magistrala messaging with mProxy with TLS MG-2126 - Update messaging with mProxy TLS May 13, 2024
@nyagamunene nyagamunene force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch from 8c479cc to e991b72 Compare May 13, 2024 12:02
Copy link
Contributor

@arvindh123 arvindh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the nginx config at here https://github.com/absmach/magistrala/blob/main/docker/nginx/nginx-key.conf#L192-L209

# MQTT
stream {
   include snippets/stream_access_log.conf;

    # Include single-node or multiple-node (cluster) upstream
    # Configure upstream with available nginx loading blancing strageies https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/
    include snippets/mqtt-upstream.conf;

    server {
        listen ${MG_NGINX_MQTT_PORT};
        listen [::]:${MG_NGINX_MQTT_PORT};
        listen ${MG_NGINX_MQTTS_PORT} ;
        listen [::]:${MG_NGINX_MQTTS_PORT} ;

        # include snippets/ssl.conf;

        proxy_pass mqtt_cluster;
    }
}

MG_MQTT_ADAPTER_WS_KEY_FILE: ${MG_MQTT_ADAPTER_WS_KEY_FILE:+/magistrala-mqtt-adapter.key}
MG_MQTT_ADAPTER_WS_SERVER_CA_FILE: ${MG_MQTT_ADAPTER_WS_SERVER_CA_FILE:+/magistrala-ca.crt}
MG_MQTT_ADAPTER_WS_CLIENT_CA_FILE: ${MG_MQTT_ADAPTER_WS_CLIENT_CA_FILE:+/magistrala-client-ca.key}
MG_MQTT_ADAPTER_WS_CERT_VERIFICATION_METHODS: ${MG_MQTT_ADAPTER_WS_CERT_VERIFICATION_METHODS:-}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
MG_MQTT_ADAPTER_WS_CERT_VERIFICATION_METHODS: ${MG_MQTT_ADAPTER_WS_CERT_VERIFICATION_METHODS:-}
MG_MQTT_ADAPTER_WS_CERT_FILE: ${MG_MQTT_ADAPTER_WS_CERT_FILE:+/magistrala-mqtt-adapter-ws.crt}
MG_MQTT_ADAPTER_WS_KEY_FILE: ${MG_MQTT_ADAPTER_WS_KEY_FILE:+/magistrala-mqtt-adapter-ws.key}
MG_MQTT_ADAPTER_WS_SERVER_CA_FILE: ${MG_MQTT_ADAPTER_WS_SERVER_CA_FILE:+/magistrala-ca-ws.crt}
MG_MQTT_ADAPTER_WS_CLIENT_CA_FILE: ${MG_MQTT_ADAPTER_WS_CLIENT_CA_FILE:+/magistrala-client-ca-ws.key}

Comment on lines 1 to 26
-----BEGIN CERTIFICATE-----
MIIEaDCCA1ACFAeBgoEhMA0RBjK9wPoCUZMOJqt0MA0GCSqGSIb3DQEBCwUAMHUx
IjAgBgNVBAMMGU1hZ2lzdHJhbGFfU2VsZl9TaWduZWRfQ0ExEzARBgNVBAoMCk1h
Z2lzdHJhbGExFjAUBgNVBAsMDW1hZ2lzdHJhbGFfY2ExIjAgBgkqhkiG9w0BCQEW
E2luZm9AbWFnaXN0cmFsYS5jb20wHhcNMjQwNDMwMTExOTM4WhcNMjYwNDMwMTEx
OTM4WjBsMRgwFgYDVQQDDA88VEhJTkdfU0VDUkVUPiAxEzARBgNVBAoMCk1hZ2lz
dHJhbGExFzAVBgNVBAsMDm1hZ2lzdHJhbGFfY3J0MSIwIAYJKoZIhvcNAQkBFhNp
bmZvQG1hZ2lzdHJhbGEuY29tMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKC
AgEAvdpn5W14xTNDtcxy8RjGCpQ5cHR1wfILN57Mwx4nv+YNrwmW8mn3Bn2+w0rT
lkNelYSgA7KlkO9aayRyDKuPJYb26rMITnizasxiTl+0PFnEv1E5KhzvoCM7lern
0/bSoeW04tM+vGYWZq+LWO7s5xE1veefwGiYqZjYBdltCdtvID8zgc8OrgB7ZztP
1jRHhpssEBHQK0dZeWaHpHnfeHK2fYc4ih0fW6Xvr+ziIbCY8popCu5Y3xU18E7b
pgs3mFpdytWar968aR0U3dYi3f5vWoq+BOu3UL67nqkGg0dHtdPnBlu4zyQ4TxAr
KUDQwsdd4HotvYNl09d6MvCMygtTT1mfTiScStHuwvWIiU5mI8PnFeYMaLmScKgj
bA812ak9jClCFE8gEgKWW35z9Bv0e/ZjHqX+98HRfw7S9icWTcoaUwOcbbnb2enx
niWpQBZzgz0J2NJn5MxCPFUzUn1WCqlWC6FGfVve3CLPCJoiF77IVSZJ+rNrwyxW
LcefQigGOuZ7NLg5Y7O2LD/xXxPkNlVail8oklaQk1c/bHd6YPiG0/9uKz2NcCL9
txFl6errRfPl1c/5ACvG69FYXADSvMZ8pPesTGLAag7JGj8pxLyLFYaLnjlbvleO
RtHCnIKcwYW8AjYiuuRBtDiP7W61CiuqpT12QZCTvPSerg0CAwEAATANBgkqhkiG
9w0BAQsFAAOCAQEATZfofPEARNP+Hysv6bWKTsFr8HUOWE2OgEPEOMHnIQ5zL9RV
Lyt1sY62DjDXibirnM55F8cM1gBrKSrfelshClTNRxKc2/wsnZwjOepLb5xZ14Z2
+zdEJKS2IOtdQkxV8uDfu2wdBTOC9AyhIdDITSUwD5M7TqSegemIceZ8QU07rNqJ
AqYTrsqVVks9+b50ggjkYmEjhiukfzpidwNRax//k5/gRXNJBCvn6oIXlbB0wgpY
Zqomsxby3t2c71YN/edd5y3kaM3FN21stPkqVFEM/SQTNctiw6ZQcMIgyvqZn3YR
U8NaiWc7xDTKY7R/U5SZBkuqPpBVeDKoCFruvg==
-----END CERTIFICATE-----
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this file

Suggested change
-----BEGIN CERTIFICATE-----
MIIEaDCCA1ACFAeBgoEhMA0RBjK9wPoCUZMOJqt0MA0GCSqGSIb3DQEBCwUAMHUx
IjAgBgNVBAMMGU1hZ2lzdHJhbGFfU2VsZl9TaWduZWRfQ0ExEzARBgNVBAoMCk1h
Z2lzdHJhbGExFjAUBgNVBAsMDW1hZ2lzdHJhbGFfY2ExIjAgBgkqhkiG9w0BCQEW
E2luZm9AbWFnaXN0cmFsYS5jb20wHhcNMjQwNDMwMTExOTM4WhcNMjYwNDMwMTEx
OTM4WjBsMRgwFgYDVQQDDA88VEhJTkdfU0VDUkVUPiAxEzARBgNVBAoMCk1hZ2lz
dHJhbGExFzAVBgNVBAsMDm1hZ2lzdHJhbGFfY3J0MSIwIAYJKoZIhvcNAQkBFhNp
bmZvQG1hZ2lzdHJhbGEuY29tMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKC
AgEAvdpn5W14xTNDtcxy8RjGCpQ5cHR1wfILN57Mwx4nv+YNrwmW8mn3Bn2+w0rT
lkNelYSgA7KlkO9aayRyDKuPJYb26rMITnizasxiTl+0PFnEv1E5KhzvoCM7lern
0/bSoeW04tM+vGYWZq+LWO7s5xE1veefwGiYqZjYBdltCdtvID8zgc8OrgB7ZztP
1jRHhpssEBHQK0dZeWaHpHnfeHK2fYc4ih0fW6Xvr+ziIbCY8popCu5Y3xU18E7b
pgs3mFpdytWar968aR0U3dYi3f5vWoq+BOu3UL67nqkGg0dHtdPnBlu4zyQ4TxAr
KUDQwsdd4HotvYNl09d6MvCMygtTT1mfTiScStHuwvWIiU5mI8PnFeYMaLmScKgj
bA812ak9jClCFE8gEgKWW35z9Bv0e/ZjHqX+98HRfw7S9icWTcoaUwOcbbnb2enx
niWpQBZzgz0J2NJn5MxCPFUzUn1WCqlWC6FGfVve3CLPCJoiF77IVSZJ+rNrwyxW
LcefQigGOuZ7NLg5Y7O2LD/xXxPkNlVail8oklaQk1c/bHd6YPiG0/9uKz2NcCL9
txFl6errRfPl1c/5ACvG69FYXADSvMZ8pPesTGLAag7JGj8pxLyLFYaLnjlbvleO
RtHCnIKcwYW8AjYiuuRBtDiP7W61CiuqpT12QZCTvPSerg0CAwEAATANBgkqhkiG
9w0BAQsFAAOCAQEATZfofPEARNP+Hysv6bWKTsFr8HUOWE2OgEPEOMHnIQ5zL9RV
Lyt1sY62DjDXibirnM55F8cM1gBrKSrfelshClTNRxKc2/wsnZwjOepLb5xZ14Z2
+zdEJKS2IOtdQkxV8uDfu2wdBTOC9AyhIdDITSUwD5M7TqSegemIceZ8QU07rNqJ
AqYTrsqVVks9+b50ggjkYmEjhiukfzpidwNRax//k5/gRXNJBCvn6oIXlbB0wgpY
Zqomsxby3t2c71YN/edd5y3kaM3FN21stPkqVFEM/SQTNctiw6ZQcMIgyvqZn3YR
U8NaiWc7xDTKY7R/U5SZBkuqPpBVeDKoCFruvg==
-----END CERTIFICATE-----

Comment on lines 1 to 52
-----BEGIN PRIVATE KEY-----
MIIJQQIBADANBgkqhkiG9w0BAQEFAASCCSswggknAgEAAoICAQC92mflbXjFM0O1
zHLxGMYKlDlwdHXB8gs3nszDHie/5g2vCZbyafcGfb7DStOWQ16VhKADsqWQ71pr
JHIMq48lhvbqswhOeLNqzGJOX7Q8WcS/UTkqHO+gIzuV6ufT9tKh5bTi0z68ZhZm
r4tY7uznETW955/AaJipmNgF2W0J228gPzOBzw6uAHtnO0/WNEeGmywQEdArR1l5
Zoeked94crZ9hziKHR9bpe+v7OIhsJjymikK7ljfFTXwTtumCzeYWl3K1Zqv3rxp
HRTd1iLd/m9air4E67dQvrueqQaDR0e10+cGW7jPJDhPECspQNDCx13gei29g2XT
13oy8IzKC1NPWZ9OJJxK0e7C9YiJTmYjw+cV5gxouZJwqCNsDzXZqT2MKUIUTyAS
ApZbfnP0G/R79mMepf73wdF/DtL2JxZNyhpTA5xtudvZ6fGeJalAFnODPQnY0mfk
zEI8VTNSfVYKqVYLoUZ9W97cIs8ImiIXvshVJkn6s2vDLFYtx59CKAY65ns0uDlj
s7YsP/FfE+Q2VVqKXyiSVpCTVz9sd3pg+IbT/24rPY1wIv23EWXp6utF8+XVz/kA
K8br0VhcANK8xnyk96xMYsBqDskaPynEvIsVhoueOVu+V45G0cKcgpzBhbwCNiK6
5EG0OI/tbrUKK6qlPXZBkJO89J6uDQIDAQABAoICAFwCCdH9cMcZJhpn/9hqKigc
V43xUtRwShFli1SQO7Fbe4w1uIq44feqXxXkIJC1C5SolqfUHzsTVJ2SJxeO2SvV
sbpik4b8Je9/J7FV3lJVKu2RHoQTS+ouyQTJuBpYhxHg8f3vZHsHepm/P2gVOoht
Jb9oDlrAFiQzJ5YGHjrltoBJQNwLDdEz+Jo5+2ITKgrJDtym1Vz0Eh+smdYTHkMQ
TPKe4r7EiFrxGy0GPz0WibkhDvz2L+nFLMPOY9KfowzZ1pSjWnkbF/JVHrS3v2QY
l7yoepiF5+xg0yp8PeTHWbB8Bn1lXDY/Fm7ypggzGjkbR5qve5VZpbK5vcYMIB5B
FSVERwNho8d665uWF9aFw56OxSFSWewb3ddN4FY5Efo/0dGtUt02HyFvuVLI5xbP
rEvZHGvrHyqzY9uqtHTdEgJgYfz05DoxRoXEgktQrdo01UxJwh/Xc0Rsh6Z/I0mF
3GWdqOodysrm+9PI5WLGDZmu3e6dXysmhjE0oMwQiAbb43LJwpR1XBPfc08vhEat
YMnafd4dsxbAPyTqLPzTg+2ovkA5VLWDYarUDNdzEvrcUalEUA6wTEN1J8WITjGQ
8t4Fqfkv669ZEYQNG3p9VP3Wm3WtUluh3vDEPWI06QMnnGLrI1M+FJXtQlbgb+aG
kUxwTr2GvpHg0ISwsaV9AoIBAQDfSw/l4iYSIf9YFD1uFoCIiHu2VWiXXWhHElC7
31mFBNoP7TQ/J6u7Bp9nody77ycJZpVYm0DPqn5tHH+gXopS/oEK7Kpxgdaqhu28
J4NVGVu4P5FiUye0Fza/96TtaOkxPa4bg+bRMDN35XGUYE0KwW4WPMk+20IhcmP2
sn2uSFLqhjCDnuVxCK8kZGt7cEtIFEvI6eckhTcD764n5okL+Aj/RqyuiAN4y7QQ
EiepayoFYXS1gFATecGOTqTpeWGuKThQPgY6t3OLGc1b6WpZt13yxv/TLA/rg20l
67f8c054mkUdzwtwavmHNmTltDb4cocJB8k66Oy8rFV9EcRLAoIBAQDZqWzDXQRo
mYLR2qmgBH/PJFmRVOb/lj7DKtiuW8CjCASOuvGQnU1FSsWHoO5TyetpuXhXabsy
uz4wT4soloTyqIRfAEjt6obEI/LN2THiGYdpiA9cvtods6LOQFzhzLYRx79lUGic
CwfvivgcoRROsQRZFSunnRQ7uBpDl3JpYGsR3BlFjPzMbdkd3M1saysLZ0TN8H3d
jqmd6V8LZka94ulXjC+KcPxId5AhySvdBrgyXSzdPo80RaHbYu2S7UDjiF/GkIVd
jpO6aNZ4HmQAwk8YKkUKPN16QpgEdNQuSulYl6CsMY2dnqPCdJlh8NCMjkUfkFSS
DhqpiUwZ1PAHAoIBAFAzUrN75bavlVoacvH+jQ39D4ou5REw0Ipubqb8EtgvJO+B
gDmK0yiquUhsn660uC2tNckMglvUXfJswP5l1oL2vHyjApkgTG0VZ4AkJQzEvefQ
lTRer60EhXqvxQIso003XTiuyJGsIWBX/dKQQDuzvAmqrZGxCPCgeH0cIP4fJgDs
T7KGuNw/hUCp/NzqOW/bMlkoggfg/SChzsiwCdtNrFWcGdvaKH0TlUoMa0hzoDUs
itHCE4DbQ4UMhDkDiu0WCw7vKCegypRUoSj5DmTI8qqHhU1gzxsFKofrQTsgYzf4
GQka2332PketZtIGR5Q29n9s138cOCOQNmhkoD8CggEAc5TNRZyyR50k46z6oBdj
iyqhSVRNafFtmJO7BjOAwtHyIZzRw9pT+vSumcvyYpn5oWW0qq5hkpntdxf2LHJp
XLXxvMTY9UWxlJwGRhsMqySlTHvMtJJAHr+SN3XkHJ9pG133m6QsOJh++UvRXtqG
fxbu2SMt0E7QPLoRcE0OmzhxAsXewZkRcLjwLzRd/TGXZPHMjpMjMzCbj1gqeVIx
RIonIK83W4uN3ClpZxd9bFwyGcRym3Wp87b6MRaCU6Aa8fgr67Q4raqVdcpQAO8a
aXDFMW7YxIv3jAGq2cr0FTUAXmP+FRInQkjIau0huel3xPuTZyRsVDabH+XhfGIf
pwKCAQB1CNtcspjw6UMpxVMCsUtcjD2/ze0ipgfisp4ICVdfY2ZaEu5XOP8ZrvqA
Adc+51uayhXvTM9qH8nK6thAQkHvwi/JyTOsuQfqqdaRbH6ywX8qWm2p+1BsHntP
4xuaPp4ViEPJwopsn2MDrvcjvpARSgHB42s5m9plnt33gIk1yf2C0RIXiPwMS8eu
jQrznv2ieXhS1GWE46V+J66Q/m/nojSi1jQQEBLNFvvQc8eMZdiFwGR9uS/GRBPH
St3HhEffsmT191Shnuvk0y//CpQLF4PeH0UqephH3qVsCJP6gXJcZH6pUgLoDfmI
/YOhmC7Rw0vBzAcqqblj25mXwfcb
-----END PRIVATE KEY-----
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this file

Suggested change
-----BEGIN PRIVATE KEY-----
MIIJQQIBADANBgkqhkiG9w0BAQEFAASCCSswggknAgEAAoICAQC92mflbXjFM0O1
zHLxGMYKlDlwdHXB8gs3nszDHie/5g2vCZbyafcGfb7DStOWQ16VhKADsqWQ71pr
JHIMq48lhvbqswhOeLNqzGJOX7Q8WcS/UTkqHO+gIzuV6ufT9tKh5bTi0z68ZhZm
r4tY7uznETW955/AaJipmNgF2W0J228gPzOBzw6uAHtnO0/WNEeGmywQEdArR1l5
Zoeked94crZ9hziKHR9bpe+v7OIhsJjymikK7ljfFTXwTtumCzeYWl3K1Zqv3rxp
HRTd1iLd/m9air4E67dQvrueqQaDR0e10+cGW7jPJDhPECspQNDCx13gei29g2XT
13oy8IzKC1NPWZ9OJJxK0e7C9YiJTmYjw+cV5gxouZJwqCNsDzXZqT2MKUIUTyAS
ApZbfnP0G/R79mMepf73wdF/DtL2JxZNyhpTA5xtudvZ6fGeJalAFnODPQnY0mfk
zEI8VTNSfVYKqVYLoUZ9W97cIs8ImiIXvshVJkn6s2vDLFYtx59CKAY65ns0uDlj
s7YsP/FfE+Q2VVqKXyiSVpCTVz9sd3pg+IbT/24rPY1wIv23EWXp6utF8+XVz/kA
K8br0VhcANK8xnyk96xMYsBqDskaPynEvIsVhoueOVu+V45G0cKcgpzBhbwCNiK6
5EG0OI/tbrUKK6qlPXZBkJO89J6uDQIDAQABAoICAFwCCdH9cMcZJhpn/9hqKigc
V43xUtRwShFli1SQO7Fbe4w1uIq44feqXxXkIJC1C5SolqfUHzsTVJ2SJxeO2SvV
sbpik4b8Je9/J7FV3lJVKu2RHoQTS+ouyQTJuBpYhxHg8f3vZHsHepm/P2gVOoht
Jb9oDlrAFiQzJ5YGHjrltoBJQNwLDdEz+Jo5+2ITKgrJDtym1Vz0Eh+smdYTHkMQ
TPKe4r7EiFrxGy0GPz0WibkhDvz2L+nFLMPOY9KfowzZ1pSjWnkbF/JVHrS3v2QY
l7yoepiF5+xg0yp8PeTHWbB8Bn1lXDY/Fm7ypggzGjkbR5qve5VZpbK5vcYMIB5B
FSVERwNho8d665uWF9aFw56OxSFSWewb3ddN4FY5Efo/0dGtUt02HyFvuVLI5xbP
rEvZHGvrHyqzY9uqtHTdEgJgYfz05DoxRoXEgktQrdo01UxJwh/Xc0Rsh6Z/I0mF
3GWdqOodysrm+9PI5WLGDZmu3e6dXysmhjE0oMwQiAbb43LJwpR1XBPfc08vhEat
YMnafd4dsxbAPyTqLPzTg+2ovkA5VLWDYarUDNdzEvrcUalEUA6wTEN1J8WITjGQ
8t4Fqfkv669ZEYQNG3p9VP3Wm3WtUluh3vDEPWI06QMnnGLrI1M+FJXtQlbgb+aG
kUxwTr2GvpHg0ISwsaV9AoIBAQDfSw/l4iYSIf9YFD1uFoCIiHu2VWiXXWhHElC7
31mFBNoP7TQ/J6u7Bp9nody77ycJZpVYm0DPqn5tHH+gXopS/oEK7Kpxgdaqhu28
J4NVGVu4P5FiUye0Fza/96TtaOkxPa4bg+bRMDN35XGUYE0KwW4WPMk+20IhcmP2
sn2uSFLqhjCDnuVxCK8kZGt7cEtIFEvI6eckhTcD764n5okL+Aj/RqyuiAN4y7QQ
EiepayoFYXS1gFATecGOTqTpeWGuKThQPgY6t3OLGc1b6WpZt13yxv/TLA/rg20l
67f8c054mkUdzwtwavmHNmTltDb4cocJB8k66Oy8rFV9EcRLAoIBAQDZqWzDXQRo
mYLR2qmgBH/PJFmRVOb/lj7DKtiuW8CjCASOuvGQnU1FSsWHoO5TyetpuXhXabsy
uz4wT4soloTyqIRfAEjt6obEI/LN2THiGYdpiA9cvtods6LOQFzhzLYRx79lUGic
CwfvivgcoRROsQRZFSunnRQ7uBpDl3JpYGsR3BlFjPzMbdkd3M1saysLZ0TN8H3d
jqmd6V8LZka94ulXjC+KcPxId5AhySvdBrgyXSzdPo80RaHbYu2S7UDjiF/GkIVd
jpO6aNZ4HmQAwk8YKkUKPN16QpgEdNQuSulYl6CsMY2dnqPCdJlh8NCMjkUfkFSS
DhqpiUwZ1PAHAoIBAFAzUrN75bavlVoacvH+jQ39D4ou5REw0Ipubqb8EtgvJO+B
gDmK0yiquUhsn660uC2tNckMglvUXfJswP5l1oL2vHyjApkgTG0VZ4AkJQzEvefQ
lTRer60EhXqvxQIso003XTiuyJGsIWBX/dKQQDuzvAmqrZGxCPCgeH0cIP4fJgDs
T7KGuNw/hUCp/NzqOW/bMlkoggfg/SChzsiwCdtNrFWcGdvaKH0TlUoMa0hzoDUs
itHCE4DbQ4UMhDkDiu0WCw7vKCegypRUoSj5DmTI8qqHhU1gzxsFKofrQTsgYzf4
GQka2332PketZtIGR5Q29n9s138cOCOQNmhkoD8CggEAc5TNRZyyR50k46z6oBdj
iyqhSVRNafFtmJO7BjOAwtHyIZzRw9pT+vSumcvyYpn5oWW0qq5hkpntdxf2LHJp
XLXxvMTY9UWxlJwGRhsMqySlTHvMtJJAHr+SN3XkHJ9pG133m6QsOJh++UvRXtqG
fxbu2SMt0E7QPLoRcE0OmzhxAsXewZkRcLjwLzRd/TGXZPHMjpMjMzCbj1gqeVIx
RIonIK83W4uN3ClpZxd9bFwyGcRym3Wp87b6MRaCU6Aa8fgr67Q4raqVdcpQAO8a
aXDFMW7YxIv3jAGq2cr0FTUAXmP+FRInQkjIau0huel3xPuTZyRsVDabH+XhfGIf
pwKCAQB1CNtcspjw6UMpxVMCsUtcjD2/ze0ipgfisp4ICVdfY2ZaEu5XOP8ZrvqA
Adc+51uayhXvTM9qH8nK6thAQkHvwi/JyTOsuQfqqdaRbH6ywX8qWm2p+1BsHntP
4xuaPp4ViEPJwopsn2MDrvcjvpARSgHB42s5m9plnt33gIk1yf2C0RIXiPwMS8eu
jQrznv2ieXhS1GWE46V+J66Q/m/nojSi1jQQEBLNFvvQc8eMZdiFwGR9uS/GRBPH
St3HhEffsmT191Shnuvk0y//CpQLF4PeH0UqephH3qVsCJP6gXJcZH6pUgLoDfmI
/YOhmC7Rw0vBzAcqqblj25mXwfcb
-----END PRIVATE KEY-----

@dborovcanin dborovcanin force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch from e991b72 to 898644d Compare May 14, 2024 11:44
@nyagamunene nyagamunene force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch 2 times, most recently from a67df6e to 4db5f11 Compare May 15, 2024 08:09
cmd/http/main.go Outdated
@@ -41,7 +42,6 @@ const (
envPrefixAuthz = "MG_THINGS_AUTH_GRPC_"
defSvcHTTPPort = "80"
targetHTTPPort = "81"
targetHTTPHost = "http://localhost"
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert the change and have it as a default tarrgetHTTPHost

address := fmt.Sprintf("%s:%s", "", cfg.Port)
target := fmt.Sprintf("%s:%s", targetHTTPHost, targetHTTPPort)
mp, err := mproxy.NewProxy(address, target, sessionHandler, logger)
httpConfig, err := mproxy.NewConfig(env.Options{Prefix: envPrefix})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specify the default Target to be used

cmd/mqtt/main.go Outdated
@@ -74,7 +77,7 @@ func main() {
if err := env.Parse(&cfg); err != nil {
log.Fatalf("failed to load %s configuration : %s", svcName, err)
}

log.Println(cfg)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove @nyagamunene

@nyagamunene nyagamunene force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch 2 times, most recently from eda7ad5 to cf69257 Compare May 21, 2024 11:45
cmd/http/main.go Outdated
@@ -163,43 +162,25 @@ func newService(pub messaging.Publisher, tc magistrala.AuthzServiceClient, logge
}

func proxyHTTP(ctx context.Context, cfg server.Config, logger *slog.Logger, sessionHandler session.Handler) error {
config := mproxy.Config{
httpConfig := mproxy.Config{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's change to something like to be able to load TLS configs too

envPrefixMProxy = "MG_HTTP_ADAPTER_MPROXY_"

httpConfig, err := mproxy.NewConfig(env.Options{Prefix: envPrefixMProxy})
if err != nil {
	return err
}
if httpConfig.Address == "" {
	httpConfig.Address = fmt.Sprintf("%s:%s", "", cfg.Port)
}
if httpConfig.Target == "" {
	httpConfig.Target = fmt.Sprintf("%s:%s", targetHTTPHost, targetHTTPPort)
}

This applies to all adapters using mproxy and update the environment variables accordingly

cmd/http/main.go Outdated
go func() {
errCh <- mp.Listen(ctx)
}()
logger.Info(fmt.Sprintf("%s service https server listening at %s:%s with TLS cert %s and key %s", svcName, cfg.Host, cfg.Port, cfg.CertFile, cfg.KeyFile))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change this message to be different it tls config is not loaded

docker/.env Outdated
MG_HTTP_ADAPTER_ADDRESS=:80
MG_HTTP_ADAPTER_PREFIX_PATH=/
MG_HTTP_ADAPTER_TARGET=http://localhost:81
# MG_HTTP_ADAPTER_CERT_FILE=./ssl/certs/magistrala-server.crt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have them commented out?

@nyagamunene nyagamunene force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch from cf69257 to 41e527b Compare May 23, 2024 09:46
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
Signed-off-by: nyagamunene <[email protected]>
@nyagamunene nyagamunene force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch from 41e527b to 53fe877 Compare May 23, 2024 10:11
Signed-off-by: nyagamunene <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature: Update Magistrala messaging with mProxy with TLS
4 participants