SSL / TLS Implementation #1

TechAspirant · 2019-11-09T06:50:05Z

Is there any update on SSL / TLS implementation ?

codepr · 2019-11-11T12:30:20Z

I work on this project on and off, mainly during spare time, it should be easy enough of an addition, that's why I prioritized more inner-logic features, anyway PRs are welcome.

nsf747 · 2019-11-13T08:19:19Z

Can you give some idea on how to add the ssl/tls functionality ??

codepr · 2019-11-19T09:30:59Z

I suppose the easiest way is to incrementally add it by using openssl/ssl.h library:

add entries for certs on configuration, possibly specifying path on the config file as well
write two read/write functions which uses a SSL structure and SSL_write/SSL_read
on the server-side, after checking if the conf requires an encrypted communication, setup SSL context, load certificates and use ssl read/write functions
Something similar to what I did on https://github.com/codepr/vessel/blob/master/src/networking.c, a PoC repository that I created some time ago.

TechAspirant · 2019-11-23T13:12:37Z

add entries for certs on configuration, possibly specifying path on the config file as well
-> for this I have added two entries in config file
Check whether connection is over SSL - [If Yes then Set 1; If No then Set 0]
SSL 0
Certificate File Path
filepath /home/user/Desktop/rootCA.crt

added entries in struct config {} to read this two enteries from config files
write two read/write functions which uses a SSL structure and SSL_write/SSL_read
-> Added ssl_send,ssl_recv,load_certificates,create_ssl_context,openssl_cleanup,openssl_init in my network.c file
on the server-side, after checking if the conf requires an encrypted communication, setup SSL context, load certificates and use ssl read/write functions
-> for this do I have to add in accept_loop function of server.c
can you show where to add above functions

codepr · 2019-11-28T11:04:17Z

You can see a rough work I started on ab112b7. Just some simple changes on reception parts on the server module, and accept of course.

Check for encryption required
Accept using SSL_new on the connecting client if encryption ON
Use ssl_recv/ssl_send if encryption ON

TechAspirant · 2019-11-28T11:56:44Z

I have create certificates for my broker using openssl command like below
req -newkey rsa:2048 -nodes -keyout key.pem -x509 -days 365 -out certificate.pem

and Mentioned Below line in conf file
certfile /home/parth/Desktop/certificate.pem
keyfile /home/parth/Desktop/key.pem

but while connecting from MQTT Client its showing Droping client at server side. Not able to connect.

TechAspirant · 2019-11-29T12:30:35Z

I am getting connected message for my client Id but Client shows continuously connection error
1575029534 i New client connected as 814e1a4f-4885-42bb-a4bb-181890d1267d1575030492087 (c0, k10)
1575029534 # Sending CONNACK to 814e1a4f-4885-42bb-a4bb-181890d1267d1575030492087 (0, 0)

SSL_get_error(client->ssl,r) returns Error 2 Which I think it means error with certificates

nsf747 · 2019-11-30T04:36:34Z

but while connecting from MQTT Client its showing Droping client at server side. Not able to connect.

Im facing the same issue. Were you able to resolve it ?

IoTiA · 2019-11-30T05:28:24Z

I start the server by making ssl=true and port=8883 it gives the error as follow...
1995538976:error:02001002:system library:fopen:No such file or directory:../crypto/bio/bss_file.c:288:fopen('','r')
1995538976:error:20074002:BIO routines:file_ctrl:system lib:../crypto/bio/bss_file.c:290:
1995538976:error:140AD002:SSL routines:SSL_CTX_use_certificate_file:system lib:../ssl/ssl_rsa.c:390:

TechAspirant · 2019-11-30T06:10:12Z

I start the server by making ssl=true and port=8883 it gives the error as follow...
1995538976:error:02001002:system library:fopen:No such file or directory:../crypto/bio/bss_file.c:288:fopen('','r')
1995538976:error:20074002:BIO routines:file_ctrl:system lib:../crypto/bio/bss_file.c:290:
1995538976:error:140AD002:SSL routines:SSL_CTX_use_certificate_file:system lib:../ssl/ssl_rsa.c:390:

For this you have to add two entries in .conf file as below
certfile ~/pathofcertificate/certificate.pem
keyfile ~/pathofcertificatekey/key.pem

TechAspirant · 2019-11-30T06:10:58Z

but while connecting from MQTT Client its showing Droping client at server side. Not able to connect.

Im facing the same issue. Were you able to resolve it ?

I am trying but no success; if you resolved then do let me know your solution

IoTiA · 2019-11-30T06:38:38Z

For this you have to add two entries in .conf file as below
certfile ~/pathofcertificate/certificate.pem
keyfile ~/pathofcertificatekey/key.pem

i add the certs in config file still i am getting same error. such as..
1995727392:error:02001002:system library:fopen:No such file or directory:../crypto/bio/bss_file.c:288:fopen('~/home/pi/certificate.pem','r')
1995727392:error:20074002:BIO routines:file_ctrl:system lib:../crypto/bio/bss_file.c:290:
1995727392:error:140AD002:SSL routines:SSL_CTX_use_certificate_file:system lib:../ssl/ssl_rsa.c:390:

TechAspirant · 2019-11-30T06:40:05Z

certificate.pem

Is certificate.pem file present at location /home/pi ?
Change from ~/home/pi/certificate.pem to /home/pi/certificate.pem

IoTiA · 2019-11-30T06:45:32Z

Now getting this permission error as follow..
1995821600:error:0200100D:system library:fopen:Permission denied:../crypto/bio/bss_file.c:288:fopen('/home/pi/key.pem','r')
1995821600:error:20074002:BIO routines:file_ctrl:system lib:../crypto/bio/bss_file.c:290:
1995821600:error:140B0002:SSL routines:SSL_CTX_use_PrivateKey_file:system lib:../ssl/ssl_rsa.c:540:

TechAspirant · 2019-11-30T06:47:55Z

Now getting this permission error as follow..
1995821600:error:0200100D:system library:fopen:Permission denied:../crypto/bio/bss_file.c:288:fopen('/home/pi/key.pem','r')
1995821600:error:20074002:BIO routines:file_ctrl:system lib:../crypto/bio/bss_file.c:290:
1995821600:error:140B0002:SSL routines:SSL_CTX_use_PrivateKey_file:system lib:../ssl/ssl_rsa.c:540:

Try by running using sudo rights

IoTiA · 2019-11-30T06:59:30Z

Thank you for your help.Now server is start without any error, but the on that port can't listen anything .i start the server but on that port not listen anything
pi@raspberrypi:~ $ sudo lsof -n -P -i | grep LISTEN
sshd 444 root 3u IPv4 14433 0t0 TCP *:22 (LISTEN)
sshd 444 root 4u IPv6 14435 0t0 TCP *:22 (LISTEN)
exim4 748 Debian-exim 3u IPv4 15519 0t0 TCP 127.0.0.1:25 (LISTEN)
exim4 748 Debian-exim 4u IPv6 15520 0t0 TCP [::1]:25 (LISTEN)

TechAspirant · 2019-11-30T07:04:28Z

sudo lsof -n -P -i | grep LISTEN

I tried running the same command sudo lsof -n -P -i | grep LISTEN
I can see my broker executable listening on mentioned port no. in conf file. Make sure port no. is not conflicting

IoTiA · 2019-11-30T07:08:21Z

which command you use to start the broker.

TechAspirant · 2019-11-30T07:11:49Z

which command you use to start the broker.

./sol -c /home/user/Desktop/mqttbroker/sol-master/conf/sol.conf

IoTiA · 2019-11-30T07:18:58Z

I am using the same command still port is not listen anything.

TechAspirant · 2019-11-30T08:27:24Z

I am using the same command still port is not listen anything.

Any error after running command ?
Can you show me once the conf file ?

IoTiA · 2019-11-30T08:37:08Z

Here i attach my config file.
config.txt

TechAspirant · 2019-11-30T08:41:06Z

Here i attach my config file.
config.txt

Have you configured same configuration in conf folder /sol.conf file ?
As after running ./sol -c /home/user/Desktop/mqttbroker/sol-master/conf/sol.conf it loads the configuration file parameters

IoTiA · 2019-11-30T08:43:13Z

no it's not load the configuration parameter. when i start the server it look like as ...
1575098767 i Sol v0.7.1 is starting
1575098767 i Network settings:
1575098767 i Socket family: Unix
1575098767 i Unix socket: /tmp/sol.sock
1575098767 i Max request size: 50Mb
1575098767 i Logging:
1575098767 i level: DEBUG
1575098767 i logpath: /tmp/sol.log
1575098767 i Max memory: 2Gb
1575098767 i Server start

IoTiA · 2019-11-30T08:44:03Z

Can you share me your /sol.conf file .

TechAspirant · 2019-11-30T08:44:42Z

no it's not load the configuration parameter. when i start the server it look like as ...
1575098767 i Sol v0.7.1 is starting
1575098767 i Network settings:
1575098767 i Socket family: Unix
1575098767 i Unix socket: /tmp/sol.sock
1575098767 i Max request size: 50Mb
1575098767 i Logging:
1575098767 i level: DEBUG
1575098767 i logpath: /tmp/sol.log
1575098767 i Max memory: 2Gb
1575098767 i Server start

Socket family: Unix this option is in default .conf file.
Got to conf folder you will see sol.conf file Edit as per your requirement.

IoTiA · 2019-11-30T08:46:03Z

for my reference can you share your config file.

TechAspirant · 2019-11-30T08:47:35Z

for my reference can you share your config file.

ip_address 192.168.1.15
ip_port 8884
log_level DEBUG
log_path /home/user/Desktop/mqttbroker/MiMqtt.log
max_memory 2GB
max_request_size 50MB
tcp_backlog 128
stats_publish_interval 10s
certfile /home/user/Desktop/rootCACert.pem
keyfile /home/user/Desktop/rootCAKey.pem

IoTiA · 2019-11-30T08:54:42Z

thank you. now server start with the config parameter.but when i try to connect it server print msg dropping client .

TechAspirant · 2019-11-30T08:56:29Z

I am also on the same boat....Still trying to figure out whats wrong. If you found any solution do let me know.

IoTiA · 2019-11-30T08:58:41Z

you having same issue
1575103826 i Sol v0.7.1 is starting
1575103826 i Network settings:
1575103826 i Socket family: Tcp
1575103826 i Address: 192.168.1.99
1575103826 i Port: 8883
1575103826 i Tcp backlog: 128
1575103826 i Keepalive: 60
1575103826 i SSL: 1
1575103826 i Max request size: 50Mb
1575103826 i Logging:
1575103826 i level: DEBUG
1575103826 i logpath: /tmp/sol.log
1575103826 i Max memory: 2Gb
1575103826 i Server start
1575103826 ! Dropping client
1575103828 ! Dropping client
1575103829 ! Dropping client
1575103830 ! Dropping client
1575103831 ! Dropping client
1575103832 ! Dropping client

TechAspirant · 2019-11-30T08:59:18Z

you having same issue
1575103826 i Sol v0.7.1 is starting
1575103826 i Network settings:
1575103826 i Socket family: Tcp
1575103826 i Address: 192.168.1.99
1575103826 i Port: 8883
1575103826 i Tcp backlog: 128
1575103826 i Keepalive: 60
1575103826 i SSL: 1
1575103826 i Max request size: 50Mb
1575103826 i Logging:
1575103826 i level: DEBUG
1575103826 i logpath: /tmp/sol.log
1575103826 i Max memory: 2Gb
1575103826 i Server start
1575103826 ! Dropping client
1575103828 ! Dropping client
1575103829 ! Dropping client
1575103830 ! Dropping client
1575103831 ! Dropping client
1575103832 ! Dropping client

Yes

IoTiA · 2019-11-30T09:00:52Z

ok. if i find any solution i let you know.

codepr · 2019-11-30T10:58:11Z

Working on it. It's something the client doesn't like about self-signed certs I think. As of now the connection is correctly established by setting ssl.CERT_NONE on the client side:

import ssl
import paho.mqtt.client as mqtt

client = mqtt.Client('ssl-test')
client.tls_set(certfile='cert.pem', keyfile='key.pem', tls_version=ssl.PROTOCOL_SSLv23, cert_reqs=ssl.CERT_NONE)
client.connect('127.0.0.1', 1883)

I think the correct way to make it work is to first generate a CA, then generate certificates signed with that CA, this way the server will correctly provide signed certificates expected by the client with the same CA set.

nsf747 · 2019-12-02T03:42:50Z

I think the correct way to make it work is to first generate a CA, then generate certificates signed with that CA, this way the server will correctly provide signed certificates expected by the client with the same CA set.

This is probably the best way to go as most other MQTT brokers use this method.

IoTiA · 2019-12-02T06:20:15Z

Still i am facing the " Dropping client" error..

ngzx1212 · 2019-12-04T09:37:03Z

Hello,
I try to run this mqtt source code with ssl configuration but it doesn,t work.
Try to connect client to mqtt server using ssl it gives error "dropping client".Any solution to
work with ssl.What changes required to fine working of ssl.

ngzx1212 · 2019-12-04T09:38:47Z

please update how to work ssl with this mqtt server.

zlxqur · 2019-12-09T10:40:26Z

I am facing the same issue with ssl. Anyone find the solution for work ssl .

zlxqur · 2019-12-10T11:45:52Z

Is there any update on ssl.

codepr · 2019-12-11T15:48:24Z

I pushed some updates, as of now TLS should work, but expect some bugs.
I tested the server using mosquitto clients.

$ mosquitto_sub -v -p 1234 -t test --cafile certs/ca.crt --cert certs/cert.crt --key certs/cert.key

and

$ mosquitto_pub -t test -m hello -p 1234 --cafile certs/ca.crt --cert certs/cert.crt --key certs/cert.key

Cya.

zlxqur · 2019-12-14T11:09:57Z

I tried to test with above command it gives the error as:
Error: Problem setting TLS options.

zlxqur · 2019-12-14T11:16:46Z

The sol server can't accept the client certificate. With client side add client certificate it gives the error.
1576318827 ! Closing connection with 145.278.1.30: -1
Segmentation fault

nsf747 · 2019-12-16T04:11:51Z

I was facing the same thing. But it worked perfectly using MQTT Box in CA Server self signed certificate mode. Apparently if you configure the client to accept the certificate from the broker, rather than pointing the crt files to the client manually, it works perfectly.

codepr · 2019-12-16T12:30:47Z

Thx for the feedbacks, I'll investigate these curious cases during holidays.

zlxqur · 2019-12-17T11:08:29Z

I was trying with this command given as follow
mosquitto_pub -t test -m hello -p 1234 --cafile certs/ca.crt --cert certs/cert.crt --key certs/cert.key
It gives the error.Error: Problem setting TLS options.
How can i use client certs to publish message with ssl/tls.

zlxqur · 2019-12-18T08:40:12Z

Still i am facing the issue with ssl. I can't work with ssl.

zlxqur · 2019-12-20T04:19:41Z

I trying to connect with the broker with ssl client cert it gives the error:
1576814559 Connection from 152.188.1.74
1576814560 Closing connection with 152.188.1.74: Client disconnected

zlxqur · 2020-01-21T06:24:37Z

I have problem in password.py file.The password .py having the error during the run time.

codepr mentioned this issue Nov 11, 2019

SSL/TLS Implementation #2

Closed

codepr added a commit that referenced this issue Dec 12, 2019

Adding cleaning resources for unexpected dc's, still bugged. WIP #1

c166fd2

hongliang5316 mentioned this issue May 5, 2020

Many data races when use -fsanitize=thread #7

Closed

SSL / TLS Implementation #1

SSL / TLS Implementation #1

Comments

TechAspirant commented Nov 9, 2019

codepr commented Nov 11, 2019

nsf747 commented Nov 13, 2019

codepr commented Nov 19, 2019 • edited Loading

TechAspirant commented Nov 23, 2019 • edited Loading

codepr commented Nov 28, 2019

TechAspirant commented Nov 28, 2019 • edited Loading

TechAspirant commented Nov 29, 2019 • edited Loading

nsf747 commented Nov 30, 2019

IoTiA commented Nov 30, 2019

TechAspirant commented Nov 30, 2019

TechAspirant commented Nov 30, 2019

IoTiA commented Nov 30, 2019

TechAspirant commented Nov 30, 2019 • edited Loading

IoTiA commented Nov 30, 2019

TechAspirant commented Nov 30, 2019

IoTiA commented Nov 30, 2019

TechAspirant commented Nov 30, 2019

IoTiA commented Nov 30, 2019

TechAspirant commented Nov 30, 2019

IoTiA commented Nov 30, 2019

TechAspirant commented Nov 30, 2019

IoTiA commented Nov 30, 2019

TechAspirant commented Nov 30, 2019

IoTiA commented Nov 30, 2019

IoTiA commented Nov 30, 2019

TechAspirant commented Nov 30, 2019

IoTiA commented Nov 30, 2019

TechAspirant commented Nov 30, 2019

IoTiA commented Nov 30, 2019

TechAspirant commented Nov 30, 2019

IoTiA commented Nov 30, 2019

TechAspirant commented Nov 30, 2019

IoTiA commented Nov 30, 2019

codepr commented Nov 30, 2019 • edited Loading

nsf747 commented Dec 2, 2019

IoTiA commented Dec 2, 2019

ngzx1212 commented Dec 4, 2019

ngzx1212 commented Dec 4, 2019

zlxqur commented Dec 9, 2019

zlxqur commented Dec 10, 2019

codepr commented Dec 11, 2019

zlxqur commented Dec 14, 2019

zlxqur commented Dec 14, 2019

nsf747 commented Dec 16, 2019 • edited Loading

codepr commented Dec 16, 2019

zlxqur commented Dec 17, 2019 • edited Loading

zlxqur commented Dec 18, 2019

zlxqur commented Dec 20, 2019

zlxqur commented Jan 21, 2020

codepr commented Nov 19, 2019 •

edited

Loading

TechAspirant commented Nov 23, 2019 •

edited

Loading

TechAspirant commented Nov 28, 2019 •

edited

Loading

TechAspirant commented Nov 29, 2019 •

edited

Loading

TechAspirant commented Nov 30, 2019 •

edited

Loading

codepr commented Nov 30, 2019 •

edited

Loading

nsf747 commented Dec 16, 2019 •

edited

Loading

zlxqur commented Dec 17, 2019 •

edited

Loading