Enable TLS for all Puppet Service Clients of Wechaty Ecosystem

[huan]

Refer to #124, we have enforced the Wechaty Puppet Service to use TLS for maximum security, and satisfy the gRPC requirements.

The old versions of wechaty-puppet-service prior to version 0.28 will not be able to work with the new versions by default.

So the actions need to be taken for our community to enable the TLS for our ecosystem should at least be include the following tasks:

1. Enable TLS for Polyglot Wechaty SDK
   • Wechaty (TypeScript)
   • Python Wechaty @wechaty/python
   • Go Wechaty @wechaty/go
   • Java Wechaty @wechaty/java
   • .NET Wechaty @wechaty/dotnet
   • PHP Wechaty @wechaty/php
   • Rust Wechaty @wechaty/rust
   • Scala Wechaty @wechaty/scala
2. Enable TLS for Puppet Services @wechaty/contributors
   • WXWork @wechaty/juzi
   • Paimon @zpaimon
   • Donut

To be compatible with the old ecosystem, the new version of wechaty-puppet-services provided the following two environment variables to be used for compatible reasons. Please notice that this solution is a workaround, and we should push all of our ecosystems to move forward to work with the latest TLS/TLS versions.

Disable TLS for Puppet Service Server

To disable server TLS:

1. Set WECHATY_PUPPET_SERVICE_NO_TLS_INSECURE_SERVER to true
2. Set options.tls.disable to true

Disable TLS for Puppet Service Client

To disable client TLS:

1. Set WECHATY_PUPPET_SERVICE_NO_TLS_INSECURE_CLIENT to true
2. Set options.tls.disable to true

Compatible with non-tls server/clients

We have workarounds to make a new version of wechaty-puppet-service work with the old non-tls server/clients.

To disable tls for server / client, we can set NO_TLS_INSECURE/options.tls.disable:

1. Should not be used for production
2. It's compatible with old wechaty-puppet-service servers and clients which are not supported TLS.
3. All wechaty-puppet-service servers and clients should be updated to the latest version to support TLS as soon as possible.
4. All Polyglot Wechaty SDK should support TLS as soon as possible.

Troubleshooting

@Gcaufy: try to enable grpc trace log, you will receive more connection details

GRPC_VERBOSITY=DEBUG GRPC_TRACE=all npm run start

Problem

You will run into Error: 14 UNAVAILABLE: No connection established if the Wechaty Puppet Service server & client does not match the TLS settings.

For examples:

1. Server requires TLS but the client does not support TLS, for example, a newer server with a legacy client
2. Server does not support TLS but the client is using TLS, for example, a legacy server with a newer client

Solution

Server	Client	Status	Solution (workaround)
TLS	no TLS	Error: 14 UNAVAILABLE	WECHATY_PUPPET_SERVICE_NO_TLS_INSECURE_SERVER=true
no TLS	TLS	Error: 14 UNAVAILABLE	WECHATY_PUPPET_SERVICE_NO_TLS_INSECURE_CLIENT=true
no TLS	no TLS	OK	N/A
TLS	TLS	OK	N/A

[su-chang]

Basic Info

Client: wechaty@0.68.1

set WECHATY_PUPPET_SERVICE_NO_TLS_INSECURE_CLIENT=true

Server: wechaty@0.62.3

Log

14:53:12 VERB PuppetService start()
14:53:12 SILL StateSwitch <PuppetService> on() is false
14:53:12 SILL StateSwitch <PuppetService> on() is false
14:53:12 VERB StateSwitch <PuppetService> on(pending) <- (false)
14:53:12 VERB GrpcClient constructor({"token":"puppet_wxwork_a4ef912c15241f33"})
14:53:12 VERB GrpcClient constructor() tlsRootCert(hash): "88ab8b62777f0fb1179ec567c5019c31d646428cc0a3fdb007fcffa1e9fac5a8"
14:53:12 VERB WechatyToken constructor("puppet_wxwork_a4ef912c15241f33")
14:53:12 VERB GrpcClient constructor() token: "puppet_wxwork_a4ef912c15241f33"
14:53:12 VERB GrpcClient constructor() endpoint: "wechaty://api.chatie.io/puppet_wxwork_a4ef912c15241f33"
14:53:12 VERB GrpcClient constructor() disableTls: "true"
14:53:12 VERB GrpcClient constructor() serverName(SNI): "puppet_wxwork"
14:53:12 VERB PuppetService bridgeGrpcEventStream(client)
14:53:12 VERB GrpcClient start()
14:53:12 VERB GrpcClient init()
14:53:12 WARN GrpcClient init() TLS disabled: INSECURE!
14:53:12 VERB GrpcClient startStream()
14:53:12 ERR PuppetService start() rejection: 14 UNAVAILABLE: Failed to parse DNS address dns:wechaty://api.chatie.io/puppet_wxwork_a4ef912c15241f33
Error: 14 UNAVAILABLE: Failed to parse DNS address dns:wechaty://api.chatie.io/puppet_wxwork_a4ef912c15241f33
    at Object.callErrorFromStatus (/Users/suchang/Desktop/PROJECT/Test/testPuppet/node_modules/wechaty-grpc/node_modules/@grpc/grpc-js/src/call.ts:81:24)
    at Object.onReceiveStatus (/Users/suchang/Desktop/PROJECT/Test/testPuppet/node_modules/wechaty-grpc/node_modules/@grpc/grpc-js/src/client.ts:574:32)
    at Object.onReceiveStatus (/Users/suchang/Desktop/PROJECT/Test/testPuppet/node_modules/wechaty-grpc/node_modules/@grpc/grpc-js/src/client-interceptors.ts:389:48)
    at /Users/suchang/Desktop/PROJECT/Test/testPuppet/node_modules/wechaty-grpc/node_modules/@grpc/grpc-js/src/call-stream.ts:276:24
    at processTicksAndRejections (internal/process/task_queues.js:77:11)
14:53:12 VERB GrpcClient stop()
14:53:12 VERB GrpcClient stopStream()
14:53:12 VERB GrpcClient no eventStream when stop, skip destroy.

Question

How to Set options.ssl.disable to true ?

[huan]

I have just tired, and it works with the latest wechaty-getting-started repo without any problem:

huan@dev:~/git/wechaty/wechaty-getting-started$ \
WECHATY_LOG=verbose \
WECHATY_PUPPET=wechaty-puppet-service \
WECHATY_PUPPET_SERVICE_TOKEN=puppet_wxwork_a4ef912c15241f33 \
WECHATY_PUPPET_SERVICE_NO_TLS_INSECURE_CLIENT=true \
npm start

> wechaty-getting-started@0.11.12 start /home/huan/git/wechaty/wechaty-getting-started
> cross-env NODE_OPTIONS='--unhandled-rejections=strict' ts-node examples/ding-dong-bot.ts

18:52:00 INFO Config registering process.on("unhandledRejection") for development/debug
18:52:00 VERB Config constructor()
18:52:00 VERB Wechaty init() getRavenDsn() return undefined, skipped.
18:52:01 VERB wechaty-puppet-service monkeyPatchMetadataFromHttp2Headers()
18:52:01 VERB ResolverWechaty setup()
18:52:01 VERB Wechaty constructor()
18:52:01 VERB StateSwitch constructor(Wechaty, "{"log":{"enableTimestamp":true,"logLevel":4,"prefixFilter":{}}}")
18:52:01 VERB StateSwitch constructor(WechatyReady, "{"log":{"enableTimestamp":true,"logLevel":4,"prefixFilter":{}}}")
18:52:01 VERB Wechaty on(scan, listener) registering... listenerCount: 0
18:52:01 VERB Wechaty on(login, listener) registering... listenerCount: 0
18:52:01 VERB Wechaty on(logout, listener) registering... listenerCount: 0
18:52:01 VERB Wechaty on(message, listener) registering... listenerCount: 0
18:52:01 VERB Wechaty <wechaty-puppet-service>(ding-dong-bot) start() v0.68.1 is starting...
18:52:01 VERB Wechaty id: cksu8jjmi0000p614drv9fnaf
18:52:01 VERB StateSwitch <WechatyReady> off(true) <- (true)
18:52:01 VERB StateSwitch <Wechaty> on(pending) <- (false)
18:52:01 VERB MemoryCard constructor("ding-dong-bot")
18:52:01 VERB MemoryCard getStorage() for storage type: N/A
18:52:01 VERB getStorage name: ding-dong-bot, options: {"type":"file"}
18:52:01 VERB StorageFile constructor(ding-dong-bot, ...)
18:52:01 VERB StorageBackend constructor(ding-dong-bot, { type: file })
18:52:01 VERB MemoryCard load() from storage: StorageFile</home/huan/git/wechaty/wechaty-getting-started/ding-dong-bot.memory-card.json>
18:52:01 VERB StorageFile load() from /home/huan/git/wechaty/wechaty-getting-started/ding-dong-bot.memory-card.json
18:52:01 VERB Wechaty initPuppet() 
18:52:01 VERB MemoryCard multiplex(puppet)
18:52:01 VERB MemoryCard static multiplex(MemoryCard<ding-dong-bot>, puppet)
18:52:01 VERB MemoryCard constructor({"name":"ding-dong-bot","multiplex":{"name":"puppet","parent":{"options":{"name":"ding-dong-bot"},"name":"ding-dong-bot","payload":{"\rpuppet\nPUPPET_WECHAT":[{"name":"MM_WX_NOTIFY_STATE","value":"1","domain":"wx.qq.com","path":"/","expires":-1,"size":19,"httpOnly":false,"secure":false,"session":true,"sameParty":false},{"name":"webwx_auth_ticket","value":"CIsBEKzkmJYJGoABaBXhI/5FCBu/5VVVz0w2aSrD16S6s/974p5vBvVjVXdKSwmeENDEIB8elyA9ZuYUetfecrVsM1m7Jwgs4qGhVw4rjZPbi3+k9b/2eFClYBckPFiaw7eUhviBS2gdttLC+gHTe691gLyRxdAPdZpWe24yQOK4hjKkd1yvS66/vPo=","domain":".wx.qq.com","path":"/","expires":1945410370,"size":205,"httpOnly":false,"secure":true,"session":false,"sameParty":false},{"name":"MM_WX_SOUND_STATE","value":"1","domain":"wx.qq.com","path":"/","expires":-1,"size":18,"httpOnly":false,"secure":false,"session":true,"sameParty":false},{"name":"webwxuvid","value":"e06c99933f23fb84ba11284a38a3d47dedfc1b6954365741db86802c47bced7c7513fd698bf893367dcecdaec2c71b27","domain":".wx.qq.com","path":"/","expires":1945410370,"size":105,"httpOnly":false,"secure":true,"session":false,"sameParty":false},{"name":"mm_lang","value":"en_US","domain":".wx.qq.com","path":"/","expires":1630093570,"size":12,"httpOnly":false,"secure":true,"session":false,"sameParty":false},{"name":"wxloadtime","value":"1630050370","domain":".wx.qq.com","path":"/","expires":1630093570,"size":20,"httpOnly":false,"secure":true,"session":false,"sameParty":false},{"name":"wxsid","value":"7zkxcLEiIONpOfL6","domain":".wx.qq.com","path":"/","expires":1630093570,"size":21,"httpOnly":false,"secure":true,"session":false,"sameParty":false},{"name":"webwx_data_ticket","value":"gSffJKZd26otJgTGu8ZlcjVd","domain":".qq.com","path":"/","expires":1630093570,"size":41,"httpOnly":false,"secure":true,"session":false,"sameParty":false},{"name":"wxuin","value":"62714345","domain":".wx.qq.com","path":"/","expires":1630093570,"size":13,"httpOnly":false,"secure":true,"session":false,"sameParty":false}]},"multiplexNameList":[],"storage":{"name":"ding-dong-bot","options":{"type":"file"},"absFileName":"/home/huan/git/wechaty/wechaty-getting-started/ding-dong-bot.memory-card.json"}}}})
18:52:01 VERB PuppetManager resolve({puppet: wechaty-puppet-service, puppetOptions: undefined})
18:52:01 VERB PuppetManager resolveName(wechaty-puppet-service)
18:52:01 VERB PuppetManager checkModule(wechaty-puppet-service)
18:52:01 VERB Puppet constructor({}) #0
18:52:01 VERB StateSwitch constructor(PuppetService, "{"log":{"enableTimestamp":true,"logLevel":4,"prefixFilter":{}}}")
18:52:01 VERB MemoryCard constructor(undefined)
18:52:01 VERB MemoryCard getStorage() for storage type: N/A
18:52:01 VERB MemoryCard load() from storage: N/A
18:52:01 VERB MemoryCard load() no storage
18:52:01 VERB Puppet constructor() watchdog timeout set to 60 seconds
18:52:01 VERB HotImport callerResolve(., /home/huan/git/wechaty/wechaty-getting-started/node_modules/wechaty-puppet/dist/src/puppet.js)
18:52:01 VERB Puppet constructor() childClassPath=/home/huan/git/wechaty/wechaty-getting-started/node_modules/wechaty-puppet-service/dist/src/client
18:52:01 VERB PayloadStore constructor({"token":"puppet_wxwork_a4ef912c15241f33"})
18:52:01 VERB PuppetService hookPayloadStore()
18:52:01 VERB Puppet setMemory()
18:52:01 VERB Wechaty initPuppetEventBridge(Puppet#0<PuppetService>(ding-dong-bot))
18:52:01 VERB Wechaty initPuppetEventBridge() puppet.on(friendship) (listenerCount:0) registering...
18:52:01 VERB Wechaty initPuppetEventBridge() puppet.on(login) (listenerCount:1) registering...
18:52:01 VERB Wechaty initPuppetEventBridge() puppet.on(logout) (listenerCount:1) registering...
18:52:01 VERB Wechaty initPuppetEventBridge() puppet.on(message) (listenerCount:0) registering...
18:52:01 VERB Wechaty initPuppetEventBridge() puppet.on(room-invite) (listenerCount:0) registering...
18:52:01 VERB Wechaty initPuppetEventBridge() puppet.on(room-join) (listenerCount:0) registering...
18:52:01 VERB Wechaty initPuppetEventBridge() puppet.on(room-leave) (listenerCount:0) registering...
18:52:01 VERB Wechaty initPuppetEventBridge() puppet.on(room-topic) (listenerCount:0) registering...
18:52:01 VERB Wechaty initPuppetEventBridge() puppet.on(scan) (listenerCount:0) registering...
18:52:01 VERB Wechaty initPuppetEventBridge() puppet.on(dirty) (listenerCount:0) registering...
18:52:01 VERB Wechaty initPuppetEventBridge() puppet.on(dong) (listenerCount:0) registering...
18:52:01 VERB Wechaty initPuppetEventBridge() puppet.on(error) (listenerCount:0) registering...
18:52:01 VERB Wechaty initPuppetEventBridge() puppet.on(heartbeat) (listenerCount:0) registering...
18:52:01 VERB Wechaty initPuppetEventBridge() puppet.on(ready) (listenerCount:0) registering...
18:52:01 VERB Wechaty initPuppetEventBridge() puppet.on(reset) (listenerCount:0) registering...
18:52:01 VERB Wechaty wechatifyUserModules(Puppet#0<PuppetService>(ding-dong-bot))
18:52:01 VERB PuppetService start()
18:52:01 VERB StateSwitch <PuppetService> on(pending) <- (false)
18:52:01 VERB GrpcClient constructor({})
18:52:01 VERB GrpcClient constructor() tlsRootCert(hash): "88ab8b62777f0fb1179ec567c5019c31d646428cc0a3fdb007fcffa1e9fac5a8"
18:52:01 VERB WechatyToken constructor("puppet_wxwork_a4ef912c15241f33")
18:52:01 VERB GrpcClient constructor() token: "puppet_wxwork_a4ef912c15241f33"
18:52:01 VERB GrpcClient constructor() endpoint: "wechaty://api.chatie.io/puppet_wxwork_a4ef912c15241f33"
18:52:01 VERB GrpcClient constructor() disableTls: "true"
18:52:01 VERB GrpcClient constructor() serverName(SNI): "puppet_wxwork"
18:52:01 VERB PuppetService bridgeGrpcEventStream(client)
18:52:01 VERB GrpcClient start()
18:52:01 VERB GrpcClient init()
18:52:01 WARN GrpcClient init() TLS disabled: INSECURE!
18:52:01 VERB WechatyResolver constructor("{"scheme":"wechaty","authority":"api.chatie.io","path":"puppet_wxwork_a4ef912c15241f33"}",)
18:52:01 VERB GrpcClient startStream()
18:52:01 VERB ResolverWechaty updateResolution()
18:52:01 VERB WechatyToken constructor({"authority":"api.chatie.io","token":"puppet_wxwork_a4ef912c15241f33"})
18:52:01 VERB WechatyToken discover() for "puppet_wxwork_a4ef912c15241f33"
18:52:02 VERB ResolverWechaty getDefaultAuthority({"scheme":"wechaty","authority":"api.chatie.io","path":"puppet_wxwork_a4ef912c15241f33"})
18:52:02 VERB PuppetService onGrpcStreamEvent({type:EVENT_TYPE_LOGIN(25), payload(32)})
18:52:02 VERB PuppetService hookPayloadStore() this.on(login) contactId: "1688851062937185"
18:52:02 VERB PayloadStore start(1688851062937185)
18:52:02 VERB FlashStore constructor(/home/huan/.wechaty/wechaty-puppet-service-v0.30/flash-store-v1.1/puppet_wxwork_a4ef912c15241f33/1688851062937185/contact-payload)
18:52:02 VERB FlashStore constructor(/home/huan/.wechaty/wechaty-puppet-service-v0.30/flash-store-v1.1/puppet_wxwork_a4ef912c15241f33/1688851062937185/room-member-payload)
18:52:02 VERB FlashStore constructor(/home/huan/.wechaty/wechaty-puppet-service-v0.30/flash-store-v1.1/puppet_wxwork_a4ef912c15241f33/1688851062937185/room-payload)
18:52:02 VERB Contact load(1688851062937185) init pool
18:52:02 VERB PuppetService contactRawPayload(1688851062937185)
18:52:02 VERB FlashStore get(1688851062937185)
18:52:02 VERB StateSwitch <PuppetService> on(true) <- (pending)
18:52:02 VERB PuppetService recover$() switchOn$ fired
18:52:02 VERB PuppetService recover$() heartbeat: undefined
18:52:02 VERB PuppetService recover$() switchOn$ fired
18:52:02 VERB PuppetService recover$() heartbeat: undefined
18:52:02 VERB Wechaty on(heartbeat, listener) registering... listenerCount: 0
18:52:02 VERB StateSwitch <Wechaty> on(true) <- (pending)
18:52:02 INFO StarterBot Starter Bot Started.
18:52:02 VERB FlashStore set(1688851062937185, object)
18:52:02 INFO StarterBot Contact<曲洪峰> login
^C

Versions:

huan@dev:~/git/wechaty/wechaty-getting-started$ npm ls wechaty
npm ls wechaty-wechaty-getting-started@0.11.12 /home/huan/git/wechaty/wechaty-getting-started
└── wechaty@0.68.1 

huan@dev:~/git/wechaty/wechaty-getting-started$ npm ls wechaty-pppet-service
wechaty-getting-started@0.11.12 /home/huan/git/wechaty/wechaty-getting-started
└─┬ wechaty@0.68.1
  └── wechaty-puppet-service@0.30.1 

Please try it again and let me know if it works for you.

[su-chang]

Thank you for your test. I have run it works in wechaty-getting-started. But still failed in my workspace, I will try to figure out it later.

[huan]

Great to know that!

Please remember that it will be always a good idea to use wechaty-getting-started to test because it can give you a fresh setup.

If you can run into any issues with the wechaty-getting-started repo, then it will always mean that we can easily reproduce it, which is very important for debugging.