Fix GPG check randomly fails

[sidepelican]

Problem

The install script randomly fails with an error like below.

Importing Swift's PGP keys...
gpg: directory '/home/runner/.gnupg' created
gpg: keybox '/home/runner/.gnupg/pubring.kbx' created
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0

Cause

Sometimes https://www.swift.org/keys/all-keys.asc returns a gzipped response, regardless of the request header's Accept-Encoding.

Pattern1

< HTTP/1.1 200 OK
< Server: Apple
< Date: Thu, 29 Aug 2024 02:19:07 GMT
< Content-Type: text/plain; charset=UTF-8
< Content-Length: 18563
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Last-Modified: Wed, 28 Aug 2024 07:22:09 GMT
< Accept-Ranges: bytes
< X-Frame-Options: SAMEORIGIN
< Strict-Transport-Security: max-age=31536000; includeSubdomains
< Cache-Control: max-age=180, public
< Etag: "4883-620b9374b8a40"
< Age: 2
< Via: https/1.1 jptyo7-edge-mx-017.ts.apple.com (acdn/255.14450), https/1.1 jptyo7-edge-fx-003.ts.apple.com (acdn/255.14450)
< X-Cache: hit-stale, hit-stale
< CDNUUID: 5da39fb1-3e44-440b-ab5d-f9dbba698545-515598867
< Connection: keep-alive
< 
[plain text output]

Pattern2(gzipped)

< HTTP/1.1 200 OK
< Server: Apple
< Date: Thu, 29 Aug 2024 02:18:41 GMT
< Content-Type: text/plain; charset=UTF-8
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Last-Modified: Wed, 28 Aug 2024 07:22:09 GMT
< X-Frame-Options: SAMEORIGIN
< Strict-Transport-Security: max-age=31536000; includeSubdomains
< Content-Encoding: gzip
< Cache-Control: max-age=180, public
< Etag: "4883-620b9374b8a40"
< Accept-Ranges: bytes
< Age: 0
< Content-Length: 11040
< Via: https/1.1 jptyo5-edge-lx-012.ts.apple.com (acdn/255.14450), https/1.1 jptyo5-edge-bx-029.ts.apple.com (acdn/255.14450)
< X-Cache: hit-stale, miss
< CDNUUID: 0bcda0aa-252d-45af-a452-36166b9e6d17-382020025
< Connection: keep-alive
< 
[binary output]

How to fix

Simply add the --compressed options to the curl command.
This option allows curl to handle gzipped bodies transparently.

Reference

• Add workaround for gzipped gpg keys to fix random CI failures swift-actions/setup-swift#680

[ADKaster]

I saw this error in a CI job I'm setting up and noticed that I can't work around it by importing the keys manually and using the --no-import-pgp-keys option, because it's not in the long opts list :(

swiftly/install/swiftly-install.sh

Line 352 in a7ef9e8

args=$(getopt --options ynohvp: --longoptions disable-confirmation,no-modify-profile,no-install-system-deps,help,version,platform:,overwrite --name swiftly-install -- "${@}")

[cmcgee1024]

This should be fixed in the new swiftly that uses Swift NIO for the http transfer, more robust to 302 redirects.

[cmcgee1024]

This should be fixed with the new swiftly 1.0.0 release due to the use of Swift NIO.

[sidepelican]

#162 (comment) Is it right?

It seems swiftly uses .shared (with no proxy) HTTPClient and it will not decompress gzipped response (default is .disabled).

https://github.com/swift-server/async-http-client/blob/01908f4f53751f24ee286353d4489efa125d2070/Sources/AsyncHTTPClient/HTTPClient.swift#L857

[cmcgee1024]

@sidepelican iirc the problem was resolved by changing the hostname for swift.org to www.swift.org, which yielded a payload that isn't gzip compressed, or at least one that the shared HTTPClient can process.

Have you encountered any problems with Linux and gpg verifications since the swiftly 1.0.0 release?

[sidepelican]

My local checking with curl, it seems that www.swift.org is still returning strange responses.
However, swiftly 1.0.0 seems working fine.
Since the number of attempts is small, it might just be a coincidence.

In any case, since the implementation in this PR is outdated, I’ll close this.

[kkebo]

Have you encountered any problems with Linux and gpg verifications since the swiftly 1.0.0 release?

Yes. So I had to execute curl --silent --retry 3 --location --fail --compressed https://swift.org/keys/all-keys.asc | gpg --import - before swiftly install.

The same issue has been fixed in swift-docker. Is there any chance to reopen (re-create?) this PR?

swiftlang/swift-docker#495

I may have made a mistake, so I will double-check.