Streamline the swiftly init process #177

cmcgee1024 · 2024-10-20T18:24:19Z

The init process just installs swiftly itself at the moment. Most
users will immediately install a swift toolchain, most likely the
latest available one. On Linux, there's confusing gpg messages that
most users don't need to be aware.

The init process will provide a summary of things that are going
to happen to the user's system, including the addition of GnuPG keys
on Linux, and the installation of the latest swift toolchain so that
they can agree, or abort the entire process. When the process runs
the user is given line-level and high level processes, not internal
details. Add a verbose mode for more details, such as the messages
that come from GnuPG on Linux.

Add an option to the init subcommand to allow swiftly to be
installed without the latest available swift toolchain so that
advanced users can decide how to install a toolchain themselves
after the swiftly installation.

Update the documentation with the increase in automation.

Add a design proposal for the new swiftly proxy system

…t-version file

…ocation clarify the boundaries of the swiftly toolchain abstraction and elaborate on how to work around them

…new swiftly run command

Change the nature of the swiftly symlinks so that they point to the swiftly executable at install time. These do not change when new toolchains are used. Toolchain selection happens each time when the proxies are run. The proxies are created for a well-known set of toolchain binaries that are constant for a wide variety of toolchain versions and platforms. Add support for .swift-version files for toolchain selection. Update the use command so that it can point out which toolchain is in use based on context, such as swift version files that are located in the current working directory or above. The fallback selection comes from the global default configuration's 'inUse' setting. When querying for what's in use the global default is shown with the "(default)" tag. If the in-use toolchain is selected by a swift-version file the path to that file is displayed. Provide a print location flag to the use subcommand that can print the file path of the toolchain that is in use in the current context. When using a new toolchain, depending on whether a swift version is selecting the current one, update the swift version file with the selected toolchain version. If no swift version file can be located, attempt to create a new one at the top of the git worktree. If there is no git worktree, then fallback to updating the global default in the configuration. Provide a global default flag for the use subcommand so that only the global default in-use toolchain is considered and not any of the swift version files.

…esult and centralized error messages

Provide a run command that allows arbitrary commands to be run in the context of the selected toolchain. Provide a one-off selection mechanism with a special syntax on the run command.

With no arguments the install subcommand will install the currently selected toolchain from the `.swift-version` files.

…swiftly install`. Guard automatic creation of .swift-version file from `swiftly use` around a prompt overridable using an `--assume-yes`. Minor cleanup

Fix all of the swift.org urls so that they use www.swift.org to avoid redirection

Fix symlink target selection for swiftly when it is system managed

Print the error in a better way

Add support for these new platforms to swiftly, the autodetection, the tests, and docker support. Remove the swiftly installer portion of the code.

…tection coverage

…om the tests

cmcgee1024 · 2024-10-20T21:44:30Z

@swift-ci test macOS

cmcgee1024 · 2024-10-21T10:52:41Z

macOS tests are passing:

Test Suite 'All tests' passed at 2024-10-20 21:52:52.505.
	 Executed 67 tests, with 0 failures (0 unexpected) in 291.251 (291.260) seconds

…x/macOS and hide platform-specific details

…ux_platforms

…lled

…ly into streamlined_init

cmcgee1024 · 2024-10-29T13:53:38Z

@swift-ci test macOS

cmcgee1024 · 2024-10-29T17:38:51Z

@swift-ci test macOS

Provide more verbose untarring messages in Linux behind the verbose flag

cmcgee1024 · 2024-10-30T17:34:47Z

@swift-ci test macOS

…_init

cmcgee1024 · 2024-11-21T14:57:44Z

@swift-ci test macOS

0xTim · 2024-11-21T16:58:54Z

Documentation/SwiftlyDocs.docc/getting-started.md


-```
-curl -L https://swiftlang.github.io/swiftly/swiftly-install.sh | bash


Is there any reason we can't offer the script as an alternative? Removing it completely makes install Swiftly and Swift a reasonably amount more complicated as it puts the onus on the user to work out what OS and architecture they're using, which they can get wrong and make it seem like Swift doesn't work.

Additionally this makes building tooling on top of Swiftly significantly more complicated and each tool has to do the same logic of working out what system it's running on and going to find the correct binary which is a lot of duplicated work. It would be good to have that centralised in the tool.

Finally, this reduces the 'copy and paste this one line' installation for Swift that we have had up to now, making the experience on installing Swift on different systems a step back IMO compared to say rustup.

There are number of ways that we're hoping to address these as we can't use a curl | bash approach.

For the regular user install workflow, the website can help to find them the most likely swiftly download candidate for their OS and architecture. More details can be found in the PR here: swiftlang/swift-org-website#830

We can also add a very short guide to help them on the next steps. Feedback is welcome on that PR.

The download for macOS brings them to the pkg installer for the easy flow. It can also be invoked headlessly on the command-line either in a system directory or the user's home directory. Details are in the docs.

Since swiftly is a Linux binary that can work on a wide variety of Linux distributions, it takes on the responsibility of Linux flavour detection, and configuration. The user needs only to untar it and run swiftly wherever it is to continue.

For automated systems, there's a document here that goes over how one would automatically install swiftly. Feedback is welcome on this document as part of a separate PR/issue: https://swiftpackageindex.com/swiftlang/swiftly/main/documentation/swiftlydocs/automated-install

In the slightly longer term, there is also the possibility of providing swiftly in system package managers to make this even easier. Swiftly is able to be run from a system location.

There are number of ways that we're hoping to address these as we can't use a curl | bash approach.
We can no longer use the install script for some reason? I agree with Tim that the one liner approach seems a bit more user friendly as it handles environment detection

I think that in the near term the website can do much to ease the new user flow to compensate, such as making sure that they have the right executable, escape hatch to the full list of executables. The URL when it's finally decided for the website, can be made to be friendly to uname -m and uname -s. System packages of swiftly will be an accelerator as well, once they are available.

But what's the actual reason "we can't use a curl | bash approach"? I fully support the plan to offer an executable that does more work and make it possible to install without requiring a shell script, but removing it all together as an option feels like a step backwards

cmcgee1024 · 2024-11-21T20:09:26Z

@swift-ci test macOS

justice-adams-apple

I think the commit history looks like it needs to be cleaned up a bit post proxy merge

justice-adams-apple · 2024-11-21T20:44:21Z

Documentation/SwiftlyDocs.docc/automated-install.md


 ```
-curl -L <trusted_location_of_swiftly> > swiftly
+curl -L <location_of_swiftly_swift_org> > swiftly.tar.gz


location_of_swiftly_swift_org

Does this exist? 🤔

It will once these website update PR's are merged with a released version of swiftly:

swiftlang/swift-org-website#828
swiftlang/swift-org-website#830

justice-adams-apple · 2024-11-21T20:49:02Z

Documentation/SwiftlyDocs.docc/getting-started.md


-```
-curl -L https://swiftlang.github.io/swiftly/swiftly-install.sh | bash


There are number of ways that we're hoping to address these as we can't use a curl | bash approach.
We can no longer use the install script for some reason? I agree with Tim that the one liner approach seems a bit more user friendly as it handles environment detection

justice-adams-apple · 2024-11-21T20:50:26Z

Documentation/SwiftlyDocs.docc/getting-started.md

 $ swift --version

-Swift version 5.8.1 (swift-5.8.1-RELEASE)
+Swift version 6.0.1 (swift-6.0.1-RELEASE)


a few lines down the doc references swift build with a commend saying latest (5.8.1) toolchain, probably just want to update that comment to 6.0.1 too

I can raise another PR to fresh the version numbers in the documentation, or make them more version independent. Suggestions are welcome on how best to handle this problem of balancing accuracy and maintainability.

Sources/SwiftlyCore/Platform.swift

justice-adams-apple · 2024-11-21T21:45:39Z

Sources/SwiftlyCore/Platform.swift

+            return swiftlyHomeBin
+        }
+
+        if installSwiftly {


This is the code that installs swiftly, yeah, and it's wrapped around a single boolean which changes the purpose of this method, should it just be its own method? We would essentially have:

findSwiftlyBin installSwiftly

Not a huge deal, just tripped me up when reviewing. Totally up to you

Thanks, I couldn't find a good abstraction, so maybe this deserves to have two similar functions beside each other for the locality of change, at least.

Co-authored-by: Justice Adams <[email protected]>

cmcgee1024 · 2024-11-22T13:05:41Z

I think the commit history looks like it needs to be cleaned up a bit post proxy merge

On this project I've tended to keep the full and ugly truth of the PR commit history because GitHub doesn't handle force pushes very well, and sometimes comments and their context gets fully lost. I try to compensate for this by most often using the squash and merge function when the PR is complete so that the main branch history is much cleaner.

cmcgee1024 · 2024-11-22T14:06:26Z

@swift-ci test macOS

cmcgee1024 · 2024-11-22T15:00:08Z

@swift-ci test macOS

…recting to swiftly init subcommand

cmcgee1024 · 2024-11-22T15:32:19Z

@swift-ci test macOS

cmcgee1024 added 30 commits August 10, 2024 08:16

DRAFT: Swiftly proxies

e4efbeb

Add a design proposal for the new swiftly proxy system

Add a swiftly install workflow where the version comes from the .swif…

3b5c404

…t-version file

update design to make auto-installation an error instead

956256f

provide a mechanism to find the currently in-use toolchain physical l…

375df29

…ocation clarify the boundaries of the swiftly toolchain abstraction and elaborate on how to work around them

add more details about the selector prefix, and methods to escape

ed68a9e

Restructure the PR to move the selector syntax from the proxies to a …

c745f4c

…new swiftly run command

Merge branch 'main' of github.com:cmcgee1024/swiftly into proxies-design

ae29e88

Rewrite the select toolchain function with a type for the selection r…

ce0d27a

…esult and centralized error messages

Update the documentation

66dd459

Create a swiftly run command

2df7357

Provide a run command that allows arbitrary commands to be run in the context of the selected toolchain. Provide a one-off selection mechanism with a special syntax on the run command.

Fix empty command case with a single ++++

8976bba

Write run command and proxy tests

3caab66

Regenerate the cli reference documentation

31f4327

Fix design document discrepancies and add install proxy argument tests

16caf80

Update the list command to decorate default, and in-use toolchains

b2aa165

Update list tests to check for in use and default labels

7504dd3

Make the version argument optional in the install subcommand

0e7b661

With no arguments the install subcommand will install the currently selected toolchain from the `.swift-version` files.

Fix case of empty bin directory when checking for overwrite

43d620a

Remove +install selector option from swift run in favour of regular `…

2e3c59d

…swiftly install`. Guard automatic creation of .swift-version file from `swiftly use` around a prompt overridable using an `--assume-yes`. Minor cleanup

Import GPG keys on every install to get new signing keys from swift.org

9dd640c

Fix all of the swift.org urls so that they use www.swift.org to avoid redirection

Make recommended documentation changes.

5e615ea

Fix symlink target selection for swiftly when it is system managed

Merge branch 'main' of github.com:cmcgee1024/swiftly into proxies-design

10a0856

Provide a better error message on swiftly install with no version

6d6050e

Print the error in a better way

Add new Linux platforms: Ubuntu 2404,2310, Debian 12, Fedora 39

883e638

Add support for these new platforms to swiftly, the autodetection, the tests, and docker support. Remove the swiftly installer portion of the code.

Add the new entries to the --platform option

0cc129c

Pull list of linux platforms out into a var.

afbd797

Remove all of the platform def environment variables to force auto-de…

d301ef3

…tection coverage

Remove last remnants of the special platform environment variables fr…

393f9d5

…om the tests

Update README, and add documentation for the new run subcommand

6989796

cmcgee1024 added 7 commits October 23, 2024 12:02

Add more detail to the getting started guide

4d833f8

Remove the word system from the missing dependencies message

bc83ee5

Add system package dependency list for Ubuntu 23.10

be65db1

Rework the getting started guide to have a platform selector for linu…

646b5cb

…x/macOS and hide platform-specific details

Merge branch 'main' of github.com:cmcgee1024/swiftly into add_new_lin…

4e7ca7c

…ux_platforms

Permit bare swiftly command to start the init workflow when not insta…

a08faad

…lled

Merge branch 'add_new_linux_platforms' of github.com:cmcgee1024/swift…

65da6d8

…ly into streamlined_init

cmcgee1024 added 2 commits October 29, 2024 11:51

Fix array index problem

970943f

Use result to simplify proxy init check

c43fba2

Quiet the macOS installer messages behind the verbose flag

bce8ab6

Provide more verbose untarring messages in Linux behind the verbose flag

cmcgee1024 added 2 commits November 21, 2024 09:48

Merge branch 'main' of github.com:cmcgee1024/swiftly into streamlined…

bbe7277

…_init

Fix merge remnants and reformat

35a05d0

cmcgee1024 marked this pull request as ready for review November 21, 2024 15:12

0xTim reviewed Nov 21, 2024

View reviewed changes

Fixes for the system managed detection and installation logic

d732a84

justice-adams-apple reviewed Nov 21, 2024

View reviewed changes

Update Sources/SwiftlyCore/Platform.swift

34e4021

Co-authored-by: Justice Adams <[email protected]>

Refactor find/install swiftly binary into two functions

55fc274

Re-introduce the bare swiftly invocation in an uninstalled state redi…

942c4e4

…recting to swiftly init subcommand

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Streamline the swiftly init process #177

Streamline the swiftly init process #177

cmcgee1024 commented Oct 20, 2024 •

edited

Loading

cmcgee1024 commented Oct 20, 2024

cmcgee1024 commented Oct 21, 2024

cmcgee1024 commented Oct 29, 2024

cmcgee1024 commented Oct 29, 2024

cmcgee1024 commented Oct 30, 2024

cmcgee1024 commented Nov 21, 2024

0xTim Nov 21, 2024

cmcgee1024 Nov 21, 2024

cmcgee1024 Nov 21, 2024

justice-adams-apple Nov 21, 2024

cmcgee1024 Nov 22, 2024

0xTim Nov 22, 2024

cmcgee1024 commented Nov 21, 2024

justice-adams-apple left a comment

justice-adams-apple Nov 21, 2024

cmcgee1024 Nov 22, 2024

justice-adams-apple Nov 21, 2024

justice-adams-apple Nov 21, 2024

cmcgee1024 Nov 22, 2024

justice-adams-apple Nov 21, 2024

cmcgee1024 Nov 22, 2024

cmcgee1024 commented Nov 22, 2024

cmcgee1024 commented Nov 22, 2024

cmcgee1024 commented Nov 22, 2024

cmcgee1024 commented Nov 22, 2024


		```
		curl -L https://swiftlang.github.io/swiftly/swiftly-install.sh \| bash

Streamline the swiftly init process #177

Are you sure you want to change the base?

Streamline the swiftly init process #177

Conversation

cmcgee1024 commented Oct 20, 2024 • edited Loading

cmcgee1024 commented Oct 20, 2024

cmcgee1024 commented Oct 21, 2024

cmcgee1024 commented Oct 29, 2024

cmcgee1024 commented Oct 29, 2024

cmcgee1024 commented Oct 30, 2024

cmcgee1024 commented Nov 21, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmcgee1024 commented Nov 21, 2024

justice-adams-apple left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmcgee1024 commented Nov 22, 2024

cmcgee1024 commented Nov 22, 2024

cmcgee1024 commented Nov 22, 2024

cmcgee1024 commented Nov 22, 2024

cmcgee1024 commented Oct 20, 2024 •

edited

Loading