diff --git a/Guides/docs.docc/Getting started.md b/Guides/docs.docc/Getting started.md new file mode 100644 index 000000000..eb2a66856 --- /dev/null +++ b/Guides/docs.docc/Getting started.md @@ -0,0 +1,162 @@ +# Getting started with Unidoc + +This guide outlines how to set up and run a local Unidoc server you can use to preview your documentation as it would appear on [Swiftinit](https://swiftinit.org). + +## Prerequisites + +You need to have a Swift Package Manager project. Unidoc itself has little relation to SPM, but it uses the Swift toolchain to compile your project, and the Swift toolchain uses SPM to build your project. + +You need to have [Docker](https://www.docker.com/) installed on your development machine. If you install it through your default package repository, we strongly recommend you ensure the installed Docker version is reasonably up-to-date (e.g. 24.0.0 or later). + +It is theoretically possible to run Unidoc without Docker, but Docker makes it much easier to do so, because you will not need to (directly) manage a local [MongoDB](https://mongodb.com) deployment. + + +## Setting up a local database + +A Unidoc server is a server that talks to a second server, a MongoDB server. Therefore, before you can start a Unidoc server, you need to set up and launch a local [MongoDB](https://github.com/tayloraswift/swift-mongodb) deployment. + +A MongoDB deployment consists of a [mongod](https://www.mongodb.com/docs/manual/reference/program/mongod/) process connected to a network that the Unidoc server is also connected to. + + +### Starting the database server + +We recommend hosting mongod inside a dedicated Docker container. This repository provides a Docker compose file that does this for you. + +To bring up the mongod instance (if it is not already online), run the following: + +```bash +docker compose -f Guides/docs.docc/local/docker-compose.yml up -d +``` + +The `-d` flag tells Docker to run the container in the background, so it does not block your terminal. + + +### About the Docker compose file + +If you are new to Docker, it is worth taking a moment to understand the Docker compose file. + +@Code(file: docker-compose.yml, title: docker-compose.yml) + +This file: + +1. Launches a container from the official [`mongo:latest`](https://hub.docker.com/_/mongo) image. +2. Sets the name of this container to `unidoc-mongod-container`. +3. Sets the **hostname** of the container to `unidoc-mongod` **within** the `unidoc-test` network. +4. Binds the container’s port 27017 to `localhost:27017`. This is the default port for the `mongod` process. +5. Mounts the startup scripts and data directory within the container. This allows the documentation data to persist across container restarts. +6. Passes a configuration file to the `mongod` process on startup. + +The `unidoc-test` network is helpful for testing, but for the purposes of this tutorial, you will mostly be accessing the `mongod` process through `localhost:27017`. + + +## Initializing the database + +The mongod instance will create a `.mongod` directory at the root of the cloned repository. This directory contains the state of the deployment, and like all database deployments, it can outlive the mongod process. This means you can kill (or crash) the mongod instance but it will not lose data unless you clear or delete its data directory. + +Initialize the replica set with: + +```bash +docker exec -t unidoc-mongod-container /bin/mongosh --file /unidoc-rs-init.js +``` + +This only needs to be done **once** per deployment lifecycle. (For example, after clearing the `.mongod` data directory.) + + +### Connecting to the database + +Once you have a `unidoc-mongod-container` running in the background, you can start a documentation server. There are many ways to run a documentation server, but if you are developing in a Docker container, the easiest way is compile Unidoc and run the server as a normal process. + +@Code(file: start-server.sh, title: start-server.sh) + + +### Generating certificates + +#### Unidoc < 0.17.0 + +If you are starting the server for the first time, you likely need to populate the `Assets/certificates/` directory with TLS certificates. See for instructions on how to do this. + +#### Unidoc ≥ 0.17.0 + +You do not need to generate certificates, as Unidoc 0.17.0 can run locally in insecure mode. + + +## Populating a local documentation server + +If you did all of the previous steps correctly, you should be able to navigate to [`localhost:8080/`](http://localhost:8080/) and view a blank homepage. + +> Note: +> If you are using HTTPS, you need to replace the scheme with `https://` and the port number with `8443`. + + +A fresh Unidoc database contains no documentation. Let’s build some now. + +### Building documentation for the standard library + +The documentation compiler lives in this repository, and is packaged as a normal SwiftPM executable target. + +To invoke the compiler, run the `unidoc-build` tool and pass it the name of the package you want to build documentation for. In our case, the “package” is `swift`, which is a special name identifying the standard library itself. + +@Code(file: load-standard-library.sh, title: load-standard-library.sh) + +If you did everything correctly, you should see output that ends with something like this: + +``` +... + +Linked documentation! + time loading sources : 0.0 seconds + time linking : 1.172033425 seconds +symbols : 16501 +Uploading symbol graph... +Successfully uploaded symbol graph! +``` + +Because you built these docs “abnormally” (meaning: not from a GitHub repository), they won’t show up in the homepage, but you can view them by navigating directly to [`localhost:8080/docs/swift`](http://localhost:8080/docs/swift). + +> Note: + You may see a lot of compiler errors when building the standard library. This is expected, as the documentation for the standard library contains many errors. + + +### Building documentation for a local project + +Building documentation for a local project is similar to building documentation for the standard library, except you need to provide a path to a directory containing the project. + +Let’s try building documentation for [`swift-nio`](https://github.com/apple/swift-nio). First, we need to clone the repository. + +```bash +cd /swift +git clone https://github.com/apple/swift-nio +``` + +**Where** you clone the repository is important, because you will need to tell Unidoc where to find the project. In this example, we cloned the repository inside a directory called `/swift`, which is a plausible place to store Git repositories in a devcontainer. + +Next, you can try building `swift-nio` with `unidoc-build`, specifying the path to the search directory (`/swift`) with the `-I` option. + +@Code(file: load-swift-nio.sh, title: load-swift-nio.sh) + +Unidoc will launch a `swift build` process, which could take a few minutes to build the package. When the build completes, it will then compile, upload, and link the documentation. Because the documentation is local, it will have the version number `__max`, and it will not show up on the homepage. You can view it by navigating directly to [`localhost:8080/docs/swift-nio`](http://localhost:8080/docs/swift-nio). + +Congratulations! You have successfully set up a local Unidoc server and previewed some documentation for a local project. + +> Warning: +> At the time of writing, there is a [Swift compiler bug](https://github.com/swiftlang/swift/issues/68767) that prevents `swift symbolgraph-extract` from emitting symbol data for Swift NIO on macOS. + + +## Differences between DocC and Unidoc + +There are a few key differences between DocC and Unidoc workflows to keep in mind. + +### Shared database + +Unlike DocC, Unidoc is specifically designed for multi-project use cases. + +Although you *can* have a separate database for each project, it is usually easier to set up a single deployment per machine, with documentation that is added and updated as needed. + +### Not a package plugin + +DocC is a package plugin, which means you “install” it by adding it as a dependency to your `Package.swift` and invoke it through SPM. Unidoc is a toolchain and you invoke it directly on an SPM project, like `swift build`. Unlike DocC, Unidoc has no project footprint. + +### Not Swiftinit + +A local Unidoc server is not [Swiftinit](https://swiftinit.org) and it does not have access to Swiftinit’s package database. If you want to have a local Swiftinit-like experience, you need to populate your local Unidoc server with your packages and the packages they depend on. + diff --git a/Guides/docs.docc/Quickstart.md b/Guides/docs.docc/Quickstart.md index 0df6e3459..2413b5d0a 100644 --- a/Guides/docs.docc/Quickstart.md +++ b/Guides/docs.docc/Quickstart.md @@ -1,162 +1,110 @@ -# Unidoc quickstart +# Previewing documentation locally -This guide outlines how to set up and run a local Unidoc server you can use to preview your documentation as it would appear on [Swiftinit](https://swiftinit.org). +This guide walks through how to use the `unidoc-preview` tool to preview documentation locally on macOS 14. This guide won’t make any effort to explain how Unidoc itself works, it is merely intended to demonstrate how to preview multi-package documentation as quickly as possible. For a more-detailed Linux-centric tour, see the guide. -## Prerequisites +In this guide, you will: -You need to have a Swift Package Manager project. Unidoc itself has little relation to SPM, but it uses the Swift toolchain to compile your project, and the Swift toolchain uses SPM to build your project. +1. Launch and initialize a `mongod` instance in a Docker container, +2. Build and run an instance of `unidoc-preview` on the macOS host, +3. Build the `unidoc-build` tool, +4. Generate documentation for the standard library, and +5. Generate documentation for two SwiftPM packages, one of which depends on the other. -You need to have [Docker](https://www.docker.com/) installed on your development machine. If you install it through your default package repository, we strongly recommend you ensure the installed Docker version is reasonably up-to-date (e.g. 24.0.0 or later). - -It is theoretically possible to run Unidoc without Docker, but Docker makes it much easier to do so, because you will not need to (directly) manage a local [MongoDB](https://mongodb.com) deployment. - - -## Setting up a local database - -A Unidoc server is a server that talks to a second server, a MongoDB server. Therefore, before you can start a Unidoc server, you need to set up and launch a local [MongoDB](https://github.com/tayloraswift/swift-mongodb) deployment. - -A MongoDB deployment consists of a [mongod](https://www.mongodb.com/docs/manual/reference/program/mongod/) process connected to a network that the Unidoc server is also connected to. - - -### Starting the database server - -We recommend hosting mongod inside a dedicated Docker container. This repository provides a Docker compose file that does this for you. - -To bring up the mongod instance (if it is not already online), run the following: +Before you begin, clone the Unidoc repository and navigate to the root of the repository: ```bash -docker compose -f Guides/docs.docc/local/docker-compose.yml up -d +git clone https://github.com/tayloraswift/swift-unidoc +cd swift-unidoc ``` -The `-d` flag tells Docker to run the container in the background, so it does not block your terminal. - - -### About the Docker compose file +## 1. Install Docker -If you are new to Docker, it is worth taking a moment to understand the Docker compose file. +The easiest way by far to preview documentation locally is to use Docker. You can download Docker Desktop for macOS from the [official website](https://www.docker.com/products/docker-desktop). -@Code(file: docker-compose.yml, title: docker-compose.yml) -This file: +## 2. Launching a `mongod` instance -1. Launches a container from the official [`mongo:latest`](https://hub.docker.com/_/mongo) image. -2. Sets the name of this container to `unidoc-mongod-container`. -3. Sets the **hostname** of the container to `unidoc-mongod` **within** the `unidoc-test` network. -4. Binds the container’s port 27017 to `localhost:27017`. This is the default port for the `mongod` process. -5. Mounts the startup scripts and data directory within the container. This allows the documentation data to persist across container restarts. -6. Passes a configuration file to the `mongod` process on startup. +Use Docker Compose to launch a `mongod` instance in a container. This container is named `unidoc-mongod-container`. It has persistent state which `mongod` stores in a directory called `.mongod` at the root of the repository. -The `unidoc-test` network is helpful for testing, but for the purposes of this tutorial, you will mostly be accessing the `mongod` process through `localhost:27017`. +```bash +docker compose -f Guides/docs.docc/local/docker-compose.yml up +``` +@Image(source: "Docker Desktop.png", alt: "Docker Desktop") { +> You should see the `unidoc-mongod-container` running in the Docker Desktop GUI. +} -## Initializing the database -The mongod instance will create a `.mongod` directory at the root of the cloned repository. This directory contains the state of the deployment, and like all database deployments, it can outlive the mongod process. This means you can kill (or crash) the mongod instance but it will not lose data unless you clear or delete its data directory. +The container is home to a MongoDB [replica set](https://www.mongodb.com/docs/manual/reference/replica-configuration/) which you need to initialize. -Initialize the replica set with: +Open a new terminal and run the following command to initialize the replica set: ```bash docker exec -t unidoc-mongod-container /bin/mongosh --file /unidoc-rs-init.js ``` -This only needs to be done **once** per deployment lifecycle. (For example, after clearing the `.mongod` data directory.) - - -### Connecting to the database - -Once you have a `unidoc-mongod-container` running in the background, you can start a documentation server. There are many ways to run a documentation server, but if you are developing in a Docker container, the easiest way is compile Unidoc and run the server as a normal process. - -@Code(file: start-server.sh, title: start-server.sh) - - -### Generating certificates - -#### Unidoc < 0.17.0 - -If you are starting the server for the first time, you likely need to populate the `Assets/certificates/` directory with TLS certificates. See for instructions on how to do this. - -#### Unidoc ≥ 0.17.0 -You do not need to generate certificates, as Unidoc 0.17.0 can run locally in insecure mode. +## 3. Running `unidoc-preview` +The `unidoc-preview` tool is an ordinary SwiftPM executable product. You can build and run it directly from your macOS host like this: -## Populating a local documentation server - -If you did all of the previous steps correctly, you should be able to navigate to [`localhost:8080/`](http://localhost:8080/) and view a blank homepage. - -> Note: -> If you are using HTTPS, you need to replace the scheme with `https://` and the port number with `8443`. - +```bash +swift run -c release unidoc-preview +``` -A fresh Unidoc database contains no documentation. Let’s build some now. +The `unidoc-preview` tool will start a web server on `http://localhost:8080`. -### Building documentation for the standard library +@Image(source: "Start page.png", alt: "Start page") { +> The `unidoc-preview` start page. +} -The documentation compiler lives in this repository, and is packaged as a normal SwiftPM executable target. +## 4. Generating documentation for the standard library -To invoke the compiler, run the `unidoc-build` tool and pass it the name of the package you want to build documentation for. In our case, the “package” is `swift`, which is a special name identifying the standard library itself. +Generate local documentation using the `unidoc-build local` subcommand. To start off, open a third terminal and generate the documentation for the standard library (`swift`). @Code(file: load-standard-library.sh, title: load-standard-library.sh) -If you did everything correctly, you should see output that ends with something like this: +You should be able to view the symbol graph and its documentation at `http://localhost:8080/tags/swift`. -``` -... - -Linked documentation! - time loading sources : 0.0 seconds - time linking : 1.172033425 seconds -symbols : 16501 -Uploading symbol graph... -Successfully uploaded symbol graph! -``` - -Because you built these docs “abnormally” (meaning: not from a GitHub repository), they won’t show up in the homepage, but you can view them by navigating directly to [`localhost:8080/docs/swift`](http://localhost:8080/docs/swift). - -> Note: - You may see a lot of compiler errors when building the standard library. This is expected, as the documentation for the standard library contains many errors. +@Image(source: "Standard library tags.png", alt: "Standard library") { +> The standard library documentation. We generated it using the default Xcode toolchain, so it’s labeled `__Xcode`. +} -### Building documentation for a local project +## 5. Generating documentation for SwiftPM packages -Building documentation for a local project is similar to building documentation for the standard library, except you need to provide a path to a directory containing the project. - -Let’s try building documentation for [`swift-nio`](https://github.com/apple/swift-nio). First, we need to clone the repository. +Now, let’s generate documentation for [swift-collections](https://github.com/apple/swift-collections), a popular SwiftPM package. Download the library’s source code to a sibling directory. ```bash -cd /swift -git clone https://github.com/apple/swift-nio +cd .. +git clone https://github.com/apple/swift-collections +cd - ``` -**Where** you clone the repository is important, because you will need to tell Unidoc where to find the project. In this example, we cloned the repository inside a directory called `/swift`, which is a plausible place to store Git repositories in a devcontainer. - -Next, you can try building `swift-nio` with `unidoc-build`, specifying the path to the search directory (`/swift`) with the `-I` option. +Generating documentation for a package is similar to generating documentation for the standard library, except you need to specify a search path to a directory containing the project. Because you downloaded the `swift-collections` repository to a sibling directory, you can use `..` for the search path. -@Code(file: load-swift-nio.sh, title: load-swift-nio.sh) - -Unidoc will launch a `swift build` process, which could take a few minutes to build the package. When the build completes, it will then compile, upload, and link the documentation. Because the documentation is local, it will have the version number `__max`, and it will not show up on the homepage. You can view it by navigating directly to [`localhost:8080/docs/swift-nio`](http://localhost:8080/docs/swift-nio). - -Congratulations! You have successfully set up a local Unidoc server and previewed some documentation for a local project. - -> Warning: -> At the time of writing, there is a [Swift compiler bug](https://github.com/swiftlang/swift/issues/68767) that prevents `swift symbolgraph-extract` from emitting symbol data for Swift NIO on macOS. - - -## Differences between DocC and Unidoc - -There are a few key differences between DocC and Unidoc workflows to keep in mind. - -### Shared database +```bash +swift run -c release unidoc-build local swift-collections -I .. +``` -Unlike DocC, Unidoc is specifically designed for multi-project use cases. +You should be able to view the symbol graph and its documentation at `http://localhost:8080/tags/swift-collections`. -Although you *can* have a separate database for each project, it is usually easier to set up a single deployment per machine, with documentation that is added and updated as needed. +@Image(source: "Swift Collections tags.png", alt: "Swift Collections") { +> The `swift-collections` documentation. +} -### Not a package plugin +Finally, let’s generate documentation for a package that depends on `swift-collections`. Download the source code for [swift-async-algorithms](https://github.com/apple/swift-async-algorithms) to another sibling directory. -DocC is a package plugin, which means you “install” it by adding it as a dependency to your `Package.swift` and invoke it through SPM. Unidoc is a toolchain and you invoke it directly on an SPM project, like `swift build`. Unlike DocC, Unidoc has no project footprint. +```bash +cd .. +git clone https://github.com/apple/swift-async-algorithms +cd - +swift run -c release unidoc-build local swift-async-algorithms -I .. +``` -### Not Swiftinit -A local Unidoc server is not [Swiftinit](https://swiftinit.org) and it does not have access to Swiftinit’s package database. If you want to have a local Swiftinit-like experience, you need to populate your local Unidoc server with your packages and the packages they depend on. +@Image(source: "Swift Async Algorithms tags.png", alt: "Swift Async Algorithms") { +> The `swift-async-algorithms` documentation. Observe that it has a linked dependency on the `swift-collections` documentation you generated earlier. +} +Please note that when you link documentation for a SwiftPM package against another package, it is your responsibility to ensure that the two versions are ABI-compatible. Many SwiftPM packages are not ABI-stable, so you should always check that the root package is being built with the same versions of its dependencies as you generated their documentation from. diff --git a/Guides/docs.docc/README.md b/Guides/docs.docc/README.md index 470ad1913..b32d4528c 100644 --- a/Guides/docs.docc/README.md +++ b/Guides/docs.docc/README.md @@ -5,6 +5,7 @@ ### Using Unidoc - +- - ### Developing Unidoc diff --git a/Guides/docs.docc/screenshots/Docker Desktop.png b/Guides/docs.docc/screenshots/Docker Desktop.png new file mode 100644 index 000000000..75bc1777b Binary files /dev/null and b/Guides/docs.docc/screenshots/Docker Desktop.png differ diff --git a/Guides/docs.docc/screenshots/Standard library tags.png b/Guides/docs.docc/screenshots/Standard library tags.png new file mode 100644 index 000000000..ad3c3d383 Binary files /dev/null and b/Guides/docs.docc/screenshots/Standard library tags.png differ diff --git a/Guides/docs.docc/screenshots/Start page.png b/Guides/docs.docc/screenshots/Start page.png new file mode 100644 index 000000000..2370b1660 Binary files /dev/null and b/Guides/docs.docc/screenshots/Start page.png differ diff --git a/Guides/docs.docc/screenshots/Swift Async Algorithms tags.png b/Guides/docs.docc/screenshots/Swift Async Algorithms tags.png new file mode 100644 index 000000000..5a130f775 Binary files /dev/null and b/Guides/docs.docc/screenshots/Swift Async Algorithms tags.png differ diff --git a/Guides/docs.docc/screenshots/Swift Collections tags.png b/Guides/docs.docc/screenshots/Swift Collections tags.png new file mode 100644 index 000000000..8c11b91b6 Binary files /dev/null and b/Guides/docs.docc/screenshots/Swift Collections tags.png differ diff --git a/Package.resolved b/Package.resolved index 145f1db18..018836b2a 100644 --- a/Package.resolved +++ b/Package.resolved @@ -7,7 +7,7 @@ "location" : "https://github.com/tayloraswift/indexstore-db", "state" : { "branch" : "occurrences-in-file", - "revision" : "2ef13f914518765f1626eeb5ef2c12271502e331" + "revision" : "da980455bf2bbbffeffe1946dfc3665eb03c3bcd" } }, { @@ -139,10 +139,10 @@ { "identity" : "swift-system", "kind" : "remoteSourceControl", - "location" : "https://github.com/apple/swift-system.git", + "location" : "https://github.com/apple/swift-system", "state" : { - "revision" : "f9266c85189c2751589a50ea5aec72799797e471", - "version" : "1.3.0" + "revision" : "6a9e38e7bd22a3b8ba80bddf395623cf68f57807", + "version" : "1.3.1" } } ], diff --git a/Sources/unidoc-build/Unidoc.Client.swift b/Sources/unidoc-build/Unidoc.Client.swift index 63c718643..667868619 100644 --- a/Sources/unidoc-build/Unidoc.Client.swift +++ b/Sources/unidoc-build/Unidoc.Client.swift @@ -336,6 +336,11 @@ extension Unidoc.Client type: type) try await self.connect { try await $0.upload(object) } + + print(""" + View the generated documentation at: + https://\(self.http.remote):\(self.port)/tags/\(symbol) + """) } } extension Unidoc.Client @@ -349,6 +354,11 @@ extension Unidoc.Client type: type) try await self.connect { try await $0.upload(object) } + + print(""" + View the generated documentation at: + http://\(self.http.remote):\(self.port)/tags/\(symbol) + """) } } extension Unidoc.Client