🚗 Rugby Roadmap

[swiftyfinch]

Hello everyone!

I'm wondering if any of you have any thoughts on the future of 🏈 Rugby.

Feel free to share your thoughts and ideas on this topic.
Whether it's about new features, potential pitfalls, or anything else you can think of.
I'd really appreciate ♥︎ hearing your thoughts.

🚢 Upcoming:

• Attach errors/warnings from the rugby build to pull request changes (xcbeautify --renderer github-actions)
  ⚙️ Creating a symlink to the latest logs folder #368
• Add an example of uploading binaries to S3 to 🐳 Remote Cache Doc
  🐳 Improve remote cache documentation #374
• 🚀 Implement ability to add headers in warmup command #370
• Add ability to use --dry-run for commands with --targets-as-regex options
  🚀 Added --try mode to commands #375
• 🚀 Added resolving env variables in plans #385

[kattouf]

Hello! First of all, thank you for your work, we use rugby for several months and like it!
Also after using it all this time we have some proposals for you, this is the first one:

Logs Formatting

1. Performance Issue

Problem

I notice that the rugby build command ~30% percent of time takes ~100% CPU and in the remaining time about ~15-25% CPU. After a little research, I figured out that about ~60-70% of the time it is log parsing, which costs a lot for our big multi-modular project (about 5 minutes only for parsing)

After code research we found that log beautifying is used only for error parsing (in previous versions), then we make changes for parse log only if xcodebuild produces non-zero error code and it saves a lot of our time in successful builds:

commit 700369c24bfd1e11e36146cee0236cbd9ed30872
Author: Kattouf <[email protected]>
Date:   Wed Mar 13 13:27:13 2024 +0700

    lazy log parsing for error case

diff --git a/Sources/RugbyFoundation/Core/Build/XcodeBuild/XcodeBuild.swift b/Sources/RugbyFoundation/Core/Build/XcodeBuild/XcodeBuild.swift
index 2b6f293..76ed42a 100644
--- a/Sources/RugbyFoundation/Core/Build/XcodeBuild/XcodeBuild.swift
+++ b/Sources/RugbyFoundation/Core/Build/XcodeBuild/XcodeBuild.swift
@@ -93,7 +93,7 @@ final class XcodeBuild {
         options: XcodeBuildOptions,
         paths: XcodeBuildPaths
     ) async throws {
-        let command = "NSUnbufferedIO=YES xcodebuild"
+        let command = "set -o pipefail && NSUnbufferedIO=YES xcodebuild"
         var arguments = arguments
         arguments.append(contentsOf: [
             "-project \(paths.project.shellFriendly)",
diff --git a/Sources/RugbyFoundation/Core/Build/XcodeBuild/XcodeBuildExecutor.swift b/Sources/RugbyFoundation/Core/Build/XcodeBuild/XcodeBuildExecutor.swift
index e8654bd..0e3ae3b 100644
--- a/Sources/RugbyFoundation/Core/Build/XcodeBuild/XcodeBuildExecutor.swift
+++ b/Sources/RugbyFoundation/Core/Build/XcodeBuild/XcodeBuildExecutor.swift
@@ -31,9 +31,14 @@ final class XcodeBuildExecutor: IXcodeBuildExecutor, Loggable {
 
     func run(_ command: String, rawLogPath: String, logPath: String, args: Any...) async throws {
         try Folder.create(at: URL(fileURLWithPath: rawLogPath).deletingLastPathComponent().path)
-        try shellExecutor.throwingShell(command, args: args, "| tee '\(rawLogPath)'")
-        if let errors = try? await beautifyLog(rawLogPath: rawLogPath, logPath: logPath), errors.isNotEmpty {
-            throw BuildError.buildFailed(errors: errors, buildLogPath: logPath, rawBuildLogPath: rawLogPath)
+        do {
+            try shellExecutor.throwingShell(command, args: args, "| tee '\(rawLogPath)'")
+        } catch let error {
+            if let errors = try? await beautifyLog(rawLogPath: rawLogPath, logPath: logPath), errors.isNotEmpty {
+                throw BuildError.buildFailed(errors: errors, buildLogPath: logPath, rawBuildLogPath: rawLogPath)
+            } else {
+                throw error
+            }
         }
     }

Rugby cache time difference with batch above:

Original	Patched

I wanted to open PR, but now I see the log beautifier used for more purposes, and we can't legally parse logs only for build errors.

Some solutions that I see

1. The easiest way is stop to beautifying logs for successful builds and use beautifier only for error parsing
2. Parse logs parallel to xcodebuild work: like connect to log stream and parse logs as they appear
3. Try to process log files in multithreaded way (as I said earlier, log parsing takes only 15-25 percent CPU time)

2. Log Format

Problem

We want to use rugby on CI and locally, but on CI difficult to locate rugby build errors, we should manually scroll to it in logs. To fix that kind of problem some CI (TeamCity in our case) supports format that we can use for mark errors and warnings to easily find them in TeamCity Web App.

Proposal

For our case will be great to have the opportunity to select between xcbeatify renderer options (originally xcbeautify --renderer github-actions). But when we have those renderer options, also necessary to not break those logs by post-formatting inside rugby app.

[swiftyfinch]

I'm glad to hear that! 🚀

[kattouf]

@swiftyfinch can you please share your vision for API to this upcoming feature?

Attach errors/warnings from the rugby build to pull request changes

I just wanted to make sure I explained my feature request correctly, so for a simpler explanation I'll use some of our local changes as an example

We would like to have the option to pass the "logs renderer" option that will affect to xcbeautify Parser:

and then we will get results like this:

You can see applied TeamCity-specific prefixes in error message lines

[swiftyfinch]

@kattouf Hello! I've made a pull request. Could you please take a look?

• ⚙️ Creating a symlink to the latest logs folder #368

The main idea behind this is to separate the implementation of this logic from 🏈 Rugby.
While I can implement it, it seems like it would be better as a standalone feature.
You can choose any tool and parse the log with it.

Additionally, the new author of xcbeautify broke some API in the 2.x version (#366), and I'm considering removing the beautification of logs altogether.
However, I understand that we still need a parser for convenient error messages.

Thank you for your attention to this matter.
I look forward to your feedback.

[swiftyfinch]

UPDATE: I have not seen your updated question.

The problem is, we should make these changes in the 2.x version of xcbeautify.
And then we can use them in rugby.
However, I'm not sure if this will be the right approach.

[kattouf]

Thanks for the explanation, I actually thought about this feature when parsing the logs was taking a long time and I didn't consider options to parse it a second time after the build.
Now thanks for your comments and PR, I understand that we can try do something similar with our CI pipeline and not tie rugby to any tool

[kattouf]

Ready to use remote cache solutions

Problem

1. Rugby generic solution for remote caching can't support storages authentication. This can be a problem in cases where you want to share the remote cache with a limited list of developers.
2. There is a short list of popular cloud storage services that Rugby users will write the same integration code for upload/download etc

Proposal

The proposal is to implement high-level service specific remote cache commands, like:

- command: warmup_s3
  address: s3.eu-west-2.amazonaws.com
  some_cred_key: hashKey
  arch: arm64

- command: upload_s3
  address: s3.eu-west-2.amazonaws.com
  some_cred_key: hashKey
  arch: arm64

And I think the first thing to do is to implement a general upload command with the steps described in doc

[swiftyfinch]

@kattouf Once again, thank you very much for the detailed description!
I really appreciate it.

1. Yes, there is already an open issue regarding authentication headers:
   #329

- command: warmup
  argument: s3.eu-west-2.amazonaws.com
  arch: arm64
  headers: 'some_cred_key: hashKey'

2. I am not sure about general or specific solutions for uploading binaries to the cloud within the context of 🏈 Rugby.
   To me, uploading seems like a different area that should be handled by suitable scripts written by CI engineers.
   But I think, I can add a small example of uploading binaries to S3 to 🐳 Remote Cache Doc.

[kattouf]

Thank you, warmup headers are also very good 👍

[dcacenabes]

I have an idea to follow up on the authentication for remote caches.

The current implementation based on a header parameter is great, and it makes our adoption in CI very straightforward. When it comes to local developer environments it becomes a little bit more tricky, since developers have to include the headers every time they run the rugby command.
Another option is to rely on the plan yaml and include the headers there, but we don't want to store any credential in our git project.

I wonder which are your thoughts on an alternative solution that does not require the headers to be part of the plan. Just thinking out loud: maybe we could have a hidden file in the root of the project, called .rugby-authentication, which contains a group of key-value pairs with the header name and the content. That file could be added to .gitignore and each developer has there their credentials. The rugby command grabs credentials from that file (when it exists).

Curious to hear your thoughts here.

[swiftyfinch]

@dcacenabes @kattouf I've opened a pull request. Could you please take a look at it?

• 🚀 Added resolving env variables in plans #385

[dcacenabes]

Checking it out, building locally and reporting back 👍

[dcacenabes]

Works like a charm 🚀

[swiftyfinch]

Thanks! I will merge this soon.

[swiftyfinch]

🚢 v2.8.0

[dcacenabes]

I have another idea/suggestion/feature request.

In my current company, we leverage Cocoapods for modularization. All our modules are development pods. At some point, we enabled our current caching solution for developer's local machines (before it was only enabled in CI). It introduced lots of friction because most developers preferred to having all the local modules available to quickly edit and debug (at the expense of faster build times). We had to introduce some changes: In CI we cache everything, locally we only cache external dependencies.

Here comes my suggestion:
In the same manner that we can exclude certain Pods by name and by regex, have the option to exclude local Pods with a single parameter. This way we can build a plan for Rugby that developers can use locally so that we don't have to maintain a list with the names of all development Pods, which would be error prone.

[swiftyfinch]

There is a problem with identifying remote pods in an Xcode project.
CocoaPods does not provide any information about the type (local or remote) of pods in Xcode targets.
We can depend on the "Development Pods" group, but it seems to be not very safe.

This feature was available in the first version of 🏈 Rugby.
However, I have decided not to rely on information from Podfile.lock in Rugby 2.x.

I'm thinking about making it so that Rugby is not limited by the CocoaPods environment in the future.
But that's just a dream of mine. 😅

Also, I don't think that only caching remote pods will be enough to achieve significant results.
What do you think about that?

[dcacenabes]

I get your point. I am used to our current solution, which works integrated as a Cocoapods plugin and since it runs as part of the pod installation process, it has information about which pods are development ones and which ones not. But Rugby works after pods have already been installed, and then it is not feasible in an easy way to detect which are local and which are remote pods 🙏

I think it makes sense to keep it as is. For our use case, probably with a combination of training+environment variables we can get developers to use it for local pods.

[swiftyfinch]

Thank you all for your ideas! 🚀

[kattouf]

I came across plugin and thought it would be possible to implement a plugin with a similar API based on Rugby. As an option for simple and seamless caching integration for cases where you don't need all the flexibility that Rugby offers.
p.s., this could be done in a separate repository “on the side”
p.s.s I'm not calling for action, just something to think about (:

[swiftyfinch]

@kattouf Thank you for your suggestion! I have considered this, but I have found that the CocoaPods plugin repository is somewhat outdated. Additionally, such a plugin adds some limitations to the Podfile. As an alternative, I have created Rugby. Anyone can use it on their own without any dependencies throughout the project, like an optional step.

I have created a separate utility for working with dependency graphs: https://github.com/swiftyfinch/xtree.
And if I have time, I would like to create a CocoaPods project generator with Swift as well.
Maybe it will be faster, or not 😅.
The main idea is to create a set of tools that are independent of each other and do not have strict dependencies between them.

[kattouf]

Anyone can use it on their own without any dependencies throughout the project, like an optional step
The main idea is to create a set of tools that are independent of each other and do not have strict dependencies between them.

Yeah, I see profit from that decision and I agree that this good way to implement that kind of tool. I just was dreamily thinking about a better developer experience to not care about any manual pre/post pod install command calls.

And if I have time, I would like to create a CocoaPods project generator with Swift as well.

Huge plans! But with expected speed profit (Maybe it will be faster, or not ) it sounds a little risky :)

[swiftyfinch]

I think a plugin might not improve the development experience or maybe I just don't have the right idea at the moment.
But I agree that it would be great to have a more convenient way to work with binaries and source code.

Thanks again for your thoughts. This gives me something to think about 👍