Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update OCLint setup to use native tools instead of xcpretty #40

Merged
merged 22 commits into from
Nov 14, 2022

Conversation

gaelfoppolo
Copy link
Contributor

@gaelfoppolo gaelfoppolo commented Nov 14, 2022

Summary

Current OCLint implementation suffers from two limitations:
a) the need to run a second build (with xcodebuild)
b) the need to use an external tool (other than OCLint itself)

This PR aims to solve both of these issues:
a) performing only one build
b) removing xcpretty, which produces the JSON Compilation Database

Details

Performing only one build

Currently we need execute two builds, with some minor changes in their parameters.
One is used for Periphery and the other is used for OCLint.
If we try to use the result of the first build for OCLint, we'll got the following issue: oclint: error: one compiler command contains multiple jobs.
To fix this we supply a flag, COMPILER_INDEX_STORE_ENABLE with the value NO. It seems OCLint internal logic (with clang) does not work with index store.
But, if we add this flag, Periphery will then not work, since the tool relies on the index store to work.
So if we want both tools to work we need two builds, one with index store enabled and the other disabled.

This is not ideal, we should be able to perform only one build.

By diffing the two compile_commands.json file generated from each build, we see only two lines were added when index store is enabled.

...
"-index-store-path", "/path/to/Index/DataStore"
...
"-index-unit-output-path", "/MyProject.build/Debug-iphonesimulator/MyProject.build/Objects-normal/x86_64/MyObjectiveCFile.o"
...

Removing these two lines from compile_commands.json allows to OCLint to work.

Note
They might be some other lines diffing and that cause OCLint to fail, in more complex projects. This cleaning part might need to be updated, but the logic will already be done.

We can now use only one build, with index store enabled.
Right before launching OClint, we "clean" compile_commands.json and that good.

Removing xcpretty

Currently we use xcpretty to generate the compile_commands.json, used by OCLint.
This file is in fact a JSON Compilation Database, a format for specifying how to replay single compilations independently of the build system.
Many tools provide a way to generate this JSON Compilation Database, so why not Xcode?

Well, it is possible for clang, which is used by Xcode behind the scene, to generate fragments of JSON Compilation Database, using the -MJ. But using this flag is complicated and would require many flag additions.

Fortunately, there is a (poorly documented) flag in LLVM: -gen-cdb-fragment-path, which will do this work for us. We only need to supply an output path and we're done! Mostly done.
Since clang produce fragments of JSON Compilation Database (one for each file), we need to concatenate those fragments together to create a compilation database.

We can now remove the use of xcpretty since Xcode will do the work for us.
The first issue (index store enabled) is still here, so we still need to perform the cleaning of the file.

DoD

  • Implementation
  • Unit tests
  • Same OCLint issues report on a several project as before
  • Cleaning of the old implementation & tests
  • Developer documentation

Links

oclint/oclint#611
https://thisiskyle.me/posts/using-oclint-without-rebuilding-your-project.html
https://gist.github.com/drumnkyle/b4328e7447b63af88e6f9a9daecc918d
https://gist.github.com/T1T4N/f4d63a44476eb5c7046cc561cb8c7f77
https://clang.llvm.org/docs/JSONCompilationDatabase.html

@gaelfoppolo gaelfoppolo changed the base branch from main to develop November 14, 2022 08:40
Copy link
Contributor

@davidy4ng davidy4ng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@gaelfoppolo gaelfoppolo merged commit aa40c4d into develop Nov 14, 2022
@gaelfoppolo gaelfoppolo deleted the feature/oclint-without-xcpretty branch November 14, 2022 16:17
gaelfoppolo added a commit that referenced this pull request Nov 16, 2022
* develop:
  Remove build warnings (local and CI) (#42)
  Refact code coverage to use common Xcode bundle result (#41)
  Update OCLint setup to use native tools instead of xcpretty (#40)

# Conflicts:
#	objc-lang/src/main/java/fr/insideapp/sonarqube/objc/issues/oclint/OCLintReportParser.java
#	objc-lang/src/main/java/fr/insideapp/sonarqube/objc/issues/oclint/OCLintSensor.java
#	objc-lang/src/test/java/fr/insideapp/sonarqube/objc/issues/oclint/OCLintJSONDatabaseBuilderTest.java
#	objc-lang/src/test/java/fr/insideapp/sonarqube/objc/issues/oclint/OCLintReportParserTest.java
#	objc-lang/src/test/java/fr/insideapp/sonarqube/objc/issues/oclint/OCLintSensorTest.java
#	sonar-apple-plugin/src/main/java/fr/insideapp/sonarqube/apple/ApplePlugin.java
gaelfoppolo added a commit that referenced this pull request Nov 28, 2022
* Add sonar.apple.jsonCompilationDatabasePath parameter

* Fix AppleTestsSensorTest

* Add JSON Compilation Database merge logic

* Upgrade JProc version to 2.8.0

* Export some logic to a separate object OCLintJSONDatabaseBuilder

* Adding OCLintJSONDatabaseBuilderTest

* Create OCLintExtractor to run  oclint-json-compilation-database

* Adding parsing step to OCLintExtractor

* Update OCLintReportParser

* Merging all the piece into OCLintSensor

* Some fixes

* Update documentation

* Updating documentation

* Update OCLintReportParserTest

* Cleaning

* Add headers

* Improve OCLint regex for the include parameter

* Update README

* Fix error in unit tests

* Fix bug (from Sonar Cloud)

* Fix issues (from Sonar Cloud)

* Fix runtime failure, the file wasn't properly closed before it was read by another ressource
@gaelfoppolo gaelfoppolo added the breaking A breaking change. label Nov 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking A breaking change.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants