C# validation extremely slow

[omusavi]

Describe the bug
We have a medium sized C# solution running with super linter. We are running super linter with RUN_LOCAL so every run ends up evaluating every file. We recently wanted to add C# linting since it was just added, but found that build times for linting all of a sudden spiked. The reason for this is the call to dotnet format takes roughly 2-3 seconds per file, whereas the tool seems to be meant to be run on csproj or sln files.

I compared a run of the the same linter running through super linter, and just the C# portion took roughly 7 minutes. I then used the troubleshooting instructions to get into the container and ran dotnet format --check against our solution file and it took 5 seconds to complete. An 8000+% increase in lint time is not acceptable for us in our build processes; when run times are in the order of minutes, the usefulness of the tool is lost and its faster to spin up a dotnet container, install dotnet format and run it.

To Reproduce
As an example, lets run super-linter on the dotnet-format project :)

1. Check out any C# project (as an example: https://github.com/dotnet/format)
2. Run docker run -e FILTER_REGEX_EXCLUDE=".*/(obj|bin)/.*" -e RUN_LOCAL=true -e VALIDATE_CSHARP=true -v `pwd`:/tmp/lint github/super-linter
3. Record the time it took to run. (dotnet-format took just under 4 minutes)
4. Run docker run -it -v `pwd`:/tmp/lint --entrypoint /bin/bash github/super-linter
5. Inside container, run dotnet format --check /tmp/lint/solutionfile.sln (replacing solutionfile.sln with the path to the solution)
6. Record time it took to run (Took 4700ms for me)

Expected behavior
Super-linter run times should not be significantly different than running the tools against the support file types.

[admiralAwkbar]

@omusavi Thanks for the input.

Would you think that if we looked for SLN files and then ran the linter against that, vs individual files, we could experience much better times?

I'm not a C# dev, and we want to make sure this works well for everyone. I'm interested in some solutions we can take to speed this up and help everyone

thanks

[omusavi]

Yes, looking as sln files or csproj files would be the preferred method for this tool. Looks like dotnet format also supports a --folder flag that looks at all files within that folder, but that might not work as well because it won't take into account the files that super-linter already ignores (such as node_modules).

Perhaps there could be a flag for specifying which extension you want to search for (sln, csproj, cs)? I think defaulting to csproj would be good because different sln files could overlap with the same csproj files (sln is just a collection of csproj)

[andrejpk]

It seems like super-linter is built on sensible defaults, so what makes the most sense in most cases? Default to just scanning SLNs and let you configure another way if you'd like? Or default to **.cs as it does now and allow choosing a csproj or sln?

[omusavi]

From my understanding of how super-linter works, when run as a github action it only runs on the files that have been modified, so if we were to set it to default to sln/csproj, it would not lint any changes to code files. However, when using RUN_LOCAL it ends up running against everything all the time, which is how we run it outside of the github context. So as not to break the experience on github actions, I think it makes sense to allow it as an option, or perhaps run it on **/*.csproj when RUN_LOCAL is set and **/*.cs when it is not?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

[omusavi]

Commenting as this is still an active issue

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

[CommonGuy]

Commenting as this is still an active issue

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

[StephenHodgson]

Woot! Ready to see this fixed soon :)

[StephenHodgson]

bump. Also is there a way to specify a solution instead of it going through each file one by one?

[StephenHodgson]

https://github.com/github/super-linter/blob/05f34c653f433dfac36856d446a0d624c46c2012/lib/functions/buildFileList.sh#L375

I think if we were to replace this line with sln || csproj it would likely speed things up pretty quickly

[StephenHodgson]

Any updates on this. This is causing my company to spend lots of money bc of the sheer amount of time it takes. It's getting to be ridiculous that it's not fixed yet.

[zvailanu98]

Describe the bug We have a medium sized C# solution running with super linter. We are running super linter with RUN_LOCAL so every run ends up evaluating every file. We recently wanted to add C# linting since it was just added, but found that build times for linting all of a sudden spiked. The reason for this is the call to dotnet format takes roughly 2-3 seconds per file, whereas the tool seems to be meant to be run on csproj or sln files.

I compared a run of the the same linter running through super linter, and just the C# portion took roughly 7 minutes. I then used the troubleshooting instructions to get into the container and ran dotnet format --check against our solution file and it took 5 seconds to complete. An 8000+% increase in lint time is not acceptable for us in our build processes; when run times are in the order of minutes, the usefulness of the tool is lost and its faster to spin up a dotnet container, install dotnet format and run it.

To Reproduce As an example, lets run super-linter on the dotnet-format project :)

1. Check out any C# project (as an example: https://github.com/dotnet/format)
2. Run docker run -e FILTER_REGEX_EXCLUDE=".*/(obj|bin)/.*" -e RUN_LOCAL=true -e VALIDATE_CSHARP=true -v `pwd`:/tmp/lint github/super-linter
3. Record the time it took to run. (dotnet-format took just under 4 minutes)
4. Run docker run -it -v `pwd`:/tmp/lint --entrypoint /bin/bash github/super-linter
5. Inside container, run dotnet format --check /tmp/lint/solutionfile.sln (replacing solutionfile.sln with the path to the solution)
6. Record time it took to run (Took 4700ms for me)

Expected behavior Super-linter run times should not be significantly different than running the tools against the support file types.

troubleshooting instructions

[ferrarimarco]

@StephenHodgson Is this happening on the latest super-linter version? Can you try the steps in #736 (comment) with the latest available super-linter version, and see if the results are the same?

We didn't change too much in this area, besides dependency updates, but let's start fresh anyway.

[StephenHodgson]

No idea, that project has wrapped up and I no longer have access to it.

[ferrarimarco]

@zkoppert This is somewhat similar to how we handled linting of Go modules #4984.

Do we want to take a similar approach here?

[zkoppert]

Yeah, that makes sense to me @ferrarimarco!

[ferrarimarco]

#5177 should mitigate this, but we should probably handle .sln files as well