[EPIC] Experiment with Syft Support

[doughertym]

https://github.com/anchore/syft provides support for creating SBOM files for a wide variety of languages and frameworks, which might be worth exploring if and how it could help the https://github.com/corgibytes/freshli-cli process more repositories.

Tasks

Preview Give feedback

1. Create freshli-agent-syft repository`
2. Create skeleton project repository
3. Create gRPC service with HealthCheck endpoint
4. Create skeleton com.corgibytes.freshli.agent.Agent gRPC service
5. Implement DetectManifests endpoint for freshli-agent-syft freshli-cli#704
   +0
6. Implement ProcessManifest endpoint for freshli-agent-syft freshli-cli#705
   +0
7. Implement unit and integration test cases for freshli-agent-syft freshli-cli#706
   +0

[doughertym]

Created https://github.com/corgibytes/freshli-agent-syft

[doughertym]

Notes from my 05/Oct/2023 journal entry:

I was able to get the skeleton of the agent up and running. But there are a few things that need to be discussed and some things that need to be finished. I got to the point where the gRPC server can be started and stopped. The two main endpoints for DetectManifests and ProcessManifest exist, but do not do anything yet. The simplest way would seem to be invoking the cli Command on the path. But that did not seem to work as expected. Maybe I was missing something though. Might need a fresh brain before going after that again. In any event, it should work out somehow. Though, unfortunately the implementors of Syft did not abstract the actual processing of the data from the Command Interface (cli interface). If they had, then it would make things easier.

The other thing is that Syft does both tasks, DetectManifests and ProcessManifest, in one as the result of running syft is to have a bom with all the information. For example:

{
  "components": [
    {
      "bom-ref": "pkg:nuget/[email protected]?package-id=50796c5039f30ee9",
      "type": "library",
      "name": "AutoMapper",
      "version": "6.1.1",
      "cpe": "cpe:2.3:a:AutoMapper:AutoMapper:6.1.1:*:*:*:*:*:*:*",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "syft:package:foundBy",
          "value": "dotnet-deps-cataloger"
        },
        {
          "name": "syft:package:language",
          "value": "dotnet"
        },
        {
          "name": "syft:package:metadataType",
          "value": "DotnetDepsMetadata"
        },
        {
          "name": "syft:package:type",
          "value": "dotnet"
        },
        {
          "name": "syft:location:0:path",
          "value": "/src/nhsweb/nhs.deps.json"
        }
      ]
    }
  ]
}

The properties.[@name = 'syft:location:0:path'].value (appended to ${folderName}) will be the Manifest location. I have not, as of yet, seen a way to just get the manifest files. Though, with some simple logic, it can be determined, so it's not impossible.

Lastly, what happens after the manifests have been processed into BOMs? If I understand the freshli-cli it will use the same agent to get the RetrieveReleaseHistory as it did to find and process the manifest. Though, I think it could be updated to do it a little differently. The code could use that value of properties.[@name = 'syft:package:language'].value to determine which language agent to use for getting the package history. This seems like the most logical approach, though the downside is that the freshli-cli would need to know how to interpret the Syft agent's BOM file specifically, which is diverging away from the intent that it be agnostic about this stuff. All that said, I am sure there is a way to abstract out the Syft specific logic enough to minimize the impact.