aquasecurity / go-dep-parser Goto Github PK

View Code? Open in Web Editor NEW

143.0 17.0 108.0 13.33 MB

Dependency Parser for Multiple Programming Languages

License: MIT License

Go 99.77% Dockerfile 0.02% JavaScript 0.21%

go-dep-parser's Introduction

go-dep-parser

Dependency Parser for Multiple Programming Languages

go-dep-parser's People

Contributors

Stargazers

Watchers

Forkers

masahiro331 tomoyamachi iwamatsu triplekill westonsteimel yashbhutwala 5l1v3r1 antoinedeschenes johannestegner isgasho dai-dao ronaudinho jaredpalmer ekmixon arajkumar sam-lane ebati ankk13 laurentdelosieresmano syllogy sherif84 sf9133 tauseefmalik wagde-orca divyang-soni hecg119 andreylevchenko santhosh1729 chrissansone emilioforrer secteria breml mowangdk mozillazg rahul2393 afdesk worldvirus freydal jerbob92 zetaab sourcegraph tamirkiviti13 kunchawla pombredanne kobus-v-schoor jkroepke dmitriylewen shiracohen33 acumenix astevenson-microsoft mycodeself kgeorgiou davidsalame1 alexkoltun mainek00n isabella232 tofay sjord sq34sy int2ecall deepfactor-io nejch vaismanlior maitre-matt wadeling guyb1 lior-org-1 juneburning seabfh catminusminus marcobhr janjwerner-confluent knqyf263 moralon1 sled erichs relaxnow emend-io cspwizard liorj-orca beyondevil glitch octogonapus lambdatest nikpivkin yashprakash07 whxscanner orange-777 mikepartelow mikep-fn rohankumardubey deanhnter noqcks yurishafet mrfyda lebauce gsoc2 himazawa codacy nxpkg

go-dep-parser's Issues

feat(pom): add dependency tree

Hi! It would be nice to expand the feature where the dependency tree is output. I think this is currently only possible for npm projects.
I have already made a pull request (#123) which would enable this feature when parsing pom.xml files.

There already exists such another pull request (#107) which it think does have the same goal. There was never an issue made, therefore i am opening one now.

For more technical information please go to my pull request.

fix bug: all of pom "indirect" fileld is false

github.com/aquasecurity/[email protected]/pkg/java/pom/parse.go:81
between line:82 -> line:99, the dependencies of result is correct.

github.com/aquasecurity/[email protected]/pkg/fanal/analyzer/language/analyze.go:48
parsedLibs, parsedDependencies, err := parser.Parse(r)

buf after parsed dependencies, the parsedLIbs add indirect field and all is false

see example like this

package.json parser fails on object notation for workspaces

Currently, the workspaces field in package.json files is only allowed to be an array of strings. However, it has also an object version.

https://classic.yarnpkg.com/blog/2018/02/15/nohoist/

Example

  "workspaces": {
    "packages": [
      "packages/*",
      "plugins/*"
    ]
  },

Ref to an active open-source project using this notation:
https://github.com/backstage/backstage/blob/master/package.json#L52

Impact e.g. on Trivy scans:

Unable to parse "package.json" to remove dev dependencies: unable to parse .: parse error: JSON decode error: json: cannot unmarshal object into Go struct field packageJSON.workspaces of type []string

feat(golang): support go.sum

Parse go.sum and list all dependencies and versions. The signature of the function should be like func Parse(r io.Reader) ([]types.Library, error).
Ref. https://github.com/aquasecurity/go-dep-parser/blob/main/pkg/bundler/parse.go

feat(jar): support offline mode

Description

Support an option for offline mode. It will not call API even though the result differs. It just looks into pom.properties and MANIFEST.MF and skips them if groupId or artifactId cannot be identified.

Infinite Recursion in `pom.xml` Looped Dependency

There is currently an infinite recursion bug when parsing poms that have looped dependencies. This may occur in a scenario as such:

root pom declares module A
module A declares another module B
B declares a dependency on module A

The parser will endlessly loop between B and A consuming memory until a stack overflow error occurs and the application crashes. Calling Parse on the pom.xml in this minimal reproducer project should demonstrate the behavior.

The parser likely needs a cycle check when parsing dependency to check if the dependency is already one of its ancestors.

Let me know if there's any additional context or reproduction help I can provide. Thanks!

Incorrect .NET deps parsing

I originally posted this on the trivy repo, but figured it should actually be posted here. Original post: aquasecurity/trivy#5208

Description

Trivy doesn't generate a correct .NET dependency tree in CycloneDX. Please see this gist for the reference .deps.json file im using https://gist.github.com/noqcks/49089249820126cbaabe59b70ba12ae4

See the desired and actual behaviour section

Desired Behavior

Dependencies are listed for this package

{
  "ref": "pkg:nuget/[email protected]",
  "dependsOn": [
    "pkg:nuget/[email protected]",
    "pkg:nuget/[email protected]",
    "pkg:nuget/[email protected]"
  ]
}

Actual Behavior

The dependencies are empty.

{
  "ref": "pkg:nuget/[email protected]",
  "dependsOn": []
},

Reproduction Steps

Copy the .deps.json file from here https://gist.github.com/noqcks/49089249820126cbaabe59b70ba12ae4

Run

trivy fs MyWebApp.deps.json --format cyclonedx



### Target

Filesystem

### Scanner

None

### Output Format

CycloneDX

### Mode

Standalone

### Debug Output

```bash
``
trivy fs MyWebApp.deps.json --format cyclonedx --debug
2023-09-18T09:13:42.744-0700	DEBUG	["cyclonedx" "spdx" "spdx-json" "github"] automatically enables '--list-all-pkgs'.
2023-09-18T09:13:42.745-0700	DEBUG	Severities: ["UNKNOWN" "LOW" "MEDIUM" "HIGH" "CRITICAL"]
2023-09-18T09:13:42.745-0700	DEBUG	Ignore statuses	{"statuses": null}
2023-09-18T09:13:42.746-0700	INFO	"--format cyclonedx" disables security scanning. Specify "--scanners vuln" explicitly if you want to include vulnerabilities in the CycloneDX report.
2023-09-18T09:13:42.759-0700	DEBUG	cache dir:  /Users/noqcks/Library/Caches/trivy
2023-09-18T09:13:42.762-0700	DEBUG	Walk the file tree rooted at 'MyWebApp.deps.json' in parallel
2023-09-18T09:13:42.783-0700	DEBUG	OS is not detected.
{
  "$schema": "http://cyclonedx.org/schema/bom-1.5.schema.json",
  "bomFormat": "CycloneDX",
  "specVersion": "1.5",
  "serialNumber": "urn:uuid:6e5fc8cb-f23a-4d7d-aae9-9d8b60335e40",
  "version": 1,
  "metadata": {
    "timestamp": "2023-09-18T16:13:42+00:00",
    "tools": [
      {
        "vendor": "aquasecurity",
        "name": "trivy",
        "version": "0.45.0"
      }
    ],
    "component": {
      "bom-ref": "658f88d9-f9eb-4fdd-be0b-a1c4772fd1fe",
      "type": "application",
      "name": "MyWebApp.deps.json",
      "properties": [
        {
          "name": "aquasecurity:trivy:SchemaVersion",
          "value": "2"
        }
      ]
    }
  },
  "components": [
    {
      "bom-ref": "073fa28b-e147-4c07-8bec-046dadbc456e",
      "type": "application",
      "name": "MyWebApp.deps.json",
      "properties": [
        {
          "name": "aquasecurity:trivy:Class",
          "value": "lang-pkgs"
        },
        {
          "name": "aquasecurity:trivy:Type",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.AspNetCore.Authentication.Abstractions",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.AspNetCore.Authentication.Core",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.AspNetCore.Connections.Abstractions",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.AspNetCore.Hosting.Abstractions",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.AspNetCore.Hosting.Server.Abstractions",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.AspNetCore.Http.Abstractions",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.AspNetCore.Http.Extensions",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.AspNetCore.Http.Features",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.AspNetCore.Http",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.AspNetCore.Server.IIS",
      "version": "2.2.6",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.AspNetCore.WebUtilities",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.Extensions.Configuration.Abstractions",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.Extensions.DependencyInjection.Abstractions",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.Extensions.FileProviders.Abstractions",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.Extensions.Hosting.Abstractions",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.Extensions.Logging.Abstractions",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.Extensions.ObjectPool",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.Extensions.Options",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.Extensions.Primitives",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.NETCore.Platforms",
      "version": "2.0.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "Microsoft.Net.Http.Headers",
      "version": "2.2.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "System.Buffers",
      "version": "4.5.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "System.ComponentModel.Annotations",
      "version": "4.5.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "System.IO.Pipelines",
      "version": "4.5.3",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "System.Memory",
      "version": "4.5.1",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "System.Runtime.CompilerServices.Unsafe",
      "version": "4.5.1",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "System.Security.Principal.Windows",
      "version": "4.5.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    },
    {
      "bom-ref": "pkg:nuget/[email protected]",
      "type": "library",
      "name": "System.Text.Encodings.Web",
      "version": "4.5.0",
      "purl": "pkg:nuget/[email protected]",
      "properties": [
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "dotnet-core"
        }
      ]
    }
  ],
  "dependencies": [
    {
      "ref": "073fa28b-e147-4c07-8bec-046dadbc456e",
      "dependsOn": [
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]",
        "pkg:nuget/[email protected]"
      ]
    },
    {
      "ref": "658f88d9-f9eb-4fdd-be0b-a1c4772fd1fe",
      "dependsOn": [
        "073fa28b-e147-4c07-8bec-046dadbc456e"
      ]
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    },
    {
      "ref": "pkg:nuget/[email protected]",
      "dependsOn": []
    }
  ],
  "vulnerabilities": []
}

Operating System

macOS

Version

Version: 0.45.0
Vulnerability DB:
  Version: 2
  UpdatedAt: 2023-09-18 12:17:08.645500979 +0000 UTC
  NextUpdate: 2023-09-18 18:17:08.645500079 +0000 UTC
  DownloadedAt: 2023-09-18 15:19:46.14853 +0000 UTC



### Checklist

- [X] Run `trivy image --reset`
- [X] Read [the troubleshooting](https://aquasecurity.github.io/trivy/latest/docs/references/troubleshooting/)

NPM v7 lockFileVersion = 2 is not supported in Trivy filesystem scan

I think the issue aquasecurity/trivy#1350 belongs here. package-lock.json version 2 uses a different format that version 1, so language dependent files are not detected.

Incorrect package name returned by pip parser

While scanning I have come across a requirements.txt from a templating engine which is being used to generate the final requirements.txt file.
For example this is a line from the above mentioned requirements.txt file
... %ifcookiecutter.command_line_interface|lower=='click'-%} ...
Since we scan all requirements.txt files while doing a file system scan we would be splitting the line by == to get package name and version (as per logic here). As we can see this result is not intended, I'm wondering if we can add some validation and skip these kind of lines which lead to incorrect/invalid packages (in not just pip parser but in other languages as well)
Below is an example of the packages being generated as part of output of trivy scan when we do a file system scan in the above mentioned scenario.
{ "Target": "opt/conda/pkgs/cookiecutter-2.6.0-py311h06a4308_0/info/test/tests/test-templates/include/{{cookiecutter.project_slug}}/requirements.txt", "Class": "lang-pkgs", "Type": "pip", "Packages": [ { "Name": "{%ifcookiecutter.command_line_interface|lower", "Identifier": { "PURL": "pkg:pypi/%7B%25ifcookiecutter.command-line-interface%7Clower@%27click%27-%25%7D" }, "Version": "'click'-%}", "Layer": {} }, { "Name": "{%ifcookiecutter.use_pytest", "Identifier": { "PURL": "pkg:pypi/%7B%25ifcookiecutter.use-pytest@%27y%27-%25%7D" }, "Version": "'y'-%}", "Layer": {} } ] }

Maven BOM in another BOM is not well parsed

BOM dependencies are well parsed when the BOM is set into the <dependencyManagement> of the analysed project.
If the analysed project is using a BOM A that itself contains another BOM B, dependencies from A are parsed but not dependencies from B.

Dependencies coming from BOM B are not found and following message is displayed :

[DEBUG]	org.camunda.bpm:camunda-engine: was not found in local/remote repositories

coming from :

go-dep-parser/pkg/java/pom/parse.go

Line 591 in 245251a

 return nil, xerrors.Errorf("%s:%s:%s was not found in local/remote repositories", groupID, artifactID, version) 

I wanted to start a fix based on a recursive method to manage BOM, but I'm not a Go developer (and unfortunately no time to learn).

This issue impacts the trivy scan #5748 as it analyses dependencies returned by this parser.

The issue can be reproduced with this example project : https://github.com/glelarge/trivy-maven-issue

Bundler uses different RegExp for parsing lockfile lines

I've been troubleshooting aquasecurity/go-gem-version#1 and realized this might be due to a misalignment on how this library parses dependencies, versus how bundler does it.

You can see the regex bundler uses to parse a dependency in lockfiles here.

This is the important piece:

      (?:#{space}\(([^-]*)                               # Space, followed by version
      (?:-(.*))?\))?                                     # Optional platform

Bundler assumes there will be no - in the version string and it parses the platform separately (e.g. -x86_64-darwin)

So for nokogiri (1.13.1-x86_64-darwin):

bundler: name=nokogiri version=1.13.1 platform=x86_64-darwin
go-dep-parser: name=nokogiri version=1.13.1-x86_64-darwin

When go-gem-version sees 1.13.1-x86_64-darwin, it will run into aquasecurity/go-gem-version#1 (and this causes Trivy to skip checks on dependencies that have platformed versions.)

I'll try work on a PR but if someone else wants to take a stab at it, please don't hesitate.

feat(poetry): skip dev dependencies

If a category is dev or something like that, we should skip it.

go-dep-parser/pkg/poetry/parse.go

Line 13 in 3c82977

Category string `toml:"category"`

Is there a reason for ignoring `artifactVersion` in GAV validation for jar files ?

When parsing jar files, go-dep-parser validates GAV info against maven repo when pom.properties is not found

And in the validation, artifactVersion is currently being ignored.

go-dep-parser/pkg/java/jar/parse.go

Lines 141 to 148 in 48d70d5

 if manifestProps.Valid() { 

 // Even if MANIFEST.MF is found, the groupId and artifactId might not be valid. 

 // We have to make sure that the artifact exists actually. 

 if ok, _ := p.client.Exists(manifestProps.GroupID, manifestProps.ArtifactID); ok { 

 // If groupId and artifactId are valid, they will be returned. 

 return append(libs, manifestProps.Library()), nil, nil 

 } 

 }

Is this intentional ? (any reason for not including version here)

Thanks in advance.

Maven coordinates resolved incorrectly

I have a project that (transitively) depends on org.springframework.data:spring-data-commons:jar:3.0.0.

Running trivy fs on my pom file appears to incorrectly resolve its maven coordinates:

DEBUG	Resolving org.springframework.data.build:spring-data-commons:3.0.0...
DEBUG	org.springframework.data.build:spring-data-commons:3.0.0 was not found in local/remote repositories

spring-data-commons' parent is org.springframework.data.build:spring-data-parent:3.0.0, so it appears that the logic that's applying the parent inheritance is incorrectly assigning the child groupId to the parent's groupId.

Question: Support for distinguishing build dependencies?

#119 adds supports for Rust dependencies built with cargo-auditable.

cargo-auditable currently injects runtime and build dependencies, and #119 only extracts the former. Should go-dep-parser also extract the latter, and if so should it also indicate that they are build dependencies somehow?

(Looking at the poetry and cargo lockfile detectors, it looks like there isn't a disambiguation, so I think I can just remove the runtime filtering in the rust binary parser?)

node dependency sometimes listed as vulnerable sometimes not

disclaimer: i am not a nodejs dependency expert

project and lockfile in questionhttps://github.com/hpi-schul-cloud/schulcloud-client/blob/e5275e4b6e3b47a779541128954603a5619a88b9/package-lock.json

mostly trivy lists glob-parent version 3.1.0 as being vulnerable, however sometimes not.
this might be since the dependency is included multiple times due to transitive dependencies

this issue is so common, that i am wondering if we do something wrong or if this is an oversight by the npm parser?
particularly this line sparked my interest:

go-dep-parser/pkg/nodejs/npm/parse.go

Line 59 in 60502da

return utils.UniqueLibraries(libs), uniqueDeps(deps), nil

is this some kind of race condition?

Release it as a binary?

Hi,

Thank you for providing this to the community, you guys rock. 🤘
Are there any plans to release this as an executable/binary so folks that aren't familiar with go can use this tool without having to write go code?

Thanks

Support for pom file parsing for maven repos with username/password

Currently maven repos with credentials doesn't seems to be supported for fetching the packages. Adding the support will be useful for private repositories like which has credentials set
Error when using repo with password

2024-01-22T21:46:39.200+0530    DEBUG   parent POM not found: 2 errors occurred:
        * 'parent.relativePath' points at wrong local POM
        * com.test:sdk-parent:1.0.34 was not found in local/remote repositories

Difference between a `Library` and `Dependency`

I resumed back my work on adding the feature for --include-dev-dependencies for Trivy and have decided that this will be the place to start as to enable the Dev bool for each depndency that is parsed.

However, I'm currently confused a bit with the difference between a Library and a Dependency in context to this package.

A library is defined here in types.go/#L5-L12
A dependency is defined here in types.go/#L19-L22

My interpretation till now is:
A library is the current library we are parsing and a dependency is all the external libraries that this library is dependent upon.

Another confusion that I'm having is, inside any language/framework directory, there's another type definition for a Dependency as for example here in nodejs/npm/parse.go

Would like to know if I am correct/ and if not would kindly request help here. Also, it would be really helpful if I could get some feedback if I'm on the correct place for adding a Dev bool flag in the Library type in the same file as hyperlinked above as.

Golang mod parser reports version that are not used in application

I'm currently looking into why the parser reports versions being used that are not even in my application.
I noticed it's because the issues were being reported in dependencies of dependencies (sometimes even further down the tree), the code was not reachable at all, so the code is not included in the application, yet it reports it as a vulnerability.

Now I do get this, it's impossible for the parser to see if a dependency is actually being used. The good thing is that Go automatically does this for you already. It does this by adding all dependencies into the go.mod file, even the transitive dependencies (marked with //indirect), as long as there is an import somewhere, it will be added to the go.mod file.

The Go blog says this about the go.sum file:

The go command uses the go.sum file to ensure that future downloads of these modules retrieve the same bits as the first download, to ensure the modules your project depends on do not change unexpectedly, whether for malicious, accidental, or other reasons.

So I don't really see why go.sum is used here, and not go.mod. Can we change the parser to use go.mod?
When you execute go mod edit -json you will get a JSON representation of the go.mod file that's very clean and easy to parse.

CC @rahul2393, since you have implemented this dep parser in #21

Add `Dev` flag to yarn.lock parsing

Hello! I've been using trivy to do some scanning of Nodejs applications that use yarn. As you are most likely aware there is no way for Trivy to separate between dev dependencies and production dependencies in this case.

I would love it if Trivy could support this. After some digging through the source code I think this repository is the place to start given that it handles the dependency parsing for packages. Also I saw that you already thought about this thanks to the following todo in yarn/parse.go:

type Dependency struct {
	Version string
	// TODO : currently yarn can't recognize Dev flag.
	// That need to parse package.json for Dev flag
	Dev          bool
	Dependencies map[string]Dependency
}

I guess the only way to support the Dev flag is to include the package.json and compare it to the yarn.lock?

Is this something that sounds reasonable to you?

Thank you for your time! :)

feat(dotnet): Add support for parsing .deps.json files for .Net core

.Net core generates a [appname].deps.json alongside the executable which contains the dependencies of that executable. Parsing it would provide information on dependencies for deployed .Net core executables.

The format is documented here: https://github.com/dotnet/cli/blob/rel/1.0.0/Documentation/specs/runtime-configuration-file.md

Why is the scan for jar packages not including dependencies, such as those declared in pom.xml

indirect dependencies of npm package-lock.json with lockfileversion=3 are not detected

package-lock.json generated with lockfileversion=3 does not include the dependencies array (its deprecated https://docs.npmjs.com/cli/v9/configuring-npm/package-lock-json#lockfileversion).

Current node LTS is 18 which uses lockfileversion=3 hence features that depend on the dependencies array being present will not work since the dependencies are not picked up.

The issue seems to be here:
https://github.com/aquasecurity/go-dep-parser/blob/main/pkg/nodejs/npm/parse.go#L57

The indirect dependencies should be taken from packages field.

Configure Maven search

Can we configure the URL of the Maven repository in Trivy?

It looks like it's set here

go-dep-parser/pkg/java/jar/parse.go

Line 29 in 5be6514

baseURL = "https://search.maven.org/solrsearch/select"

I ask for two reasons

Maven is rate limited, using this tool on a CI server means that queries start getting blocked
Many people/organisations have a local cache of Maven which would be faster to access from a CI server

Ideally I would like to see a CLI flag for the Trivy tool to set the an alternative repository search tool, e.g. Nexus or Artifactory.

The OWASP dependency scanning tool has the same functionality, I don't know if their implementation is useful as a reference point

https://owasp.org/www-project-dependency-check/

New Conan parser does not handle revisions properly

Hi,

I saw the new parser for conan.lock files was pushed a few hours ago, so I decided to give a try on some of my projects.
However, the parser fails to parse my sample file because I have enabled Conan's revision support (see https://docs.conan.io/en/latest/versioning/revisions.html).
In my case, the references inside the lock file look like pkg/version#rrev (rrev = recipe revision), e.g. openssl/3.0.5#8743768a28916117e28629f0c4921a29.

I end up with the following error when parsing the file:
WARN version error (3.0.5#8743768a28916117e28629f0c4921a29): malformed version: 3.0.5#8743768a28916117e28629f0c4921a29

Unfortunately I cannot share my sample file, sorry.

I think the fix would be a single one-line change at https://github.com/aquasecurity/go-dep-parser/blob/main/pkg/c/conan/parse.go#L97 to also split the string on "#" (in addition to "@") when looking for the package's version.

Regards,
François

The Java base class jar package version is not recognized

hi,I am using this program to recognize Java language files. During the recognition process, I found that the version of the jar package of the basic Java library cannot be recognized, such as ()
US_export_policy.jar
charsets.jar
cldrdata.jar
dnsns.jar
icedtea-sound.jar
jaccess.jar
java-atk-wrapper.jar
jce.jar
jsse.jar 2
local_policy.jar
localedata.jar
management-agent.jar
/usr/share/elasticsearch/lib/plugin-cli-5.6.12.jar 2
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/resources.jar
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/sunec.jar
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/sunjce_provider.jar
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/sunpkcs11.jar
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/zipfs.jar

Consider using debug/buildinfo instead of custom parse code

With the release of 1.18, go binary dependencies can be parsed using debug/buildinfo package. I shared a version in PR .

Support pom.xml with modules element

In case the pom.xml file structure includes modules element, go-dep-parser doesn't return all the dependencies.

We have to run over the other pom.xml files from the modules and parse the dependencies in each one.

feat(pom): does not support running from another directory

if we run for instance from directory that we want to scan

cd originalrepo
trivy fs -d .

we can see currently 33 files that trivy can find. However, if we execute

cd ..
trivy fs -d originalrepo

if displays only 28 files that is found by trivy

Sometimes the test in java/pom fails

Running tests using the command go test -v pkg/java/pom/parse_test.go with some low probability falls:

=== RUN   TestPom_Parse
=== RUN   TestPom_Parse/local_repository
=== RUN   TestPom_Parse/remote_repository
=== RUN   TestPom_Parse/offline_mode
=== RUN   TestPom_Parse/inherit_parent_properties
=== RUN   TestPom_Parse/inherit_project_properties_from_parent
    parse_test.go:525: 
                Error Trace:    /Users/tososomaru/projects/go-dep-parser/pkg/java/pom/parse_test.go:525
                Error:          Not equal: 
                                expected: []types.Library{types.Library{ID:"", Name:"com.example:child", Version:"2.0.0", Indirect:false, License:"", ExternalReferences:[]types.ExternalRef(nil), Locations:[]types.Location(nil), FilePath:""}, types.Library{ID:"", Name:"org.example:example-api", Version:"2.0.0", Indirect:false, License:"", ExternalReferences:[]types.ExternalRef(nil), Locations:[]types.Location(nil), FilePath:""}}
                                actual  : []types.Library{types.Library{ID:"", Name:"com.example:child", Version:"2.0.0", Indirect:false, License:"", ExternalReferences:[]types.ExternalRef(nil), Locations:[]types.Location(nil), FilePath:""}, types.Library{ID:"", Name:"org.example:example-api", Version:"3.0.0", Indirect:false, License:"", ExternalReferences:[]types.ExternalRef(nil), Locations:[]types.Location(nil), FilePath:""}}
                            
                                Diff:
                                --- Expected
                                +++ Actual
                                @@ -14,3 +14,3 @@
                                   Name: (string) (len=23) "org.example:example-api",
                                -  Version: (string) (len=5) "2.0.0",
                                +  Version: (string) (len=5) "3.0.0",
                                   Indirect: (bool) false,
                Test:           TestPom_Parse/inherit_project_properties_from_parent
=== RUN   TestPom_Parse/inherit_properties_in_parent_depManagement_with_import_scope
=== RUN   TestPom_Parse/dependencyManagement_prefers_child_properties
=== RUN   TestPom_Parse/inherit_parent_dependencies
=== RUN   TestPom_Parse/inherit_parent_dependencyManagement
=== RUN   TestPom_Parse/transitive_parents
=== RUN   TestPom_Parse/parent_relativePath
=== RUN   TestPom_Parse/parent_in_a_remote_repository
=== RUN   TestPom_Parse/soft_requirement
=== RUN   TestPom_Parse/soft_requirement_with_transitive_dependencies
=== RUN   TestPom_Parse/hard_requirement_for_the_specified_version
=== RUN   TestPom_Parse/version_requirement
=== RUN   TestPom_Parse/import_dependencyManagement
=== RUN   TestPom_Parse/import_multiple_dependencyManagement
=== RUN   TestPom_Parse/exclusions
=== RUN   TestPom_Parse/exclusions_with_wildcards
=== RUN   TestPom_Parse/multi_module
=== RUN   TestPom_Parse/multi_module_soft_requirement
=== RUN   TestPom_Parse/overwrite_artifact_version_from_dependencyManagement_in_the_root_POM
=== RUN   TestPom_Parse/transitive_dependencyManagement_should_not_be_inherited
=== RUN   TestPom_Parse/parent_not_found
=== RUN   TestPom_Parse/dependency_not_found
=== RUN   TestPom_Parse/module_not_found_-_unable_to_parse_module
--- FAIL: TestPom_Parse (0.02s)
    --- PASS: TestPom_Parse/local_repository (0.00s)
    --- PASS: TestPom_Parse/remote_repository (0.00s)
    --- PASS: TestPom_Parse/offline_mode (0.00s)
    --- PASS: TestPom_Parse/inherit_parent_properties (0.00s)
    --- FAIL: TestPom_Parse/inherit_project_properties_from_parent (0.00s)
    --- PASS: TestPom_Parse/inherit_properties_in_parent_depManagement_with_import_scope (0.00s)
    --- PASS: TestPom_Parse/dependencyManagement_prefers_child_properties (0.00s)
    --- PASS: TestPom_Parse/inherit_parent_dependencies (0.00s)
    --- PASS: TestPom_Parse/inherit_parent_dependencyManagement (0.00s)
    --- PASS: TestPom_Parse/transitive_parents (0.00s)
    --- PASS: TestPom_Parse/parent_relativePath (0.00s)
    --- PASS: TestPom_Parse/parent_in_a_remote_repository (0.00s)
    --- PASS: TestPom_Parse/soft_requirement (0.00s)
    --- PASS: TestPom_Parse/soft_requirement_with_transitive_dependencies (0.00s)
    --- PASS: TestPom_Parse/hard_requirement_for_the_specified_version (0.00s)
    --- PASS: TestPom_Parse/version_requirement (0.00s)
    --- PASS: TestPom_Parse/import_dependencyManagement (0.00s)
    --- PASS: TestPom_Parse/import_multiple_dependencyManagement (0.00s)
    --- PASS: TestPom_Parse/exclusions (0.00s)
    --- PASS: TestPom_Parse/exclusions_with_wildcards (0.00s)
    --- PASS: TestPom_Parse/multi_module (0.00s)
    --- PASS: TestPom_Parse/multi_module_soft_requirement (0.00s)
    --- PASS: TestPom_Parse/overwrite_artifact_version_from_dependencyManagement_in_the_root_POM (0.00s)
    --- PASS: TestPom_Parse/transitive_dependencyManagement_should_not_be_inherited (0.00s)
    --- PASS: TestPom_Parse/parent_not_found (0.00s)
    --- PASS: TestPom_Parse/dependency_not_found (0.00s)
    --- PASS: TestPom_Parse/module_not_found_-_unable_to_parse_module (0.00s)
FAIL
FAIL    command-line-arguments  0.187s
FAIL

git status
On branch main
Your branch is up to date with 'origin/main'.

Add CircleCI to run the tests

This repository has some tests so we should automate them with CI

panic: runtime error: invalid memory address or nil pointer dereference

goroutine 23145 [running]:
github.com/aquasecurity/go-dep-parser/pkg/golang/mod.Parse({0x7fd5e5e7f3d0, 0xc010539f70})
    /runner/_work/image-scanner/image-scanner/vendor/github.com/aquasecurity/go-dep-parser/pkg/golang/mod/parse.go:29 +0x13c
github.com/aquasecurity/fanal/analyzer/language.Analyze({0x28b10eb, 0x5}, {0xc009ec0500, 0x4f}, {0x7fd5e5e7f3d0?, 0xc010539f70?}, 0x5d3f38?)
    /runner/_work/image-scanner/image-scanner/vendor/github.com/aquasecurity/fanal/analyzer/language/analyze.go:16 +0x68
github.com/aquasecurity/fanal/analyzer/language/golang/mod.gomodAnalyzer.Analyze({}, {0x26b57a0?, 0x0?}, {{0x0, 0x0}, {0xc009ec0500, 0x4f}, {0x2efe778, 0xc000bfad20}, {0x7fd5e5e7f3a0, ...}, ...})
    /runner/_work/image-scanner/image-scanner/vendor/github.com/aquasecurity/fanal/analyzer/language/golang/mod/mod.go:39 +0xc5
github.com/aquasecurity/fanal/analyzer.AnalyzerGroup.AnalyzeFile.func1({0x2ef93e8, 0x4520270}, {0x2efaed0?, 0xc010539f70})
    /runner/_work/image-scanner/image-scanner/vendor/github.com/aquasecurity/fanal/analyzer/analyzer.go:258 +0x253
created by github.com/aquasecurity/fanal/analyzer.AnalyzerGroup.AnalyzeFile
    /runner/_work/image-scanner/image-scanner/vendor/github.com/aquasecurity/fanal/analyzer/analyzer.go:253 +0x3d8

https://github.com/aquasecurity/go-dep-parser/blob/main/pkg/golang/mod/parse.go#L29

ping @jerbob92 you need to check modFileParsed.Go != nil

POM runtime scope ignored

If I understand, only compile scope dependencies are listed:

go-dep-parser/pkg/java/pom/parse.go

Lines 381 to 385 in 423cd04

 if (d.Scope != "" && d.Scope != "compile") || d.Optional { 

 continue 

 } 

 dependencies = append(dependencies, d.ToArtifact(opts))

So when I use Trivy, it seems what vulnerabilities in runtime scope dependencies are not found.
For example, if spring-cloud-starter-netflix-eureka-client is present in the pom.xml :

org.springframework.cloud:spring-cloud-starter-netflix-eureka-client:4.0.3 [compile]
   -> com.netflix.eureka:eureka-client:2.0.1 [compile]
       -> org.codehaus.jettison:jettison:1.4.0 [runtime]

org.codehaus.jettison:jettison:1.4.0 is not analyzed, and these vulnerabilities are not found:

Using CycloneDX sbom:

├────────────────────────────────────────────┼────────────────┼──────────┤        ├───────────────────┼─────────────────────────────────────┼──────────────────────────────────────────────────────────────┤
│ org.codehaus.jettison:jettison             │ CVE-2022-40150 │ HIGH     │        │ 1.4.0             │ 1.5.2                               │ jettison: memory exhaustion via user-supplied XML or JSON    │
│                                            │                │          │        │                   │                                     │ data                                                         │
│                                            │                │          │        │                   │                                     │ https://avd.aquasec.com/nvd/cve-2022-40150                   │
│                                            ├────────────────┤          │        │                   │                                     ├──────────────────────────────────────────────────────────────┤
│                                            │ CVE-2022-45685 │          │        │                   │                                     │ jettison: stack overflow in JSONObject() allows attackers to │
│                                            │                │          │        │                   │                                     │ cause a Denial of...                                         │
│                                            │                │          │        │                   │                                     │ https://avd.aquasec.com/nvd/cve-2022-45685                   │
│                                            ├────────────────┤          │        │                   │                                     ├──────────────────────────────────────────────────────────────┤
│                                            │ CVE-2022-45693 │          │        │                   │                                     │ jettison: If the value in map is the map's self, the...      │
│                                            │                │          │        │                   │                                     │ https://avd.aquasec.com/nvd/cve-2022-45693                   │
│                                            ├────────────────┤          │        │                   ├─────────────────────────────────────┼──────────────────────────────────────────────────────────────┤
│                                            │ CVE-2023-1436  │          │        │                   │ 1.5.4                               │ jettison: Uncontrolled Recursion in JSONArray                │
│                                            │                │          │        │                   │                                     │ https://avd.aquasec.com/nvd/cve-2023-1436                    │
│                                            ├────────────────┼──────────┤        │                   ├─────────────────────────────────────┼──────────────────────────────────────────────────────────────┤
│                                            │ CVE-2022-40149 │ MEDIUM   │        │                   │ 1.5.1                               │ jettison: parser crash by stackoverflow                      │
│                                            │                │          │        │                   │                                     │ https://avd.aquasec.com/nvd/cve-2022-40149                   │
├────────────────────────────────────────────┼────────────────┤          │        ├───────────────────┼─────────────────────────────────────┼──────────────────────────────────────────────────────────────┤

I think runtime dependencies should be included.

`go-dep-parser` handles incorrectly a package name with a hyphen

there is a problem if a package name contains a hyphen.

go-dep-parser doesn't detect next packages for python:

oauth2-client==4.0.0
python-gitlab==2.0.0

Reason:
rStripByKey(line, hashMarker) cuts incorrectly such names.

go-dep-parser/pkg/python/pip/parse.go

Lines 44 to 49 in 4a30ebc

 func rStripByKey(line string, key string) string { 

 if pos := strings.IndexAny(line, key); pos >= 0 { 

 line = strings.TrimRightFunc((line)[:pos], unicode.IsSpace) 

 } 

 return line 

 }

add dependency tree and line numbers to rust cargo lock files

Removal of golang.org/x/xerrors

golang.org/x/x errors can be replaced by "errors" or "fmt" modules
Quoted from https://pkg.go.dev/golang.org/x/xerrors#pkg-overview

These functions were incorporated into the standard library's errors package in Go 1.13: - Is - As - Unwrap
Also, Errorf's %w verb was incorporated into fmt.Errorf.
Use this package to get equivalent behavior in all supported Go versions.

Support pnpm-lock.yaml

Description

Parse pnpm-lock.yaml so as to extract package names/versions.

Specification:
https://github.com/pnpm/spec/blob/master/lockfile/5.md

Example:
https://github.com/pnpm/pnpm/blob/main/pnpm-lock.yaml

Issue

aquasecurity/trivy#1200

Add direct parent field to NPM dep tree

In addition to the 'DependsOn' field of the 'Dependency' struct, we want to have another field called "DirectParents".
It will contain the direct parents of the library (could be more than one).
That will give us the ability to locate the root library from the child dependency.

	if manifestProps.Valid() {
	// Even if MANIFEST.MF is found, the groupId and artifactId might not be valid.
	// We have to make sure that the artifact exists actually.
	if ok, _ := p.client.Exists(manifestProps.GroupID, manifestProps.ArtifactID); ok {
	// If groupId and artifactId are valid, they will be returned.
	return append(libs, manifestProps.Library()), nil, nil
	}
	}

	if (d.Scope != "" && d.Scope != "compile") \|\| d.Optional {
	continue
	}

	dependencies = append(dependencies, d.ToArtifact(opts))

	func rStripByKey(line string, key string) string {
	if pos := strings.IndexAny(line, key); pos >= 0 {
	line = strings.TrimRightFunc((line)[:pos], unicode.IsSpace)
	}
	return line
	}

aquasecurity / go-dep-parser Goto Github PK

go-dep-parser's Introduction

go-dep-parser

go-dep-parser's People

Contributors

Stargazers

Watchers

Forkers

go-dep-parser's Issues

Description

Description

Desired Behavior

Actual Behavior

Reproduction Steps

Operating System

Version

Description

Issue

Recommend Projects

Recommend Topics

Recommend Org