unified-agent-distribution's People
Contributors
Stargazers
Watchers
Forkers
ritu-gahalot headers13 elikkatzgit amitechno arunpillai6 tottotza1 ta185044 nevskrem julianvilas nishathf connectionmaster indusai rmarros manikumar4550 nagrawal3 randygeyer-ws joshnewton31080 matti santhosh4git truongdnascript whitesource-ft sharkness samjcs jiyamalik james-nesbitt meramsey thisdevopsnerd yashkohli88 hajicirruslabs hrlourenco jgeraigery dsztanko anipokhrel phillyc svendr alainlompo csongs-forks techdna cameroncheung-mend kydell pavantatikonda pedro-almeida-diconium bbenzikry arupghoshshell mohammedshaikalis murphyzh0 bignuoliunified-agent-distribution's Issues
Presence of Cocoapods Podfile causes WhiteSource to not index cpp files
Our repo has a Cocoapods Podfile in one of the sub-dirs, and lots of C++ source files scattered throughout the repo. I am seeing that when I run the unified WS agent on our repo, it fails to index the C++ source files by default.
After some trial and error, I see that I need to disable Cocoapods dependency resolution (by setting cocoapods.resolveDependencies=false in the config file) for the agent to actually index the cpp files.
Is this expected? If yes, I think we can agree this is not ideal behavior; ideally we'd want both Cocoapods dependency resolution as well as regular indexing of files.
If there is some other forum where I should be posting about this, please do let me know, thanks.
Thanks
Produce a static binary
Running the unified agent has a dependency on java, which complicates its use.
We have a large gitlab estate and want to provide a centralized mend job for all of our users to import. If the unified agent was a single statically linked executable, we could tell our users "add this ci file import wherever you build your code, and it will just work". However, because it's a jar all of our users have to add java to their build environments.. which is especially painful since we don't need java for anything else.
Using something like graalVM's native image builder seems like a promising solution https://www.graalvm.org/22.0/reference-manual/native-image/
generateScanReport doesn't work with UA Version 21.12.1
Hi
We have the parameter generateScanReport=true and scanReportFilenameFormat=static for UA 21.12.1 and the scan_report.json is not written to whitesource/scan_report.json. We even have this from the logs:
scan report created successfully at /vsts/agent/_work/24/s/./whitesource/scan_report.json
But when checking there is NO folder or NO file with the scan_report.json name.
We had before the upgrade the UA 21.11.2 and the report was generated correctly, we have to disable now the WS scanning because the automation was base on that file.
This started happening the moment UA was upgraded o 21.12.1 (was working on 21.11.2)
BR
Joaquin
Dotnet Central Package Management
Hey,
So mend doesn't seem to pick up on our changes since we moved over to central package management. Is there some configuration setting that needs to be set? CPM basically means we have one props file that encompasses all our packages.
https://learn.microsoft.com/en-us/nuget/consume-packages/central-package-management
Version 21.6.2 wrongly interprets`gradle.preferredEnvironment`
Hello Whitesource Team!
According to the documentation of Unified Agent Configuration when gradle.preferredEnvironment is set to wrapper the gradlew command will be used instead of gradle to execute gradle CLI.
However, since 21.6.2 the library seems to accept only gradle.preferredEnvironment=gradlew for this purpose, ignoring gradle.preferredEnvironment=wrapper and introducing incompatibility between 21.6.2 and 21.6.1 versions.
Could you correct the documentation page or accept the old wrapper value in 21.6.2?
Thank you.
gradle.preferredEnvironment ignored
Hello Whitesource-Team,
we're using version 22.1.2 of Unified Agent. Running with gradle.preferredEnvironment set to "wrapper" is ignored and ends with error
"Command - executeProcess - error in execute command 'cmd /c gradle -v', Exit Status 1".
Are there probably undocumented changes in settings?
Thanks in advance
Peter
sendScanResults fails with NullPointerException in v22.4.1
Our pipeline worked fine with the previous versions, with the latest upgrade to v22.4.1 happening yesterday, all scan-pipelines fail with a NullPointerException. When checking the provided jar there's also an "internal error" when trying to open the file "WssServiceClientImpl".
Will there be a fixed version or did this include a breaking change that must be adapted on our side?
Stacktrace:
[DEBUG] [2022-05-01 14:43:41,294 +0200[ - Failed to send request to WhiteSource server: Server error: java.lang.NullPointerException
org.whitesource.agent.client.WssServiceException: Server error: java.lang.NullPointerException
at org.whitesource.agent.client.WssServiceClientImpl.extractResultData(WssServiceClientImpl.java:492)
at org.whitesource.agent.client.WssServiceClientImpl.service(WssServiceClientImpl.java:339)
at org.whitesource.agent.client.WssServiceClientImpl.checkPolicyCompliance(WssServiceClientImpl.java:209)
at org.whitesource.agent.client.WhitesourceService.checkPolicyCompliance(WhitesourceService.java:623)
at org.whitesource.request.ProjectsSender.checkPolicies(ProjectsSender.java:730)
at org.whitesource.request.ProjectsSender.sendRequest(ProjectsSender.java:270)
at org.whitesource.request.ProjectsSender.sendProjects(ProjectsSender.java:165)
at org.whitesource.fs.Main.sendScanResults(Main.java:209)
at org.whitesource.fs.Main.main(Main.java:95)
[INFO] [2022-05-01 14:43:41,296 +0200[ - Process finished with exit code SERVER_FAILURE (Failed to send request to WhiteSource server: Server error: java.lang.NullPointerException
About pnpm support
Hi, Team.
I'm using pnpm in my project and CVE issues are not detected.
Does unified agent support pnpm (pnpm-lock.yaml)? If not, do you plan to support it in the future?
generateScanReport doesn't work with UA Version 21.12.2
From this issue #24 scan report doesnt work in Version 21.12.2
In log file i can see "scan report created successfully at /vsts/agent/_work/24/s/./whitesource/scan_report.json", but if i check this source, i cant find ./whitesource/scan_report.json...
I need this report for create new item in my board, its important for me...
Do you have any ideas how to fix this?
Whitesource agent jar not scanning specified dir in jenkins + throwing Exception thread "main" java.lang.OutOfMemoryError: GC overhead limit exceed
whitesource agent jar is scanning directory specified along with it ,it is scanning another directory in jenkins.
[INFO] [2022-01-22 01:39:36,332 +0000] - Scanning directories [/var/lib/jenkins/jobs/Tools/jobs/Whitesource/jobs/Artifactory-WhiteSource-Scan-common-folder/workspace/pipeline-scripts/ARTIFACTS, /tmp/ws-ua_20220122010754_IPJCFV/archiveExtraction_XHMMNY/RZMXZB/20220122010756] for matching source/binary file types (may take a few minutes) [INFO] [2022-01-22 01:53:11,287 +0000] - Total files found according to the includes/excludes pattern: 820,809
java -jar $WSS/wss-unified-agent.jar -c $WSS/config.properties -d /var/lib/jenkins/jobs/Tools/jobs/Whitesource/jobs/Artifactory-WhiteSource-Scan-common-folder/workspace/pipeline-scripts/ARTIFACTS
It should scan only files present in /var/lib/jenkins/jobs/Tools/jobs/Whitesource/jobs/Artifactory-WhiteSource-Scan-common-folder/workspace/pipeline-scripts/ARTIFACTS, but it is taking files present in /tmp/ws-ua_20220122010754_IPJCFV/archiveExtraction_XHMMNY/RZMXZB/20220122010756.
In path specified we dont have 8,20,000 , I dont know from where 8,20,000 files are coming.
Also at the end we are getting below error
[INFO] [2022-01-22 02:25:49,888 +0000] -
-------------------- End: Scan Files Matching Includes Pattern ---------
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.Arrays.copyOf(Arrays.java:3332) at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448) at java.lang.StringBuilder.append(StringBuilder.java:136) at ch.qos.logback.core.pattern.FormattingConverter.write(FormattingConverter.java:39) at ch.qos.logback.core.pattern.PatternLayoutBase.writeLoopOnConverters(PatternLayoutBase.java:115) at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:165) at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:39) at ch.qos.logback.core.encoder.LayoutWrappingEncoder.encode(LayoutWrappingEncoder.java:116) at ch.qos.logback.core.OutputStreamAppender.subAppend(OutputStreamAppender.java:230) at ch.qos.logback.core.rolling.RollingFileAppender.subAppend(RollingFileAppender.java:235) at ch.qos.logback.core.OutputStreamAppender.append(OutputStreamAppender.java:102) at ch.qos.logback.core.UnsynchronizedAppenderBase.doAppend(UnsynchronizedAppenderBase.java:84) at ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:51) at ch.qos.logback.classic.Logger.appendLoopOnAppenders(Logger.java:270) at ch.qos.logback.classic.Logger.callAppenders(Logger.java:257) at ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:421) at ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:383) at ch.qos.logback.classic.Logger.debug(Logger.java:494) at org.whitesource.utils.logger.LoggerFS.debug(LoggerFS.java:118) at org.whitesource.agent.FileSystemScanner.createProjects(FileSystemScanner.java:537) at org.whitesource.fs.scanOrigins.GeneralScanOrigin.getProjects(GeneralScanOrigin.java:189) at org.whitesource.fs.scanOrigins.GeneralScanOrigin.scan(GeneralScanOrigin.java:98) at org.whitesource.fs.scanOrigins.ScanOrigin.runOriginScan(ScanOrigin.java:36) at org.whitesource.fs.FileSystemAgent.createProjects(FileSystemAgent.java:174) at org.whitesource.fs.Main.scanProjects(Main.java:115) at org.whitesource.fs.Main.main(Main.java:90)
npm.resolveDependencies = true, but resolving dependencies failed
i have a node project, but when running a scan i get:
npm.resolveDependencies = true, but resolving dependencies failed
which results in 0 libraries scanned:
"libraries": []
what could be the issue?
Missing wss-unified-agent.jar
My Azure devops pipeline is pulling the agent from https://github.com/whitesource/unified-agent-distribution/raw/master/standAlone/wss-unified-agent.jar, but the file appears to be missing. Is this intentional? I see it was deleted on Apr 12, but never replaced.
Specify WS project names
Hello WhiteSource Colleagues,
After our migration to WhiteSource Unified Agent I see that in the scan result leafIds were changed to respective folders' names. WS project names are getting created automatically as per the repo structure. My question is: can we specify custom leafIds to not use default values to not break our functionality based on names used before the migration?
BR,
Dmitry
[Bug] Too many false-positive
I tried it on my Android repo but, almost all of the issued cases were false-positive. For example Drjacky/MVIModularizationTemplate#19
It says there is an issue with kotlin-stdlib-1.4.31.jar but, neither my repo nor the internal library(root dependency) use that version:
Third-library: https://github.com/detekt/sarif4k/blob/main/build.gradle.kts
User license associated to this repo?
Hello, I want to use this JAR in my project. What is the end-user license associated to this jar?
pipenv lock -r: Error: No such option: -r
When we try to scan Python code with Mend Java agent (release v23.7.1) and Pipenv we get the following error:
[WARN] [2023-08-07 16:20:39,771 +0000] - Command - executeProcess - error in execute command 'pipenv lock -r', Exit Status 2
[WARN] [2023-08-07 16:20:39,772 +0000] - Read error line #1: Usage: pipenv lock [OPTIONS]
[WARN] [2023-08-07 16:20:39,772 +0000] - Read error line #2: Try 'pipenv lock -h' for help.
[WARN] [2023-08-07 16:20:39,772 +0000] - Read error line #3: Error: No such option: -r
[ERROR] [2023-08-07 16:20:39,772 +0000] - Error occurred while running command pipenv lock -r in /home/jenkins/agent/workspace/...
It seems that the agent under the hood using the old syntax of pipenv which is not available anymore.
Is there any way to handle/configure the option to avoid that kind of issue?
Mend Unified Agent Could not find Jar files on v24.3.2
After upgrading the unified agaent, it can no longer find some jars within our project. Which results in the following error:
[DEBUG] [2024-04-15 18:37:30,658 -0400] - converting node to dependency : commons-codec
[DEBUG] [2024-04-15 18:37:30,658 -0400] - converting node to dependency : gson
[DEBUG] [2024-04-15 18:37:30,658 -0400] - Could not find jar file for com.google.code.gson-gson
This then results in the following whitesource error:
[EUA010]: A dependency filename was not found or is possibly corrupted (artifactId:gson)
Would anyone know how to resolve this? Please let me know if you would need more information from me.
Since JDK 11.0.20 the unified agent is failing with : failed to execute Unified Agent scan: Failed to determine UA version
When Using JDK 11.0.20 or higher the unified agent is failing with:
failed to execute Unified Agent scan: Failed to determine UA version: running command 'java' failed: cmd.Run() failed: exit status 1
When checking the stacktrace we found [1].
Then we checked the release notes of JDK 11.0.20. and found that the unified agent manifest file is now just to big.
Could you please check this and provide a manifest file which fulfills the basic requirements of Java JDK 11.0.20.
Thanks for your help.
[2]
https://www.oracle.com/java/technologies/javase/11-0-20-relnotes.html#JDK-8300596
[1]
rror: An unexpected error occurred while trying to open file .\wss-unified-agent.jar
java.io.IOException: Unsupported size: 8577957 for JarEntry META-INF/MANIFEST.MF. Allowed max size: 8000000 bytes
at java.base/java.util.jar.JarFile.getBytes(JarFile.java:810)
at java.base/java.util.jar.JarFile.getManifestFromReference(JarFile.java:421)
at java.base/java.util.jar.JarFile.getManifest(JarFile.java:408)
at java.base/sun.launcher.LauncherHelper.getMainClassFromJar(LauncherHelper.java:553)
at java.base/sun.launcher.LauncherHelper.loadMainClass(LauncherHelper.java:778)
at java.base/sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:686)
UA issue when pnpm is used alongside with Lerna
Hi, here's my setup:
pnpm-lock.yamlfile sits side-by-side withlerna.jsonnpmClientinmy lerna.jsonis set topnpmnpm.resolveDependencies=trueandnpm.resolveLockFile=truein mywss-unified-agent.config
According to https://docs.mend.io/bundle/unified_agent/page/unified_agent_configuration_parameters.html#JavaScript:
If a Lerna.json file is identified, then Lerna’s npmClient will determine if to run npm or yarn resolution.
The pnpm-lock.yaml file will be resolved if it is the only lock file identified next to the package.json file.
When the scan runs, though, the UA seems to prioritise lerna.json over pnpm-lock.yaml. Thing is, the UA doesn't seem to support pnpm as Lerna's npmClient because it keeps complaining that package-lock.json file is not there when resolving the dependencies. My expectation is either:
pnpm-lock.yamltakes precedence overlerna.json, or- pnpm is supported as Lerna's npmClient when
lerna.jsonis used.
Thoughts? Thank you.
error in execute command 'mvn -T1 dependency:tree -DoutputFile=whitesource_mvn_dependency_tree.txt -B', Exit Status 1
Hi team,
Hope you are doing good and staying well 💯
I've temporary done a workaround for the issue by running a clean install.
However, according to this article: https://stackoverflow.com/questions/28835418/mvn-dependencytree-fails-on-trivial-project. It seems that we have to clean and install all the jar files locally before running the dependency tree.
This seems to be a maven behaviour.
This seems to happen for SNAPSHOT builds.
Certificate signed jar expired
Currently our CICD pipelines is checking the signature on the code in the jar before running it. Using the following command:
jarsigner -verify -strict wss-unified-agent.jar
This currently fails since some of the content is signed by an expired certificate.
This is the feedback from the jarsigner
- Signed by "CN=whitesource software inc, O=whitesource software inc, STREET=79 Madison Ave, L=New York, ST=New York, OID.2.5.4.17=10016, C=US"
Digest algorithm: SHA-256
Signature algorithm: SHA256withRSA, 2048-bit key
Timestamped by "CN=DigiCert Timestamp 2022 - 2, O=DigiCert, C=US" on Mon Dec 26 11:12:32 UTC 2022
Timestamp digest algorithm: SHA-256
Timestamp signature algorithm: SHA256withRSA, 4096-bit key
jar verified.
Warning:
This jar contains entries whose signer certificate has expired.
With additional information on the certificates used for signing:
>>> Signer
X.509, CN=whitesource software inc, O=whitesource software inc, STREET=79 Madison Ave, L=New York, ST=New York, OID.2.5.4.17=10016, C=US
Signature algorithm: SHA256withRSA, 2048-bit key
[certificate expired on 12/12/2022, 00:59]
X.509, CN=Sectigo RSA Code Signing CA, O=Sectigo Limited, L=Salford, ST=Greater Manchester, C=GB
Signature algorithm: SHA384withRSA, 2048-bit key
[certificate is valid from 02/11/2018, 01:00 to 01/01/2031, 00:59]
X.509, CN=USERTrust RSA Certification Authority, O=The USERTRUST Network, L=Jersey City, ST=New Jersey, C=US
Signature algorithm: SHA384withRSA, 4096-bit key
[trusted certificate]
>>> TSA
X.509, CN=DigiCert Timestamp 2022 - 2, O=DigiCert, C=US
Signature algorithm: SHA256withRSA, 4096-bit key
[certificate is valid from 21/09/2022, 02:00 to 22/11/2033, 00:59]
X.509, CN=DigiCert Trusted G4 RSA4096 SHA256 TimeStamping CA, O="DigiCert, Inc.", C=US
Signature algorithm: SHA256withRSA, 4096-bit key
[certificate is valid from 23/03/2022, 01:00 to 23/03/2037, 00:59]
X.509, CN=DigiCert Trusted Root G4, OU=www.digicert.com, O=DigiCert Inc, C=US
Signature algorithm: SHA384withRSA, 4096-bit key
[certificate is valid from 01/08/2022, 02:00 to 10/11/2031, 00:59]
java.lang.SecurityException: class "yD" on Corretto-11.0.15.9.1 with Log4jHotPatch
Running the wss-unified-agent.jar on Amazon Linux with Corretto-11.0.15.9.1 leads to the following exception
java.lang.SecurityException: class "yD"'s signer information does not match signer information of other classes in the same package
at java.base/java.lang.ClassLoader.checkCerts(ClassLoader.java:1151)
...
org.whitesource.agent.dependency.resolver.js.npm.NpmAdditionalDependencies.getAnalysisModules(NpmAdditionalDependencies.java:77)
...
(Full Stacktrace below [1].)
In particular, this happens when the unified agent tries to scan npm Dependencies
npm.resolveDependencies=true
npm.resolveAdditionalDependencies=true
The root cause is, that the JVM distribution on Amazon Linux contains a Log4jHotPatch.jar with a Log4jHotPatch.class that is in the root package:
[10.993s][info][class,load] Log4jHotPatch source: file:/usr/share/log4j-cve-2021-44228-hotpatch/jdk11/Log4jHotPatch.jar
The (obsfucated) class 'yD' in the wss-unified-agent.jar is als in the root package.
The exception occurs because the two classes are from different jars (with different signers) but from the same package (root, i.e., no package).
[1] Full Stacktrace
DEBUG] [2022-05-05 16:57:17,470 +0200] - error:
java.lang.SecurityException: class "yD"'s signer information does not match signer information of other classes in the same package
at java.base/java.lang.ClassLoader.checkCerts(ClassLoader.java:1151)
at java.base/java.lang.ClassLoader.preDefineClass(ClassLoader.java:906)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1015)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:698)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
at whitesource.analysis.vulnerabilities.ExternalDependencyExecutor.<init>(Unknown Source)
at org.whitesource.agent.dependency.resolver.js.npm.NpmAdditionalDependencies.getAnalysisModules(NpmAdditionalDependencies.java:77)
at org.whitesource.agent.dependency.resolver.js.JSDependencyResolver.invokeAdditionalDependencies(JSDependencyResolver.java:520)
at org.whitesource.agent.dependency.resolver.js.JSDependencyResolver.npmLsResolution(JSDependencyResolver.java:440)
at org.whitesource.agent.dependency.resolver.js.JSDependencyResolver.invokeNpm(JSDependencyResolver.java:391)
at org.whitesource.agent.dependency.resolver.js.JSDependencyResolver.resolve(JSDependencyResolver.java:281)
at org.whitesource.agent.dependency.resolver.js.JSDependencyResolver.resolveDependencies(JSDependencyResolver.java:214)
at org.whitesource.agent.dependency.resolver.DependencyResolutionService.resolveDependenciesOfResolver(DependencyResolutionService.java:380)
at org.whitesource.agent.dependency.resolver.DependencyResolutionService.resolveDependencies(DependencyResolutionService.java:206)
at org.whitesource.agent.FileSystemScanner.createProjects(FileSystemScanner.java:409)
at org.whitesource.fs.scanOrigins.GeneralScanOrigin.getProjects(GeneralScanOrigin.java:194)
at org.whitesource.fs.scanOrigins.GeneralScanOrigin.scan(GeneralScanOrigin.java:101)
at org.whitesource.fs.scanOrigins.ScanOrigin.runOriginScan(ScanOrigin.java:36)
at org.whitesource.fs.FileSystemAgent.createProjects(FileSystemAgent.java:165)
at org.whitesource.fs.Main.scanProjects(Main.java:116)
at org.whitesource.fs.Main.main(Main.java:91)
WSS agent is not able to scan docker images .tar files with gzip layers
If I save the docker images with docker save everything is working properly, but if I use the go-containerregistry format for creating the .tar file, then WSS agent fails with the following error:
[DEBUG] [2022-01-12 11:13:47,206 +0100] - Proxy host not provided
[DEBUG] [2022-01-12 11:13:47,214 +0100] - Base url:'https://saas.whitesourcesoftware.com'
[INFO] [2022-01-12 11:13:47,343 +0100[ - UnifiedAgent version (pluginVersion) : 21.12.2
[INFO] [2022-01-12 11:13:47,349 +0100[ -
...
------------------------------------------------------------------------
-------------------- Start: Docker Resolver Scan -----------------------
------------------------------------------------------------------------
[DEBUG] [2022-01-12 11:13:47,359 +0100[ - Scanning /Users/xxxx/Projects/test/image.tar
[DEBUG] [2022-01-12 11:13:47,374 +0100[ - DockerUtils - filterTarFiles - START - 1
[INFO] [2022-01-12 11:13:47,374 +0100[ - Filtering docker images list by includes and excludes lists
[DEBUG] [2022-01-12 11:13:47,374 +0100[ - DockerUtils - filterTarFiles - END - 1
[DEBUG] [2022-01-12 11:13:47,394 +0100[ - TarDiscoverer - buildEntitiesFromFiles - START - 1 files
[INFO] [2022-01-12 11:13:47,394 +0100[ - file 1: /Users/xxxx/Projects/test/image.tar
[DEBUG] [2022-01-12 11:13:47,395 +0100[ - TarDiscoverer - isOCILayOut - START - scan image.tar
[DEBUG] [2022-01-12 11:13:47,406 +0100[ - TarDiscoverer - isOCILayOut - blobs:false index:false ocilayout:false
[DEBUG] [2022-01-12 11:13:47,407 +0100[ - TarDiscoverer - isOCILayOut - END - result:false
[DEBUG] [2022-01-12 11:13:47,409 +0100[ - TarDiscoverer - buildEntitiesFromFiles - END - 1 entries
[INFO] [2022-01-12 11:13:47,409 +0100[ - Handle 1 docker images
[INFO] [2022-01-12 11:13:47,410 +0100[ - Image 1 of 1
[INFO] [2022-01-12 11:13:47,410 +0100[ -
------------------------------------------------------------------------
-------------------- Start: Docker Tar File ----------------------------
------------------------------------------------------------------------
[INFO] [2022-01-12 11:13:47,781 +0100[ - Extracting file /private/var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/image.tar - Size 489350656 Bytes (466 MBs)- Free Space 777072738304 Bytes (741074 MBs)
[DEBUG] [2022-01-12 11:13:47,782 +0100[ - AbstractLayerScanner - scan - START - image.tar - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/image.tar
[DEBUG] [2022-01-12 11:13:47,782 +0100[ - DockerEntityExtraction - extractTarFile - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/image.tar
[DEBUG] [2022-01-12 11:13:48,849 +0100[ - DockerEntityExtraction - extractTarFile - END - true
[DEBUG] [2022-01-12 11:13:48,849 +0100[ - DockerImageScanner - scanLayers - START - Scanning image 'image.tar'
[DEBUG] [2022-01-12 11:13:48,849 +0100[ - DockerUtils - getImageManifest - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347
[DEBUG] [2022-01-12 11:13:48,896 +0100[ - DockerUtils - getImageManifest - END - true
[DEBUG] [2022-01-12 11:13:48,896 +0100[ - get image layers file names from image manifest file in '/var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347'
[DEBUG] [2022-01-12 11:13:48,896 +0100[ - DockerUtils - getDockerImageInfo - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347
[WARN] [2022-01-12 11:13:48,897 +0100[ - failed to parse docker config file /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/sha256:7ae0d930f715a751ad98e0c457b2989aceb1670d218f7239bce05a55fe82262f
[DEBUG] [2022-01-12 11:13:48,897 +0100[ - DockerUtils - getDockerImageInfo - END - false
[ERROR] [2022-01-12 11:13:48,897 +0100[ - failed to parse docker image metadata
[DEBUG] [2022-01-12 11:13:48,897 +0100[ - DockerImageScanner - scanLayers - END - failed to parse docker image metadata
[DEBUG] [2022-01-12 11:13:48,897 +0100[ - AbstractLayerScanner - identifyBaseImage - START - image.tar
[DEBUG] [2022-01-12 11:13:48,897 +0100[ - AbstractLayerScanner - identifyBaseImage - END - 0
[DEBUG] [2022-01-12 11:13:48,897 +0100[ - AbstractLayerScanner - getDependencyInfosFromLayers - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347 6
[INFO] [2022-01-12 11:13:48,903 +0100[ - AbstractLayerScanner - scanLayersInManifest - START
[DEBUG] [2022-01-12 11:13:48,903 +0100[ - DockerEntityExtraction - extractTarFile - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/8b8a142162d22658bdf0283afcd00a9dd216c6637943ffe5f2ba53c4e3da0bd9.tar.gz
[DEBUG] [2022-01-12 11:13:48,904 +0100[ - ArchiveExtractor - getArchiveInputStream - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/8b8a142162d22658bdf0283afcd00a9dd216c6637943ffe5f2ba53c4e3da0bd9.tar.gz null
[WARN] [2022-01-12 11:13:48,904 +0100[ - Error extracting file 8b8a142162d22658bdf0283afcd00a9dd216c6637943ffe5f2ba53c4e3da0bd9.tar.gz: Cannot invoke "String.hashCode()" because "<local4>" is null
[DEBUG] [2022-01-12 11:13:48,904 +0100[ - DockerEntityExtraction - extractTarFile - END - false
[INFO] [2022-01-12 11:13:48,904 +0100[ - Did not extract file 8b8a142162d22658bdf0283afcd00a9dd216c6637943ffe5f2ba53c4e3da0bd9.tar.gz
[DEBUG] [2022-01-12 11:13:48,904 +0100[ - AbstractLayerScanner - scanLayersInManifest - failed to extract file 8b8a142162d22658bdf0283afcd00a9dd216c6637943ffe5f2ba53c4e3da0bd9.tar.gz
[DEBUG] [2022-01-12 11:13:48,904 +0100[ - DockerEntityExtraction - extractTarFile - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/a9bd5901c37d1c230822f6597ed14ead62137e086e71600be02ddea7803634e7.tar.gz
[DEBUG] [2022-01-12 11:13:48,905 +0100[ - ArchiveExtractor - getArchiveInputStream - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/a9bd5901c37d1c230822f6597ed14ead62137e086e71600be02ddea7803634e7.tar.gz null
[WARN] [2022-01-12 11:13:48,905 +0100[ - Error extracting file a9bd5901c37d1c230822f6597ed14ead62137e086e71600be02ddea7803634e7.tar.gz: Cannot invoke "String.hashCode()" because "<local4>" is null
[DEBUG] [2022-01-12 11:13:48,905 +0100[ - DockerEntityExtraction - extractTarFile - END - false
[INFO] [2022-01-12 11:13:48,905 +0100[ - Did not extract file a9bd5901c37d1c230822f6597ed14ead62137e086e71600be02ddea7803634e7.tar.gz
[DEBUG] [2022-01-12 11:13:48,905 +0100[ - AbstractLayerScanner - scanLayersInManifest - failed to extract file a9bd5901c37d1c230822f6597ed14ead62137e086e71600be02ddea7803634e7.tar.gz
[DEBUG] [2022-01-12 11:13:48,905 +0100[ - DockerEntityExtraction - extractTarFile - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/bbb11f9128797ca9d6b7d6bd9538a3753d52e6db9957f459c3ccf4f752b2ba08.tar.gz
[DEBUG] [2022-01-12 11:13:48,906 +0100[ - ArchiveExtractor - getArchiveInputStream - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/bbb11f9128797ca9d6b7d6bd9538a3753d52e6db9957f459c3ccf4f752b2ba08.tar.gz null
[WARN] [2022-01-12 11:13:48,906 +0100[ - Error extracting file bbb11f9128797ca9d6b7d6bd9538a3753d52e6db9957f459c3ccf4f752b2ba08.tar.gz: Cannot invoke "String.hashCode()" because "<local4>" is null
[DEBUG] [2022-01-12 11:13:48,906 +0100[ - DockerEntityExtraction - extractTarFile - END - false
[INFO] [2022-01-12 11:13:48,906 +0100[ - Did not extract file bbb11f9128797ca9d6b7d6bd9538a3753d52e6db9957f459c3ccf4f752b2ba08.tar.gz
[DEBUG] [2022-01-12 11:13:48,906 +0100[ - AbstractLayerScanner - scanLayersInManifest - failed to extract file bbb11f9128797ca9d6b7d6bd9538a3753d52e6db9957f459c3ccf4f752b2ba08.tar.gz
[DEBUG] [2022-01-12 11:13:48,906 +0100[ - DockerEntityExtraction - extractTarFile - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/ffe985fdf6a85e0e6682bacb597bf3aed6a2cab08403ad556f75c7cafdce874c.tar.gz
[DEBUG] [2022-01-12 11:13:48,923 +0100[ - ArchiveExtractor - getArchiveInputStream - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/ffe985fdf6a85e0e6682bacb597bf3aed6a2cab08403ad556f75c7cafdce874c.tar.gz null
[WARN] [2022-01-12 11:13:48,923 +0100[ - Error extracting file ffe985fdf6a85e0e6682bacb597bf3aed6a2cab08403ad556f75c7cafdce874c.tar.gz: Cannot invoke "String.hashCode()" because "<local4>" is null
[DEBUG] [2022-01-12 11:13:48,923 +0100[ - DockerEntityExtraction - extractTarFile - END - false
[INFO] [2022-01-12 11:13:48,924 +0100[ - Did not extract file ffe985fdf6a85e0e6682bacb597bf3aed6a2cab08403ad556f75c7cafdce874c.tar.gz
[DEBUG] [2022-01-12 11:13:48,924 +0100[ - AbstractLayerScanner - scanLayersInManifest - failed to extract file ffe985fdf6a85e0e6682bacb597bf3aed6a2cab08403ad556f75c7cafdce874c.tar.gz
[DEBUG] [2022-01-12 11:13:48,924 +0100[ - DockerEntityExtraction - extractTarFile - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/d4da8abc34e9a2cecf0517d0c31715cf9d26101ec76ab8ebdb13dbae2e5a02ad.tar.gz
[DEBUG] [2022-01-12 11:13:48,925 +0100[ - ArchiveExtractor - getArchiveInputStream - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/d4da8abc34e9a2cecf0517d0c31715cf9d26101ec76ab8ebdb13dbae2e5a02ad.tar.gz null
[WARN] [2022-01-12 11:13:48,925 +0100[ - Error extracting file d4da8abc34e9a2cecf0517d0c31715cf9d26101ec76ab8ebdb13dbae2e5a02ad.tar.gz: Cannot invoke "String.hashCode()" because "<local4>" is null
[DEBUG] [2022-01-12 11:13:48,925 +0100[ - DockerEntityExtraction - extractTarFile - END - false
[INFO] [2022-01-12 11:13:48,925 +0100[ - Did not extract file d4da8abc34e9a2cecf0517d0c31715cf9d26101ec76ab8ebdb13dbae2e5a02ad.tar.gz
[DEBUG] [2022-01-12 11:13:48,925 +0100[ - AbstractLayerScanner - scanLayersInManifest - failed to extract file d4da8abc34e9a2cecf0517d0c31715cf9d26101ec76ab8ebdb13dbae2e5a02ad.tar.gz
[DEBUG] [2022-01-12 11:13:48,925 +0100[ - DockerEntityExtraction - extractTarFile - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/b39dad574c8e281b2b0d988cccbf1efb110a42c7695d2fdc8a721130b7f88216.tar.gz
[DEBUG] [2022-01-12 11:13:48,926 +0100[ - ArchiveExtractor - getArchiveInputStream - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/b39dad574c8e281b2b0d988cccbf1efb110a42c7695d2fdc8a721130b7f88216.tar.gz null
[WARN] [2022-01-12 11:13:48,926 +0100[ - Error extracting file b39dad574c8e281b2b0d988cccbf1efb110a42c7695d2fdc8a721130b7f88216.tar.gz: Cannot invoke "String.hashCode()" because "<local4>" is null
[DEBUG] [2022-01-12 11:13:48,926 +0100[ - DockerEntityExtraction - extractTarFile - END - false
[INFO] [2022-01-12 11:13:48,926 +0100[ - Did not extract file b39dad574c8e281b2b0d988cccbf1efb110a42c7695d2fdc8a721130b7f88216.tar.gz
[DEBUG] [2022-01-12 11:13:48,926 +0100[ - AbstractLayerScanner - scanLayersInManifest - failed to extract file b39dad574c8e281b2b0d988cccbf1efb110a42c7695d2fdc8a721130b7f88216.tar.gz
[INFO] [2022-01-12 11:13:48,926 +0100[ - AbstractLayerScanner - scanLayersInManifest - END
[DEBUG] [2022-01-12 11:13:48,926 +0100[ - DockerImageSystemPackagesManager - cleanResultFrom - No deleted files were detected
[DEBUG] [2022-01-12 11:13:48,927 +0100[ - AbstractLayerScanner - getDependencyInfosFromLayers - END
[DEBUG] [2022-01-12 11:13:48,928 +0100[ - DockerImageScanner - scanLayers - END - Found 0 system packages
[DEBUG] [2022-01-12 11:13:49,036 +0100[ - AbstractLayerScanner - scan - END - 0 dependencies found in /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/image.tar
[INFO] [2022-01-12 11:13:49,037 +0100[ -
------------------------------------------------------------------------
-------------------- End: Docker Tar File ------------------------------
------------------------------------------------------------------------
[INFO] [2022-01-12 11:13:49,039 +0100[ -
------------------------------------------------------------------------
-------------------- End: Docker Resolver Scan -------------------------
------------------------------------------------------------------------
To me it looks like the difference is that the layers in the "legacy' format are only .tar files, while in the new format the layers are compressed as gzip so .tar.gz files.
docker load does work with the ".tar.gz" layers.
Add Bazel to the Dockerized Unifed-Agent
Add Bazel to the Dockerized Unifed-Agent.
Current release 21.8.1 is not working properly - Golang
I´m using the latest wss-unified-agent.jar, which is currently 21.8.1. However, since 21.8.1 was released Whitesource is not finding dependencies anymore. If I use the tag v21.7.2 instead, the Whitesource scan runs as before and is finding dependencies in the code again.
whitesource - Effective Usage Analysis - unifiedagent execution facing issue
While executing python code whitesource scan facing below issue.
22:51:49 -------------------- Start: Effective Usage Analysis -------------------
22:51:49 ------------------------------------------------------------------------
22:51:49 [ERROR] [2022-10-05 17:21:48,920 +0000] - Effective Usage Analysis will not run if 0 dependencies are found. Check that the -d parameter specifies a valid project path
22:51:49 make: *** [whitesource-python] Error 255
do i want to add any param in wss-unified-agent.config file ?
Publish an updated version on a Maven repo
As is almost needless to say, projects wishing to use a tool would have a better time integrating if the tool were available on a Maven repository.
The latest version of wss-unified-agent available on a Maven repository is 19.7.3, published in 2019:
https://mvnrepository.com/artifact/org.whitesource/wss-unified-agent/19.7.3
This version is actually missing 3/4 of the contents of the jar, rendering it unusable. The current version available on this repo, though, does work, and should probably be published to the Maven repo, Otherwise, it's inconvenient for projects to actually get the jar and use it.
Unified Agent does not take -projectName into account
We're trying to scan a docker image and upload the results to Whitesource.
The docker image is present as a TAR file and has a generic name, i.e. image.tar.
When running the unified agent we use the parameter -projectName to target a specific project.
However, the project name is set to the name of the file without the extension, i.e. image.
So, it results in a project named image being created in the Whitesource user interface.
The same happens if we manually create the project via the UI and then pass the projectToken via the -projectToken parameter.
In effect, we cannot scan our docker images.
Example: Scan ingress-nginx-controller
We run the following command:
java -jar /wss-unified-agent.jar -d . -c wss_configs/wss_config_defaultbackend.cfg -project ingress-nginx-controller -product 'OUR PRODUCT NAME' -productVersion 1 -wss.url
https://saas.whitesourcesoftware.com/agent -apiKey ((redacted)) -userKey ((redacted))
We would expect a project of name "ingress-nginx-controller" to be created, but this does not happen. Instead the project "image" is created.
Below you will find our config file with for the first case (use projectName instead of projectToken):
[INFO] [2021-07-14 12:15:10,647 +0000] - UnifiedAgent version (pluginVersion) : 21.6.2.2
[INFO] [2021-07-14 12:15:10,674 +0000] -
apiKey=******
archiveExtractionDepth=10
case.sensitive.glob=false
checkPolicies=true
configFilePath=wss_configs/wss_config_defaultbackend.cfg
docker.includes=.*.tar
docker.scanImages=true
docker.scanTarFiles=true
failErrorLevel=ALL
fileSystemScan=true
followSymbolicLinks=true
forceCheckAllDependencies=true
forceUpdate.failBuildOnPolicyViolation=false
forceUpdate=true
generateProjectDetailsJson=true
generateScanReport=true
includes=**
offline=false
productName=OUR PRODUCT NAME
productVersion=1
projectName=ingress-nginx-controller
projectTag=
python.resolveGlobalPackages=true
resolveAllDependencies=false
updateType=OVERRIDE
userKey=******
wss.url=https://saas.whitesourcesoftware.com/agent
[INFO] [2021-07-14 12:15:50,845 +0000] - Newly created projects:
[INFO] [2021-07-14 12:15:50,845 +0000] - # image - 1
[INFO] [2021-07-14 12:15:50,845 +0000] - No projects were updated.
[INFO] [2021-07-14 12:15:50,845 +0000] - Project name: image - 1, URL: https://saas.whitesourcesoftware.com/Wss/WSS.html#!project;id=...
[INFO] [2021-07-14 12:15:50,845 +0000] - Support Token: ...
[INFO] [2021-07-14 12:15:50,847 +0000] - generating scan report
[INFO] [2021-07-14 12:16:08,539 +0000] - scan report created successfully at /tmp/build/e55deab7/repository/./whitesource/image - 1-2021-07-14T121608+0000-scan_report.json
[INFO] [2021-07-14 12:16:08,542 +0000] - Log files are found at: ./whitesource/Wed-Jul-14-2021-12.15.10
[INFO] [2021-07-14 12:16:08,548 +0000] -
------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------- WhiteSource Scan Summary: --------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------------------------------------
======================================================================================================================================================
Scan Origin: Local Docker Images
======================================================================================================================================================
Step Completion Status Elapsed Comments
======================================================================================================================================================
Fetch Configuration COMPLETED 00:00:00.238 --------
Docker Resolver Scan COMPLETED 00:00:18.484 --------
tar-name COMPLETED 00:00:18.484 image.tar
Scan Files Matching Includes Pattern COMPLETED 00:00:00.896 3753 source/binary files
post-upgrade COMPLETED -------- 11 source/binary files
11 COMPLETED -------- 1 source/binary files
COMPLETED -------- 1703 source/binary files
tar COMPLETED -------- 4 source/binary files
crt COMPLETED -------- 140 source/binary files
dist COMPLETED -------- 4 source/binary files
conf COMPLETED -------- 50 source/binary files
trigger COMPLETED -------- 7 source/binary files
post-install COMPLETED -------- 11 source/binary files
script COMPLETED -------- 2 source/binary files
0 COMPLETED -------- 151 source/binary files
1 COMPLETED -------- 25 source/binary files
pre-install COMPLETED -------- 4 source/binary files
vfat COMPLETED -------- 1 source/binary files
sh COMPLETED -------- 4 source/binary files
pem COMPLETED -------- 140 source/binary files
cnf COMPLETED -------- 4 source/binary files
so COMPLETED -------- 25 source/binary files
pl COMPLETED -------- 9 source/binary files
pub COMPLETED -------- 22 source/binary files
pre-upgrade COMPLETED -------- 4 source/binary files
Alpine COMPLETED -------- 40 packages
Check Policies COMPLETED 00:00:10.548 --------
Update Inventory COMPLETED 00:00:11.030 0 updated projects
======================================================================================================================================================
Elapsed running time: 00:00:40.300
WhiteSource Unified Agent is taking extraordinarily long time to complete or eventually timeout in maven scan.
Hi,
I am trying to do a scan for Java-Maven based project but its getting timeout all the time.
The issue is discussed here:
https://whitesource.atlassian.net/wiki/spaces/WSK/pages/2481225794/Unsupported+Maven+Dependency+Plugin+versions+3.0.x+3.1.x+3.2.x+Workaround
But both the workarounds are not working for us. I am still facing the same issue and not able to complete the scan.
Is there any fix being released in the upcoming days?
The scan is working fine on my local but getting failed on the Jenkins machine.
Possible to sign or release a checksum alongside release?
Hello - it would be beneficial for us to verify the JAR after downloading it in our pipelines. Could you release a checksum or (even better) sign the JAR and let us verify it with a public key? That way we could verify it after downloading in a pipeline but before executing it. Thanks
wss-unified-agent tries to resolve mvn dependencies in gradle-only project and fails
I have a gradle Project.
But the wss-unified-agent apparently finds maven dependencies and tries to resolve them by running maven.
But mvn is not installed on the machine.
[DEBUG] [2022-05-25 07:10:22,253 +0200] - GradleDependencyResolver - resolveDependencies - END - 3
[INFO] [2022-05-25 07:10:22,260 +0200] - Trying to resolve MAVEN dependencies
[DEBUG] [2022-05-25 07:10:22,260 +0200] - Command - executeProcess - timeout set to 60s
[DEBUG] [2022-05-25 07:10:22,261 +0200] - Command - executeProcess - start execute command 'mvn -v' in '.'
[WARN] [2022-05-25 07:10:22,261 +0200] - Command - executeProcess - Command 'mvn -v' run Failed, Exception: Cannot run program "mvn" (in directory "."): error=2, No such file or directory
[DEBUG] [2022-05-25 07:10:22,261 +0200] - MAVEN installation status: false
[DEBUG] [2022-05-25 07:10:22,262 +0200] - The command was run under [..]/product/.
[INFO] [2022-05-25 07:10:22,262 +0200] - topFolder = [..]/org.jacoco.agent-0.8.7.jar_3a83c50b4a016f281c4e9f3500d16b55/META-INF/maven/org.jacoco/org.jacoco.agent
[DEBUG] [2022-05-25 07:10:22,262 +0200] - topFolder = [..]/build/tmp/expandedArchives/org.jacoco.agent-0.8.7.jar_3a83c50b4a016f281c4e9f3500d16b55/META-INF/maven/org.jacoco/org.jacoco.agent
[DEBUG] [2022-05-25 07:10:22,262 +0200] - Command - executeProcess - timeout set to 900s
[DEBUG] [2022-05-25 07:10:22,262 +0200] - Command - executeProcess - start execute command 'mvn -v' in '[..]/build/tmp/expandedArchives/org.jacoco.agent-0.8.7.jar_3a83c50b4a016f281c4e9f3500d16b55/META-INF/maven/org.jacoco/org.jacoco.agent'
[WARN] [2022-05-25 07:10:22,263 +0200] - Command - executeProcess - Command 'mvn -v' run Failed, Exception: Cannot run program "mvn" (in directory "[..]/build/tmp/expandedArchives/org.jacoco.agent-0.8.7.jar_3a83c50b4a016f281c4e9f3500d16b55/META-INF/maven/org.jacoco/org.jacoco.agent"): error=2, No such file or directory
[DEBUG] [2022-05-25 07:10:22,263 +0200] - Failed to get maven version
[WARN] [2022-05-25 07:10:22,263 +0200] - Please install maven
...
[ERROR] [2022-05-25 07:10:22,269 +0200] - Process finished with exit code ERROR (-1)
Exception on version parse
I'm getting exception errors from the unified agent
-------------------- Start: Pre-Step And Resolve Dependencies ----------
------------------------------------------------------------------------
[INFO] [2022-06-28 14:05:35,944 +0000] - Trying to resolve POETRY dependencies
[WARN] [2022-06-28 14:07:05,827 +0000] - Couldn't parse line kryon-config-core 2022.6.1000000001, error: class java.lang.StringIndexOutOfBoundsException - begin 0, end -1, length 17
[WARN] [2022-06-28 14:07:05,827 +0000] - Couldn't parse line kryon-logger 2022.6.1000000001, error: class java.lang.StringIndexOutOfBoundsException - begin 0, end -1, length 12
Please support this type of versioning, thanks
Specify tmp directory for Docker Image scan
For scanning large Docker images, we must change the temporary directory as its space is limited and our scan therefore fails. Documentation states that "[t]he Docker image is saved to the temporary directory defined in your environment and is deleted immediately after the scan." -> so how can we change this said "temporary directory" or can it only use "/tmp"?
Scanning distroless docker images: not able to extract layer.tar
I'm trying to increase security by using distroless image for my containers, using this image from google as base image https://github.com/GoogleContainerTools/distroless/tree/master/java
When I'm trying to scan it locally with the command
java -jar wss-unified-agent.jar -apiKey $API_KEY -wss.url $URL -c whitesource-docker.properties -project mytestproject -product mytestproduct
I receive some warns in the log and 0 system packages. As result, no new projects created
[INFO] [2020-02-21 18:09:01,051 +0700] - Extracting file /tmp/WhiteSource-Docker_507e746a-ee9f-4158-b2c5-22d13f4da998/gcr.io-distroless-java.tar - Size 197453824 Bytes (188 MBs)- Free Space 227714412544 Bytes (217165 MBs)
[WARN] [2020-02-21 18:09:01,589 +0700] - Error extracting file layer.tar: /tmp/WhiteSource-Docker_507e746a-ee9f-4158-b2c5-22d13f4da998/gcr.io-distroless-java/7bff0f034dc797b09394b6136265054e798556abd7438fa31417f04302f23db7/./usr/share/doc/ca-certificates/copyright (No such file or directory)
[WARN] [2020-02-21 18:09:01,590 +0700] - Was not able to extract layer.tar (docker image TAR file)
[INFO] [2020-02-21 18:09:03,711 +0700] - Found 0 system packages in image 'gcr.io/distroless/java'
In my whitesource-docker.properties I have this option
docker.includes=my_distroless gcr.io/distroless/java
Is it expected behavior?
Correct metadata in the wss-unified-agent jar file
The wss-unified-agent jar file's META-INF/MANIFEST.MF is missing the version number normally expected to be in jar files.
As best I can tell, the current version is actually 21.1.2.1? Based on the maven POM file for wss-unified-agent-main found in the same jar file.
Whitesource documentation for security alerts is unclear
Hey,
the Whitesource documentation regarding "Alerts" is unclear.
https://whitesource.atlassian.net/wiki/spaces/WD/pages/809894145/Managing+Alerts
It states:
Alerts work the following way: Upon scan completion, a customer’s inventory is synchronized to WhiteSource, and the application analyzes the customer’s open-source libraries and source files and compares them to the WhiteSource knowledge base and policy definitions. If security vulnerabilities, licensing and compatibility issues, or policy violations, etc. exist, alerts are triggered for the organization.
In particular, the term "Upon scan completion" suggests that the inventory is only analyzed once upon scan completion.
However, we face the situation that we have an alert in "Security Alerts: View By Vulnerability" which was created on the 14th of August, but the scan was triggered on the 13th.
I see two options for the process:
- The inventory is analyzed regularly, e.g. daily, after the scan results have been uploaded initially.
- There is a delay of one day between the scan and the update/creation of the alerts.
Could you please clarify the process?
Thanks a lot in advance.
(In case Github issues are not the best way to talk about the documentation, please let us know!)
Rollback to previous versions
As the https://github.com/whitesource/unified-agent-distribution/blob/master/standAlone/wss_agent.sh file always points to the latest release and master config, if there is an issue with the latest release (e.g #18) pointing at the previous tag won't result in downgrading the agent.
If I download the agent script from release x.y.z, I would expect to run the x.y.z version of the agent
nuget.resolveAssetsFiles discovers internal dependencies
There is not intended behavior with nuget.resolveAssetsFiles=true configuration parameter.
It scans .\obj\project.assets.json and if any entry listed there is considered by scanner as third-party dependency while some of those are if fact internal dependencies on other projects.
the assets file should be scanned with filter on type, so only "type": "package" is listed and "type": "project" is ignored
Github Action
Hi all
Do you plan to support an official setup-whitesource-unified-agent (or similar) action? I can make a contribution but not if you plan to do it within 7 days 🤣
Also, there is https://github.com/TheAxZim/Whitesource-Scan-Action which is nice but does not support all CLI arguments.
Any insight here?
Error message while running whitesource on azure devops pipelines using azure-vmss-agents
Error message while running whitesource on azure devops pipelines using azure-vmss-agents
Error sending request:
"reason": "Host: app-eu.whitesourcesoftware.com. is not in the cert's altnames: DNS:.whitesourcesoftware.com, DNS:whitesourcesoftware.com",
"host": "app-eu.whitesourcesoftware.com",
"cert": {
"subject": {
"CN": ".whitesourcesoftware.com"
},
"issuer": {
"C": "US",
"ST": "Arizona",
"L": "Scottsdale",
"O": "GoDaddy.com, Inc.",
"OU": "http://certs.godaddy.com/repository/",
"CN": "Go Daddy Secure Certificate Authority - G2"
},
"subjectaltname": "DNS:*.whitesourcesoftware.com, DNS:whitesourcesoftware.com",
"infoAccess": {
"OCSP - URI": [
"http://ocsp.godaddy.com/"
],
"CA Issuers - URI": [
"http://certificates.godaddy.com/repository/gdig2.crt"
]
}
Sending check policies request to WhiteSource server failes with above error message
Java/Gradle semantic analysis does not fail when gradle error is encountered
When scanner hits a jar file compiled against newer version of Java, it just throws an exception and continues as regular:
[INFO] [2024-02-23 19:30:50,803 +0000] - Trying to resolve GRADLE dependencies
[INFO] [2024-02-23 19:30:53,893 +0000] - topFolder = /workspace/cloned-git-repository
[WARN] [2024-02-23 19:31:08,657 +0000] - Command - executeProcess - error in execute command '/workspace/cloned-git-repository/gradlew --init-script /tmp/blablabla/init.ws.gradle whitesourceDependenciesTask -DWS_INIT_GRADLE_OUTPUT_PATH=/tmp/blablabla/gradle_XNAMTL/20240223193053/gradle_dependencies -Dorg.gradle.parallel= -DWS_INIT_GRADLE_INCLUDE_DEPENDENCIES=true', Exit Status 1
[WARN] [2024-02-23 19:31:08,658 +0000] - Read error line #1: FAILURE: Build failed with an exception.
[WARN] [2024-02-23 19:31:08,658 +0000] - Read error line #2: * Where:
[WARN] [2024-02-23 19:31:08,658 +0000] - Read error line #3: Initialization script '/tmp/blablabla/gradle_XNAMTL/20240223193053/init.ws.gradle'
[WARN] [2024-02-23 19:31:08,658 +0000] - Read error line #4: * What went wrong:
[WARN] [2024-02-23 19:31:08,658 +0000] - Read error line #5: Could not compile initialization script '/tmp/blablabla/gradle_XNAMTL/20240223193053/init.ws.gradle'.
[WARN] [2024-02-23 19:31:08,658 +0000] - Read error line #6: > startup failed:
[WARN] [2024-02-23 19:31:08,658 +0000] - Read error line #7: General error during semantic analysis: Unsupported class file major version 61
[WARN] [2024-02-23 19:31:08,658 +0000] - Read error line #8: java.lang.IllegalArgumentException: Unsupported class file major version 61
[WARN] [2024-02-23 19:31:08,658 +0000] - Read error line #9: at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:196)
...
------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------- WhiteSource Scan Summary: --------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------------------------------------
======================================================================================================================================================
Scan Origin: Local File System
======================================================================================================================================================
Step Completion Status Elapsed Comments
======================================================================================================================================================
Fetch Configuration COMPLETED 00:00:00.391 --------
Pre-Step And Resolve Dependencies COMPLETED 00:02:07.832 8 total dependencies (3 unique)
MAVEN COMPLETED 00:00:28.334 0 dependencies
GRADLE **COMPLETED** 00:01:38.458 0 dependencies
....
Is this a correct behavior? The scan reports 0 gradle dependencies found/scanned and just continues, while it could abort the analysis and report the offending class name
Of course we can see this in logs, but we don't usually look there if the scanner says it's fine
Docker image with minimal footprint
Can you provide a docker image with minimal footprint that includes just the real prereqs and the lib (and not full OS libs or other package managers)? People can derive from that and add their addtional libs of their need.
You can save additional space when you build your own java disctribution with jlink (e.g. described here https://www.baeldung.com/jlink).
Slow npm dependency resolution since 21.6.3
With the 21.6.2.2 release of the Unified Agent, our WhiteSource scan of the npm dependencies takes ~20 seconds:
[2021-08-16T14:55:23.984Z] [INFO] [2021-08-16 14:55:23,873 +0000] - Trying to resolve NPM dependencies
[2021-08-16T14:55:24.643Z] [INFO] [2021-08-16 14:55:24,545 +0000] - topFolder = /****/client
[2021-08-16T14:55:47.108Z] [INFO] [2021-08-16 14:55:46,552 +0000] - Trying to resolve NUGET dependencies
However, from 21.6.3 onward (and just tested against 21.7.2), this same step, scanning the same source code with the same configuration file, takes 20+ minutes.
Did 21.6.3 introduce a behavior that can be opted out of via config, or is this simply a bug?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.