Comments (9)
chadlwilson
commented on June 15, 2024
Looks like the JVM the agent is using is screwed up. I don't think step 4 is a safe way to do things, there are native libs as well as binaries and normally you want to swap out the whole install.
While you could replace the whole jre folder, n one of the wrapper properties files there is a var you can override to point to the JVM install location, to change to point to the one you installed at /opt/jdk-17. Being a bit vague as away from a machine at the moment.
Either JDK 11 or 17 should work fine, for what it's worth, as long as the agent is pointed to its install location correctly.
from gocd.
chadlwilson
commented on June 15, 2024
#11354 has the proper property to change I believe
from gocd.
resworld
commented on June 15, 2024
Hi @chadlwilson
Thank you for fast reply. I've tried with adding:
wrapper.java.command=/opt/jdk-17/bin/java
but still facing same problem
from gocd.
chadlwilson
commented on June 15, 2024
What happens if the system Java version is the same as the one you are trying to use?
I would look carefully at the various logs and/or forked processes across the agent processes (bootstrapper, launcher etc) to make sure it is launching with the JDK version you think it should be.
from gocd.
resworld
commented on June 15, 2024
I've tried this on new instance:
$ yum install https://download.oracle.com/java/17/latest/jdk-17_linux-aarch64_bin.rpm -y
$ java --version
java 17.0.7 2023-04-18 LTS
Java(TM) SE Runtime Environment (build 17.0.7+8-LTS-224)
Java HotSpot(TM) 64-Bit Server VM (build 17.0.7+8-LTS-224, mixed mode, sharing)
$ curl https://download.gocd.org/gocd.repo -o /etc/yum.repos.d/gocd.repo && yum install go-agent -y
$ cat > /usr/share/go-agent/wrapper-config/wrapper-properties.conf <<EOF
wrapper.app.parameter.100=-serverUrl
wrapper.app.parameter.101=http://<go-server-ip-addr>:8153/go
wrapper.java.command=/usr/lib/jvm/jdk-17-oracle-aarch64/bin/java
EOF
$ systemctl start go-agent
Now I am having just java 17 on the system and the error is same:
==> /var/log/go-agent/go-agent-stderr.log <==
java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at com.thoughtworks.gocd.Boot.run(Boot.java:90)
at com.thoughtworks.gocd.Boot.main(Boot.java:56)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentController' defined in class path resource [applicationContext.xml]: Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1619)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:756)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:542)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:83)
at com.thoughtworks.go.agent.AgentMain.main(AgentMain.java:43)
... 6 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
at com.sun.jna.NativeLong.<clinit>(NativeLong.java:35)
at oshi.util.FileUtil.readNativeLongFromBuffer(FileUtil.java:208)
at oshi.driver.linux.proc.Auxv.queryAuxv(Auxv.java:39)
at oshi.software.os.linux.LinuxOperatingSystem.<clinit>(LinuxOperatingSystem.java:87)
at oshi.SystemInfo.createOperatingSystem(SystemInfo.java:85)
at oshi.util.Memoizer$1.get(Memoizer.java:61)
at oshi.SystemInfo.getOperatingSystem(SystemInfo.java:76)
at com.thoughtworks.go.agent.service.SystemInfo.determineOperatingSystemCompleteName(SystemInfo.java:36)
at com.thoughtworks.go.agent.service.SystemInfo.<clinit>(SystemInfo.java:28)
at com.thoughtworks.go.server.service.AgentRuntimeInfo.refreshOperatingSystem(AgentRuntimeInfo.java:200)
at com.thoughtworks.go.server.service.AgentRuntimeInfo.fromAgent(AgentRuntimeInfo.java:73)
at com.thoughtworks.go.agent.AgentController.initRuntimeInfo(AgentController.java:167)
at com.thoughtworks.go.agent.AgentController.init(AgentController.java:135)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1748)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1685)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1615)
... 18 more
Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.UnsatisfiedLinkError: /tmp/jna434117078481342640.tmp: /tmp/jna434117078481342640.tmp: failed to map segment from shared object: Operation not permitted [in thread "main"]
at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:388)
at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:232)
at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:174)
at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2389)
at java.base/java.lang.Runtime.load0(Runtime.java:755)
at java.base/java.lang.System.load(System.java:1953)
at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1045)
at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:1015)
at com.sun.jna.Native.<clinit>(Native.java:221)
at com.sun.jna.platform.linux.Udev.<clinit>(Udev.java:37)
at oshi.software.os.linux.LinuxOperatingSystem.<clinit>(LinuxOperatingSystem.java:74)
... 34 more
And I will add other log files, as maybe there's something useful that I am missing to mention:
==> /var/log/go-agent/go-agent-stderr.log <==
[Thu May 11 13:30:41 UTC 2023] Starting process:
[Thu May 11 13:30:41 UTC 2023] Working directory : /var/lib/go-agent
[Thu May 11 13:30:41 UTC 2023] Application arguments: [-serverUrl, http://<go-server_ip_addr>:8153/go, -sslVerificationMode, FULL]
[Thu May 11 13:30:41 UTC 2023] GoCD Version: 23.1.0-16079
[Thu May 11 13:30:41 UTC 2023] Java Version: 17.0.7
[Thu May 11 13:30:41 UTC 2023] Operating System: Linux(4.18.0-348.20.1.el7.aarch64)
Could not find file `config/agent-logback.xml'. Attempting to load from classpath.
Using classpath resource `jar:onejar:lib/agent-23.1.0-16079-classes.jar!/config/agent-logback.xml'.
==> /var/log/go-agent/go-agent.log <==
2023-05-11 13:30:42,392 INFO [main] ClassPathXmlApplicationContext:583 - Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@72057ecf: startup date [Thu May 11 13:30:42 UTC 2023]; root of context hierarchy
2023-05-11 13:30:42,443 INFO [main] XmlBeanDefinitionReader:317 - Loading XML bean definitions from class path resource [applicationContext.xml]
2023-05-11 13:30:42,707 INFO [main] XmlBeanDefinitionReader:317 - Loading XML bean definitions from class path resource [applicationContext-plugin-infra.xml]
2023-05-11 13:30:43,010 INFO [main] DefaultListableBeanFactory:821 - Overriding bean definition for bean 'pluginLoader' with a different definition: replacing [Generic bean: class [com.thoughtworks.go.plugin.infra.PluginLoader]; scope=singleton; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in URL [jar:onejar:lib/go-plugin-infra-23.1.0-16079.jar!/com/thoughtworks/go/plugin/infra/PluginLoader.class]] with [Generic bean: class [com.thoughtworks.go.plugin.infra.PluginLoader]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in class path resource [applicationContext-plugin-infra.xml]]
2023-05-11 13:30:43,275 INFO [main] AutowiredAnnotationBeanPostProcessor:156 - JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
2023-05-11 13:30:43,503 INFO [main] AgentStatusHttpd:88 - Agent status HTTP API server running on http://localhost:8152.
2023-05-11 13:30:45,276 WARN [main] ClassPathXmlApplicationContext:550 - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentController' defined in class path resource [applicationContext.xml]: Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
==> /var/log/go-agent/go-agent-bootstrapper.log <==
2023-05-11 13:32:21,611 INFO [WrapperJarAppMain] AgentBootstrapper:78 - Launcher returned with code 1(0x1)
2023-05-11 13:32:22,708 INFO [WrapperJarAppMain] DefaultAgentLauncherCreatorImpl:87 - Attempt No: 1 to cleanup launcher temp files
2023-05-11 13:32:22,795 INFO [WrapperJarAppMain] AgentBootstrapper:108 - Waiting for 10000 ms before re-launch....
2023-05-11 13:32:32,912 INFO [WrapperJarAppMain] DefaultAgentLauncherCreatorImpl:67 - Attempting to load com.thoughtworks.go.agent.launcher.AgentLauncherImpl as specified by manifest key Go-Agent-Launcher-Class
2023-05-11 13:32:32,923 INFO [WrapperJarAppMain] ParentClassAccessFilteringClassloader:43 - Loading com.thoughtworks.cruise.agent.common.launcher.AgentLauncher using java.net.URLClassLoader
2023-05-11 13:32:32,947 INFO [WrapperJarAppMain] AgentBootstrapper:74 - Attempting create and start launcher...
2023-05-11 13:32:32,952 INFO [WrapperJarAppMain] ParentClassAccessFilteringClassloader:43 - Loading com.thoughtworks.cruise.agent.common.launcher.AgentLaunchDescriptor using java.net.URLClassLoader
from gocd.
chadlwilson
commented on June 15, 2024
You didn't include this relevant part of the error log earlier so I was guessing a bit. Relevant bit being Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.UnsatisfiedLinkError: /tmp/jna434117078481342640.tmp: /tmp/jna434117078481342640.tmp: failed to map segment from shared object: Operation not permitted
Are there permissions issues on your /tmp? Is your tmp mounted as noexec ? Could be similar issue as for https://www.elastic.co/guide/en/elasticsearch/reference/current/executable-jna-tmpdir.html
from gocd.
chadlwilson
commented on June 15, 2024
If that's the issue, this problem is probably not specific to arm64. There is a system property -Djna.tmpdir=/other/dir to override the directory it uses (or the whole Java temp dir), however due to the way the go agent bootstraps itself youd need to supply it in a special way which escapes my mind right now. There might be an example in the wrapper-properties.conf?
from gocd.
resworld
commented on June 15, 2024
Hi @chadlwilson,
you are right about /tmp folder, there is noexec. This error line was missing in previous tries, maybe because of two versions of Java installed on the system. I've tried to use jna.tmpdir in the wrapper-properties:
wrapper.java.additional.100=-Djna.tmpdir=/var/tmp
I can see Java is running with that option now:
go 9555 4.2 0.7 9092032 119872 ? Sl 11:26 0:05 /usr/lib/jvm/jdk-17-oracle-aarch64/bin/java -Dgocd.agent.log.dir=/var/log/go-agent -Djna.tmpdir=/var/tmp
But still facing same problem with /tmp:
Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.UnsatisfiedLinkError: /tmp/jna13366009454418608986.tmp: /tmp/jna13366009454418608986.tmp: failed to map segment from shared object: Operation not permitted [in thread "main"]
I've removed the noexec from the /tmp and now go-agent starts. It's not ideal as I want this flag to the /tmp partition, but...
Ok, once again - thank you and we can close the issue.
from gocd.
chadlwilson
commented on June 15, 2024
Yeah, because the arg needs to go to the agent process, not the bootstrapper which is where the wrapper commands go to. There are actually two Java processes spawned so the arg will be on the wrong one. We need to get the arg to propagate to the actual agent process rather than the wrapping 'bootstrapper'.
Try adding to your wrapper-properties.conf
set.AGENT_STARTUP_ARGS=%AGENT_STARTUP_ARGS% -Djna.tmpdir=/var/tmp
Will re-open this to remind myself to look at changing this to not rely on the default. Might make sense for the agent to be opinonated as it already has its own dirs for working area anyway.
from gocd.
Related Issues (20)
- Unable to create database upgrade script for database. The problem was: Could not acquire change log lock. Currently locked by gocd-app-server-64c8f9fc7f-8dhxj (10.42.0.191) since 7/2/23, 9:21 PM HOT 6
- fails test connection due gocd wanting to use old 2021 github authentication method HOT 1
- Material authentication changing to pipeline is not picking latest git credentials HOT 12
- ENV AGENT_BOOTSTRAPPER_ARGS with Docker Compose is provided as a whole String HOT 1
- Clean up old code related to PluggableTask and Package material config/preferences HOT 2
- Release GoCD 23.4.0
- Console view toolbar not shown after scrolling on v23.4.0 HOT 2
- Ability to have empty material HOT 3
- Command Execution Issue on Windows Agents in GOCD v23.3 HOT 2
- GoCD isn't parsing new config repo. No pipeliens are shown HOT 2
- Go-Agent on a different subnet HOT 1
- Unable to pick stage from Add Material menu in Chrome on MacOS Sonoma HOT 7
- pipeline artifact URL is incorrect HOT 2
- Restrict Agent Profile To Environment HOT 6
- Release GoCD 23.5.0
- Unable to move pipeline group when using pipeline filter HOT 2
- Installing chromium on alpine3.18 gocd docker agent causes crash loop HOT 3
- Can't collect artifacts from a folder containing a "%" or ";" in its name HOT 4
- HTTP -> HTTPS redirect in loadBalancer leads to error code 422 when editing cruise-config.xml in gocd-server gui with and gocd-server error InvalidAuthenticityToken HOT 8
- Unable to trigger GoCD pipeline via API HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gocd.