fedora-ci / jenkins-pipeline-library Goto Github PK
View Code? Open in Web Editor NEWJenkins Pipeline Library for Fedora CI
License: MIT License
jenkins-pipeline-library's People
Contributors
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
Stargazers
Watchers
jenkins-pipeline-library's Issues
RFE: sendMessage add support to pass an xunit file as parameter
sendMessage currently supports testingFarmResult, but some pipelines might not use testing farm, it would be nice if the function could support passing an xunit file as parameter.
Add function to get koji task_id from koji build_id
Add groovy function to get koji task id from koji build_id.
Some of the messages (koji tag for example) contain only build_id's, while CI message schema requires ARTIFACT_ID based on task_id of the build
RFE: sendMessage add support to ci messages for pull requests
The function should be able to send message for pull requests.
example of messages:
https://apps.fedoraproject.org/datagrepper/id?id=2020-01a0ec44-c2f6-4f22-9f6c-6b01f0d34c76&is_raw=true&size=extra-large
setBuildNameFromArtifactId throws exception on bad artifactId
Issue
Currently setBuildNameFromArtifactId throws exception when artifactId has wrong format or is not associated with a real koji task-id.
Expected
Meaningful error message
Suggestion
Relax the requirement: if function cannot parse artifactID to get more info, use it as is.
Add variable to the Koji library, which contains the current rawhide tag
Fedora Rawhide uses moving versioned tag. Currently it is 'f34'
The more or less reliable way to get this value is to request it from koji in a form:
kojisession.getFullInheritance('rawhide')[0]['name']
Let's add the variable to Koji module, so it can be used uniformly through all scripts and pipelines, especially in triggers.
java.io.NotSerializableException: de.timroes.axmlrpc.XMLRPCClient
trying to sendMessage after import org.fedoraproject.jenkins.koji.Koji seems to cause an exception:
Sending message for job 'dist-git-pipeline'.
Message id: 2020-e59bc1c3-d5e7-4ade-929c-358fe1a8d5fa
Message topic: org.centos.prod.ci.koji-build.test.queued
Message headers:
{}
JSON message body:
{"artifact":{"component":"ksh","scratch":true,"id":48624869,"nvr":"ksh-2020.0.0-4.fc33","type":"koji-build","issuer":"bpeck/jenkins-continuous-infra.apps.ci.centos.org"},"pipeline":{"build":"","name":"","id":"dist-git-pipeline"},"test":{"namespace":"fedora-ci.koji-build","category":"funcitonal","type":"tier0"},"generated_at":"2020-08-18T08:14:58.721Z","contact":{"docs":"https://github.com/fedora-ci/dist-git-pipeline","name":"dist-git-pipeline","team":"Fedora CI","irc":"#fedora-ci","url":"https://github.com/fedora-ci/dist-git-pipeline","email":"[email protected]"},"run":{"log":"https://osci-jenkins-2.ci.fedoraproject.org/job/dist-git-pipeline/81/console","log_stream":"https://osci-jenkins-2.ci.fedoraproject.org/job/dist-git-pipeline/81/console","url":"https://osci-jenkins-2.ci.fedoraproject.org/job/dist-git-pipeline/81/","log_raw":"https://osci-jenkins-2.ci.fedoraproject.org/job/dist-git-pipeline/81/consoleText"},"version":"0.2.1"}
Sent successfully with messageId: 2020-e59bc1c3-d5e7-4ade-929c-358fe1a8d5fa
[Pipeline] }
[Pipeline] // timeout
[Pipeline] echo
FAIL: Could not send message to RabbitMQ on topic org.centos.prod.ci.koji-build.test.queued
[Pipeline] echo
java.io.NotSerializableException: de.timroes.axmlrpc.XMLRPCClient
[Pipeline] sleep
Sleeping for 30 sec
[Pipeline] }
ERROR: Execution failed
an exception which occurred:
in field org.fedoraproject.jenkins.koji.Koji.client
in object org.fedoraproject.jenkins.koji.Koji@c169996
in field com.cloudbees.groovy.cps.impl.BlockScopeEnv.locals
in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@3042a548
in field com.cloudbees.groovy.cps.impl.CallEnv.caller
in object com.cloudbees.groovy.cps.impl.FunctionCallEnv@1b02e736
in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@6eb36d51
in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@6f0a5bd
in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@7bba1a24
in field com.cloudbees.groovy.cps.impl.CpsClosureDef.capture
in object com.cloudbees.groovy.cps.impl.CpsClosureDef@1c0cfeab
in field com.cloudbees.groovy.cps.impl.CpsClosure.def
in object org.jenkinsci.plugins.workflow.cps.CpsClosure2@6789b587
in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.closures
in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@6436ce5f
in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@6436ce5f
Caused: java.io.NotSerializableException: de.timroes.axmlrpc.XMLRPCClient
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:926)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
at java.util.HashMap.internalWriteEntries(HashMap.java:1793)
at java.util.HashMap.writeObject(HashMap.java:1363)
at sun.reflect.GeneratedMethodAccessor417.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callWriteObject(JDKSpecific.java:156)
at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:191)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1028)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
at java.util.HashMap.internalWriteEntries(HashMap.java:1793)
at java.util.HashMap.writeObject(HashMap.java:1363)
at sun.reflect.GeneratedMethodAccessor417.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callWriteObject(JDKSpecific.java:156)
at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:191)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1028)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.lambda$writeObject$0(RiverWriter.java:144)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:237)
at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:143)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:553)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:530)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgramIfPossible(CpsThreadGroup.java:517)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:441)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
pipeline.id from ci messafe field should be the same even if message is sent on different jenkins job
The current code generates id based on jenkins job name. This causes s problem if for example the queue topic is sent by a trigger jenkins job and running/complete by a different job.
Add how to test the library to README
Currently it is not clear for newcomer how to run and how to add tests to the library
Invalid task tracebacks
A nice error message would help more, see:
https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/dist-git-pipeline/job/tmt/5298/console
repoHasTests() step is doing too much
What started as a relatively simple step (repoHasTests()) turned into a quite complicated thing that is trying to do way too much.
The behavior of the step is not even fully documented -- but this is on purpose, i.e. we don't want to fix the docs.
It would be better to split the functionality into 2 or 3 separate steps:
- clone step
- extract information step(s)
This way we could clone the repository and then run any number of steps that would extract all the info we need.
Currently one of the hidden (undocumented) features of the step is that it also reads ci.fmf file and checks if a specific configuration option is there (resultsdb-testcase).
Add library step to put pipeline metadata in job description
Since most of our pipelines have pipelineMetadata map defined to comply to fedora messaging standard, let's use this metadata as a job description.
Please add 'source' entry
https://pagure.io/fedora-ci/messages/blob/master/f/schemas/rpm-build.yaml
No entry source.
It is not required, but good to have.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.