Comments (13)
jenkins-infra-bot
commented on September 24, 2024
This issue affects anyone who, for various reasons, needs to mirror the Jenkins repository, so I would like to raise awareness for it as well. As it was previously mentioned, generating the necessary metadata is just one command away.
Thank you.
from helpdesk.
jenkins-infra-bot
commented on September 24, 2024
This seems to be fixed now. Steve Koppelman, can you confirm, please?
from helpdesk.
jenkins-infra-bot
commented on September 24, 2024
http://pkg.jenkins-ci.org/redhat/repodata/repomd.xml is available
from helpdesk.
jenkins-infra-bot
commented on September 24, 2024
I've confirmed this to work for http. Is there any scope to offer via https? spacewalk-repo-sync is failing for me when trying to call --url=https://pkg.jenkins-ci.org/redhat/. The URL https://pkg.jenkins-ci.org/redhat/repodata/repomd.xml does load in a browser, though the SSL cert is not signed for pkg.jenkins-ci.org from what I can see.
from helpdesk.
jenkins-infra-bot
commented on September 24, 2024
jtyr:
I would like to reopen this issue as the repodata is not available on the mirrored sites. That makes it impossible to set the YUM repo URL to a mirror (e.g. http://jenkins.mirror.isppower.de/redhat/).
There is also disparity between the number of packages available through the repodata and what's actually available on the mirror site:
# From the repodata: $ curl -s http://pkg.jenkins-ci.org/redhat/$(curl -s http://pkg.jenkins-ci.org/redhat/repodata/repomd.xml | grep -Po '(?<=location href=").[^-]*-primary.xml.gz') | gunzip | grep 'location href=' | wc -l 343On the mirror:
$ curl -s http://mirrors.jenkins.io/redhat/ | grep '<a href="jenkins-' | wc -l
274
from helpdesk.
jenkins-infra-bot
commented on September 24, 2024
Could we get the metadata generated here as well http://mirrors.jenkins-ci.org/
the repodata folder should be publicly viewable so the dir listing should show the repodata folder and the repodata folder should have the repomd.xml file in it.
from helpdesk.
jenkins-infra-bot
commented on September 24, 2024
repodata folder exists here: http://pkg.jenkins-ci.org/redhat-stable/repodata/repomd.xml
However.... any of the old packages are REDICULOUSLY slow to download. between 3-15K/second is the max I can get.
There are 61 packages in there, and I'm on #15.. lol.
from helpdesk.
jenkins-infra-bot
commented on September 24, 2024
check out this timing...
new packages v2+ are fine, download in ~2seconds. Old stuff takes ~40 min.
02:35:40 15/61 : jenkins-1.532.3-1.1-0.noarch03:30:44 16/61 : jenkins-1.580.1-1.1-0.noarch
04:24:48 17/61 : jenkins-1.651.2-1.1-0.noarch
04:24:49 18/61 : jenkins-1.642.2-1.1-0.noarch
05:19:53 19/61 : jenkins-2.19.1-1.1-0.noarch
05:19:56 20/61 : jenkins-1.625.3-1.1-0.noarch
06:12:35 21/61 : jenkins-1.424.2-1.1-0.noarch
06:49:47 22/61 : jenkins-2.46.1-1.1-0.noarch
06:49:49 23/61 : jenkins-1.609.1-1.1-0.noarch
07:53:07 24/61 : jenkins-2.32.2-1.1-0.noarch
07:53:10 25/61 : jenkins-1.651.1-1.1-0.noarch
09:01:02 26/61 : jenkins-1.424.3-1.1-0.noarch
09:42:20 27/61 : jenkins-2.32.3-1.1-0.noarch < 2 seconds!
09:42:22 28/61 : jenkins-1.424.5-1.1-0.noarch <
10:20:09 29/61 : jenkins-1.565.1-1.1-0.noarch < 40 minutes!
11:16:18 30/61 : jenkins-1.480.1-1.1-0.noarch
11:57:15 31/61 : jenkins-2.46.2-1.1-0.noarch
11:57:17 32/61 : jenkins-1.424.1-1.1-0.noarch
12:34:59 33/61 : jenkins-1.565.2-1.1-0.noarch
13:32:13 34/61 : jenkins-1.466.2-1.1-0.noarch
14:11:40 35/61 : jenkins-1.642.3-1.1-0.noarch
15:06:09 36/61 : jenkins-2.32.1-1.1-0.noarch
15:06:11 37/61 : jenkins-2.7.3-1.1-0.noarch
15:06:12 38/61 : jenkins-1.409.2-1.1-0.noarch
15:45:32 39/61 : jenkins-1.596.1-1.1-0.noarch
16:36:30 40/61 : jenkins-1.509.2-1.1-0.noarch
17:14:16 41/61 : jenkins-1.509.1-1.1-0.noarch
from helpdesk.
jenkins-infra-bot
commented on September 24, 2024
This has been resolved, more recent requests are not about the Redhat repo.
Old stuff takes ~40 min.
That's because our mirror providers asked us to limit the disk space we're using (IIRC), so obsolete releases nobody should be running for any reason are on a machine with very limited bandwidth to not explode our credit cards (well, Tyler's).
Works as designed.
from helpdesk.
jenkins-infra-bot
commented on September 24, 2024
This is still an issue for mirroring the content to satellite. The fact that repodata folder returns a Forbidden is the most likely issue. The error in Satellite is the following:
"Traceback (most recent call last):\n File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 240, in trace_task\n R = retval = fun(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 473, in __call__\n return super(Task, self).__call__(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 103, in __call__\n return super(PulpTask, self).__call__(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 437, in __protected_call__\n return self.run(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py\", line 762, in sync\n sync_report = sync_repo(transfer_repo, conduit, call_config)\n File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 658, in wrap_f\n return f(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py\", line 81, in sync_repo\n report = self._current_sync.run()\n File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py\", line 321, in run\n reason=self.repomd_not_found_reason)\nPulpCodedException: Error retrieving metadata: Service Unavailable\n",
~#curl http://pkg.jenkins-ci.org/redhat/repodata/ <title>403 Forbidden</title>Forbidden
You don't have permission to access /redhat/repodata/ on this server.
from helpdesk.
jenkins-infra-bot
commented on September 24, 2024
Andrei Neagoe Could you please be more specific about what you're doing and what you expect here? Note how your error message has nothing to do with the issue description, repomd.xml exists, there's just no directory index.
from helpdesk.
jenkins-infra-bot
commented on September 24, 2024
Daniel Beck Repo is defined with type yum and this URL: http://pkg.jenkins-ci.org/redhat/
I was now able to sync after switching the URL used to https://pkg.jenkins.io/redhat/
The problem seems to have been with our transparent proxy that blocks access to http://pkg.jenkins-ci.org/redhat/repomd.xml but is completely happy with the other one (both http and https). I also see that https://pkg.jenkins-ci.org is using a wrong certificate (for pkg.jenkins.io).
This ticket can be closed again.
from helpdesk.
jenkins-infra-bot
commented on September 24, 2024
Confirmed today that the repomd.xml file is available as expected.
from helpdesk.
Related Issues (20)
- Junit plugin fails to download and causes Jenkins setup wizard to fail HOT 6
- Move the jenkins-stats repositories to the jenkins-infra organization HOT 1
- I did not get password from jenkins HOT 5
- openshift-jenkins docker image build fails with error "Failed to download plugin:<plugin name>" HOT 1
- Create `jenkinsci/build-token-root-plugin-developers` HOT 3
- "`NoPluginFoundForPrefixException`: No plugin found for prefix 'help' in the current project and in the plugin groups" when running on ACP HOT 17
- Delete comments and spam bot 'browncheir' HOT 2
- problem logging in HOT 4
- [April 2024] Cloudflare API tokens are expired HOT 6
- Jenkins Security Scan fail due to Artifactory timeout HOT 1
- Update-center / crawler root certificate expires the 04 of May (failing job 1 month before) HOT 4
- Archive Pegdown formatter plugin repository HOT 1
- Grant docs team permission on jenkins-infra/plugins-wiki-docs HOT 9
- Release of `jenkins/*` container images on DockerHub errors with `HTTP/429` HOT 5
- Add markdown-formatter plugin to ci.jenkins.io
- Not able to create account HOT 3
- Not able to create account HOT 2
- User haroldprude submitted a spam comment to issues.jenkins.io HOT 1
- [Artifactory (repo.jenkins-ci.org)] perform a systematic audit of all non-plugin artifacts in our `releases` repository to identify and quarantine unused artifacts HOT 1
- Weekly repo issue HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from helpdesk.