[REQUEST]: Bulk Export API access for Pacstall about repology-updater HOT 12 CLOSED

@D-Brox this is not true. We can offer our own API for personal servers that serves the cached bulk export.

from repology-updater.

@saenai255 yes, as long as you do no more than 1 rps and set an unique user-agent header

from repology-updater.

I think this would only work with the official server. Personal servers would still need to use the old method.

from repology-updater.

Doable (via https://github.com/repology/repology-exports), no ETA though. Please give me a hint on what export format would be most convenient for this task.

from repology-updater.

JSON would be the easiest format for us to implement. Regarding Repology Exports, it seems to only parse a data file and doesn't actually download or connect to any API.

from repology-updater.

JSON indeed, I'd like to know which fields you need there.

Regarding Repology Exports, it seems to only parse a data file and doesn't actually download or connect to any API.

That's not intended to be run by you, it's intended to be setup on the Repology side to produce static bulk json exports on a regular basis.

from repology-updater.

Ideally, the more data we have, the better.

However, the essential fields are: repo, subrepo, srcname, binname, visiblename, version, origversion, status, and projectname. Here, projectname would be the path variable used when searching for a particular package. As an example, in the following endpoint /api/v1/project/firefox the projectname would be firefox

Our users rely on these fields to filter packages and find the correct one.

from repology-updater.

Would it be fine if we used this export script until you finish yours? It fetches all the Repology project pages sequentially and aggregates them in an SQLite database. We plan to run this script about twice a day.

from repology-updater.

On our server, we're getting the error:

2023/08/19 10:35:53 ERROR: Failed to export Repology projects: failed to fetch repology project page
Get "https://repology.org/api/v1/projects/": dial tcp 185.185.68.145:443: i/o timeout

but not on any of my personal computers, so I believe that our server is being blocked for some reason.

from repology-updater.

so I believe that our server is being blocked for some reason

These are listed one message above. If these are fixed, I need your IP to unblock.

from repology-updater.

Our IP is 104.238.165.50.

from repology-updater.

Our IP is 104.238.165.50.

Unblocked, please make sure you follow terms of use.

from repology-updater.