Replication between two harbor fails (UNAUTHORIZED error) about harbor HOT 10 OPEN

I have the same issue. Espically when syncing many images (> 100) from A to B in one replication task.

Harbor
Version v2.5.1-b0506782

from harbor.

Is there any configuration recently?
Could you please try to push/pull image with the same robot and admin account respectively via cli?

from harbor.

@stonezdj
First, thank you for your response.
I checked both image push and pull after logging in as robot and admin, and it worked fine.
Also, of course, test connection when registering harbor registries succeeded without any problem.

from harbor.

We are facing the same issue where the replication intermittently fails either with 401 or 403.

Edit: This is a screenshot from the same replication mechanism at different times:

All of the failures are 403 Unauthorized even though it's the exact same config that goes from 200 to 403 and back to 200

from harbor.

I have the same issue. Espically when syncing many images (> 100) from A to B in one replication task.

Harbor Version v2.5.1-b0506782

I have the same issue with the Harbor version v2.10.2-1a741cb7. I can't replicate all the images from one harbor instance to another.

from harbor.

For us, the solution was to correctly set the max_open_conns.
The value was simply too high, which was saturating the DB.

The initial requests through the Harbor Auth middleware to fetch and authenticate the robot accounts from the DB were failing and harbor does not seem to stop the flow if the auth fails.

from harbor.

I run into the same issue.
For some reason, it works if I first do a docker push. Then the replication will always work again. As if the repository is not created by replication but by docker push.

@Joseph94m may I ask to what value you set max_open_conns?

from harbor.

We had max_open_conns set to 100 but that was causing lots of issues with the backend database because its max connection was also 100 and at least 3 of these connexion slots are reserved for superusers in the DB.

When we reduced the max_open_conns to ~ 50.
That issue was fixed.
I think any value < 97 should have worked for us.

Try checking the logs of your DB see if any requests get rejected.

from harbor.

Thanks!

In the meanwhile I figured out that it was not the max_open_conns but the fact that I made a typo in the protocol, I used http instead of https for the remote registry. After fixing that it started to work.

Odd that the connection test and pushing deleted images worked, but I guess that would be part of another issue.

from harbor.

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

from harbor.