MacOS build failure with OpenSSL 3.0 about opensc HOT 6 CLOSED

@metsma did you manage to have a look into this? We can probably ignore the errors or try to fix them (but there is dozens of them).

I am always surprised by the macos ci that works in PR but fails on master. Is there something we can make it failing the same way for both of them?

Or is it unrelated to the OpenSSL version change? By comparing the logs, I see the macos was updated from 12.6.9 to 12.7.2, but it still works in the PRs.

from opensc.

I will try find time for this.

from opensc.

I did look into, but this requires more in depth knowledge of all OpenSC internals.
Etc. all RSA bit/byte sizes are converted between size_t (unsigned long) until unsigned int and int (OpenSSL BN_bin2bn).
Not sure how it will affect if we change all them to size_t or int?
About usage and algorithm parameters we should probably use more fixed size. Currently I can see there is int, unsigned int, unsigned long. (Maybe u16? or even u32?)

from opensc.

Thank you for having a look into that! I agree that some syncing on a specific type would be beneficial. I think most of the internal types is already using size_t so I would propose to use size_t where possible (where it is really only size and not a error code or something).

Algorithms or fixed enums should really used that enum number if possible. In other cases, the unsigned long usage mostly stems from the PKCS#11 level specification where this type is used for most of the stuff so my proposal would be to try to stick to this.

I started changing some of the stuff, but there is still a lot to go.

I thought if you have some insight of what changed. If it is because of the new OpenSSL, clang, mac or something completely independent ...

from opensc.

I think this is related to master builds signed packages and then OpenSCTokenApp is pulled to build.
OpenSC package use xcode project and thus different compiler flags.

from opensc.

I think the #2993 will fix the master build, but I do not know how to check this in the PR (I tested with the flags locally and in additional specially crafted linux run with clang and these specific flags). I know, its huge, but inputs/reviews are welcomed.

from opensc.