open-quantum-safe / oqs-provider Goto Github PK

oqsprovider fails to interoperate with test.openquantumsafe.org when using hybrid algorithms. Review this for reference: openssl/openssl#16989 (comment) -> extraction/insertion (encode/decode) of EC (public) key components seems to be not fully standardized. Thoughts welcome. Edit: Mistake introduced by #40: Hybrid KEMs do not have length fields as per https://datatracker.ietf.org/doc/html/draft-ietf-tls-hybrid-design-00

With open-quantum-safe/liboqs#1369 landed, not entirely unexpected, BIKE doesn't interoperate any more in TLS1.3 with test.openquantumsafe.org (and has no code point for bikel5).
This issue is to document the question & decision whether and to which value(s) to update the code points for these algorithms.
@dstebila @dkostic @christianpaquin @crockeea : Do you have any code points already defined for BIKE(-R4)-L1...5 or shall I simply chose the "next free" ones? Retaining the current ones I'd consider a suboptimal choice, but I'm open to any suggestion.

The qsc-encoder integration requires an online connection even when only running make. This disables the possibility to develop code when offline.

Is this something easily remedied, @bhess? If not, would you mind I set USE_ENCODING_LIB=OFF by default?

Also, what happens in case of a release? Will all required code be included or will this online dependency be permeated into a ZIP file?

I built the provider in msys mingw64 under Windows 11.

When running the commands

set OPENSSL=[path_to]\openssl.exe
set OPENSSL_CONF=[path_to]\openssl.cfg
set ALGO=falcon1024
set KEY=falcon1024_srv.key
set PUBKEY=falcon1024_srv.pubkey
set CERT=falcon1024_srv.crt
set ISSUERCERT=falcon1024_CA.crt
set ISSUERKEY=falcon1024_CA.key
:: Issuer Cert & Key
%OPENSSL% req -x509 -new -newkey %ALGO% -keyout %ISSUERKEY% -out %ISSUERCERT% -nodes -subj "/CN=oqstest CA" -days 365
:: Subject Key & csr
%OPENSSL% req -new -newkey %ALGO% -keyout %KEY% -out %ALGO%_srv.csr -nodes -subj "/CN=oqstest server"
:: Subject Cert
%OPENSSL% x509 -req -in %ALGO%_srv.csr -out %CERT% -CA %ISSUERCERT% -CAkey %ISSUERKEY% -CAcreateserial -days 365
:: Subject Pub Key
%OPENSSL% x509 -in %CERT% -pubkey -noout > %PUBKEY%
:: Sign / Verify
%OPENSSL% dgst -sign %KEY% -out dgstsignfile inputfile
%OPENSSL% dgst -signature dgstsignfile -verify %PUBKEY% inputfile

with empty inputfile, the following error occurs
9CDB0000:error:4000000D:lib(128):oqs_sig_verify:reason(13):C:/projects/openssl_jetzt_tuts/oqs_provider/oqsprov/oqs_sig.c:335: 9CDB0000:error:0300009E:digital envelope routines:do_sigver_init:no default digest:crypto/evp/m_sigver.c:277:.

This error does not occur when using a non-empty inputfile. Is the behaviour on empty inputfiles intended?

hello
I am getting the following errors when I install oqs-provider on windows with mingw32.

$ ninja -j 4
[11/28] Building C object test/CMakeFiles/oqs_test_signatures.dir/test_common.c.obj
C:/msys64/home/oqs-provider/oqs-provider-main/test/test_common.c: In function 'alg_is_enabled':
C:/msys64/home/oqs-provider/oqs-provider-main/test/test_common.c:27:20: warning: implicit declaration of function 'index' [-Wimplicit-function-declarat
ion]
27 | while((comma = index(alglist, ','))) {
| ^~~~~
C:/msys64/home/oqs-provider/oqs-provider-main/test/test_common.c:27:20: warning: incompatible implicit declaration of built-in function 'index' [-Wbuil
tin-declaration-mismatch]
[13/28] Building C object test/CMakeFiles/oqs_test_kems.dir/test_common.c.obj
C:/msys64/home/oqs-provider/oqs-provider-main/test/test_common.c: In function 'alg_is_enabled':
C:/msys64/home/oqs-provider/oqs-provider-main/test/test_common.c:27:20: warning: implicit declaration of function 'index' [-Wimplicit-function-declarat
ion]
27 | while((comma = index(alglist, ','))) {
| ^~~~~
C:/msys64/home/oqs-provider/oqs-provider-main/test/test_common.c:27:20: warning: incompatible implicit declaration of built-in function 'index' [-Wbuil
tin-declaration-mismatch]
[14/28] Linking C executable test\oqs_test_signatures.exe
FAILED: test/oqs_test_signatures.exe
cmd.exe /C "cd . && C:\msys64\mingw32\bin\gcc.exe test/CMakeFiles/oqs_test_signatures.dir/oqs_test_signatures.c.obj test/CMakeFiles/oqs_test_sig
natures.dir/test_common.c.obj -o test\oqs_test_signatures.exe -Wl,--out-implib,test\liboqs_test_signatures.dll.a -Wl,--major-image-version,0,--min
or-image-version,0 C:/openvpn3/lib/libcrypto.dll.a -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvap
i32 && cd ."
C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/12.2.0/../../../../i686-w64-mingw32/bin/ld.exe: test/CMakeFiles/oqs_test_signatures.dir/test_com
mon.c.obj:test_common.c:(.text+0x165): undefined reference to **index'** collect2.exe: error: ld returned 1 exit status [15/28] Linking C executable test\oqs_test_kems.exe FAILED: test/oqs_test_kems.exe cmd.exe /C "cd . && C:\msys64\mingw32\bin\gcc.exe test/CMakeFiles/oqs_test_kems.dir/oqs_test_kems.c.obj test/CMakeFiles/oqs_test_kems.dir/test_c ommon.c.obj -o test\oqs_test_kems.exe -Wl,--out-implib,test\liboqs_test_kems.dll.a -Wl,--major-image-version,0,--minor-image-version,0 C:/openvpn 3/lib/libcrypto.dll.a -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 && cd ." C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/12.2.0/../../../../i686-w64-mingw32/bin/ld.exe: test/CMakeFiles/oqs_test_kems.dir/test_common.c. obj:test_common.c:(.text+0x165): undefined reference to index'
collect2.exe: error: ld returned 1 exit status
[17/28] Building C object oqsprov/CMakeFiles/oqsprovider.dir/oqs_encode_key2any.c.obj
ninja: build stopped: subcommand failed.

this error is in the mingw32.
This error is due to the index() function. Is it possible to use strchr instead of this function?

when using visual studio this error does not exist and the following error occurs:
fatal error: 'stdatomic.h' file not found.

i think something wrong.
Thank you in advance!

Both for KEMs (#16) as well as for SIGs (quite some upstream integration points missing; possibly requiring some X.509/encode wizardry)

Goal: add an abstraction and implementation to support ASN.1 encoding of P8/SPKI as specified by Internet-drafts:

https://datatracker.ietf.org/doc/draft-ietf-lamps-dilithium-certificates/
https://datatracker.ietf.org/doc/draft-uni-qsckeys-dilithium/00/
https://datatracker.ietf.org/doc/draft-uni-qsckeys-falcon/00/
https://datatracker.ietf.org/doc/draft-uni-qsckeys-kyber/00/
https://datatracker.ietf.org/doc/draft-uni-qsckeys-sphincsplus/00/

API would contain:

• encode: take the "raw" keys used by libOQS internally, return the specified encoding.
• decode: take the encoding according to a specification, return the "raw" keys used by libOQS internally.

API should allow to encode/decode private keys with/without optional public key component (#88).

The drafts get their own encode/decode functions and ASN.1 wrapping / encoding / TLV will be hidden behind them. Encoding will be selected by ENV variable. It would make interop to different IETF-drafts and IETF-draft-versions easier.

Describe the bug
The CI tests show error messages like "Could not create OQS signature encoding algorithm draft-uni-qsckeys-falcon-00/sk-pk (Falcon-512, 1.3.9999.3.6). Defaulting to no encoding."

To Reproduce
See for example https://app.circleci.com/pipelines/github/open-quantum-safe/oqs-provider/410/workflows/a97aa465-7023-4479-89e0-844fcb926e7f/jobs/431 (scroll to last test sequence and check logfiles).

Expected behavior
The tests should fail and not pass -- otherwise they don't have value, right @bhess?

Environment (please complete the following information):
CircleCI

Update/add: I cannot reproduce this behaviour locally, so here's at least a screenshot:

RFC seems to be at version 01 (https://datatracker.ietf.org/doc/draft-uni-qsckeys/01/) while encoding supported seems to be at level "00". Not aware whether this has a functional difference, but the available encoding strings thus appear stale.

Hello.
I attempted to run the following command:

LD_LIBRARY_PATH=~/.local/lib64 ~/.local/bin/openssl genpkey -algorithm p521_kyber1024 -outform PEM -out ~/oqs_key.pem -provider-path ~/src/git/oqs-provider/_build/oqsprov  -provider default -provider oqsprovider

but got the following error:

Error writing key
4097678B1E7F0000:error:1D800065:ENCODER routines:OSSL_ENCODER_to_bio:reason(101):crypto/encode_decode/encoder_lib.c:55:No encoders were found. For standard encoders you need at least one of the default or base providers available. Did you forget to load them?
4097678B1E7F0000:error:04800073:PEM routines:do_pk8pkey:error converting private key:crypto/pem/pem_pk8.c:133:

This appears be #25, but that issue is closed, and apparently encoders are supposed to be implemented already. Can you please tell me if I am, perhaps, doing something incorrectly?

Thank you.

Provider currently cannot print out key data, e.g., in openssl req -text invocations.

Goal: Use provider to run openssl genpkey -algorithm <SIG> -out <SIG>_srv.key such that it outputs the same file as the OQS-OpenSSL1_1_1 fork.
This depends on #2 and #3.

Either replacing or amending test.openquantumsafe.org with an instance running oqs-provider to also test integrations and OpenSSL3. Would also allow implementing open-quantum-safe/openssl#388 (comment)

Build and test oqsprovider on OSX (x64 and aarch64). Includes adding CI.

Plain QSC algs work, hybrids have failures. Reason unknown.

Create ready-to-deploy binaries for

• Linux (.DEB)
• OSX , ideally via homebrew (resolving #46)
• Windows (.DLL) (resolving #47 and #80)
• RedHat (.RPM)

Availability via GitHub artifact(s) or via other means? www.openquantumsafe.org? Input welcome (@dstebila ?)

Preferably for standardized algorithms (requires resolving #95 first)

In all cases, except for Windows, a separate install of liboqs shall be performed/used/triggered by the install package for oqsprovider.

In the case of Windows, it is considered better for usability (for the average Windows user) to only create a single DLL also including all required liboqs symbols and not requiring a separate liboqs install first. Suggestions welcome as to which install mechanism to use for the DLL (Ideas, @christianpaquin ?)

In the case of OSX, build and install shall be able to make use of the already existing homebrew formula for liboqs.

In the case of Linux distributions, build and install shall be able to make use of possibly already pre-installed liboqs packages.

It seems OSSL3 now installs libs into INSTALLDIR/lib64 where it's not found by cmakes "find_package" any more
-> Local fix required (soft-linking libssl and libcrypto into .local/lib) or upstream bug?

Tagging @levitte: Is this an intentional change --probably very recent, as we always follow OSSL master here-- (installing OSSL libs into INSTALLDIR/lib64 instead of INSTALLDIR/lib) or should I open an issue in https://github.com/openssl/openssl/issues (didn't find anything there searching for these tags)?

In preparation for the IETF hackathon (and the next release), this issue is to check all algorithm IDs are in line with recent liboqs code updates:

• Falcon (see #125)
• BIKE (see #116)

Anything else? @bhess @xvzcf @dstebila : Did we really only update these two algorithms (breaking KATs/requiring TLS ID changes) since the last Hackathon in November 2022? (period of interest since last interop test added)

Hi.

Many people build OpenSSL as a static library.
It appears that OQS provider does NOT work properly with a static OpenSSL as some failing tests demonstrate (e.g. oqs_endecode).
We have been experimenting/trying to build OpenSSL statically with OQS provider 'embedded' the same way as the legacy provider is built.
This is our first attempt and it seems to be working.
Obviously there are advantages and disadvantages here.
The main advantage is that we end up with only one file.

We have run all OQS tests (oqs_signatures, oqs_kems, oqs_groups & oqs_endecode) and they have all passed.

Do we need to run/do more testing here? We can make the binary available (if that helps!).

Cheers and thanks for this project.

Eliminate need for a patch file.

Consequences:
Negative: Introducing the necessity to have openssl test code checked out too.
Positive: Eliminate breakage possibility when openssl test harness changes (e.g., openssl/openssl#18132)

There are a number of files in oqsprov/ that have the OpenSSL copyright boilerplate, which gives copyright to the OpenSSL Authors, which I understand is historical, since this was previously a patchset for OpenSSL proper.

I don't think that's something you want at this point, and I can't see anything wrong with simply removing that boilerplate. What you still have in there that could remind anyone of the built-in OpenSSL providers, it's mostly bread-and-butter lines that are only vaguely similar.

Build time for oqs-provider significantly deteriorated when qsc-encoder integration was done. A glance at disk usage shows the problem vividly: The src of qsc-encoder is more than 2x as large as all of OpenSSL, 3x the size of a fully built liboqs and thus roughly 60% of a complete oqs-provider build (incl. opensslv3 binaries and libraries):

--> Is it really necessary to add KATs for all algorithms (again)? That alone seems to be 255MBytes. Then there's PPTX files and huge .git index files: The latter are 38x larger in qsc-encoder than in all of oqs-provider (3.5MB vs 138MB).

@bhess: Do you agree that this is not OK and have time to remedy this?

@bhess: These hybrids somehow (probably due to upstream code fix) don't pass testing any more. Please let me know if you have time to look into this or whether I should temporarily disable those hybrids, or should dig deeper.

For reference: OK test when doing OSSL3.0 release vs Failed test from today (simple CCI re-run without code change).

Discussed in #110

Build and test oqsprovider on Windows (x64). Includes adding CI.

I build openssl3 as a dynamic library from https://github.com/baentsch/openssl/tree/sigload to test TLS1.3 OQS signature support (https://github.com/open-quantum-safe/oqs-provider/blob/main/test/oqs_test_tlssig.c), but test failed (Used local key pair and certificate files). From error message, it doesn't seems to add TLS_SIGALG_INFO.
So I check provider version information, oqsprovider informations can display and its status is active；But when I use openssl command to create quantum-safe key pair and certificate(like dilithium), only generate empty files...

Implement provider-based openssl cms [...] such that it interacts successfully with CMS implementation in OQS-OpenSSL1_1_1 fork.
This depends on #7

Goal: Execute provider-based openssl [x509] req -new -newkey <SIG> [...] such that it creates [X509] files interoperable with the OQS-OpenSSL1_1_1 fork.

This depends on #2 and #3.

Separate code paths exist for utilizing specific MD algorithms requested by users for signing. There should be tests exercizing them. Note to self: This condition looks fishy: Should be for mdctx, arguably.

Updated

To make some details clear, as previous overly-generic description invited useless overly-generic observations, like "could not replicate on Windows".

Describe the bug
ctest crashes with SIGTRAP.

To Reproduce
Steps to reproduce the behavior:

1. Build or get installed OpenSSL system-wide, e.g., in /opt/local/libexec/openssl3. For this test I used OpenSSL-3.1.0, and used Macports to get the binary installed.
2. Build and install liboqs system-wide. I used liboqs master, and installed it in opt/local: /opt/local/lib for the shared library, /opt/local/include for the header files.
3. Clone, build, and install this provider (don't forget to edit openssl.cnf as appropriate).
4. export OPENSSL_APP=/opt/local/libexec/openssl3/bin/openssl, export OPENSSL_MODULES=/opt/local/libexec/openssl3/lib/ossl-modules
5. Optional? To make the environment closer to mine, install pkcs11-provider and GOST engine system-wide, and adjust openssl.cnf to point at them.
6. Further complication Install oqs-provider and make it available system-wide by adding it to openssl.cnf.
7. Go to _build and do ctest --output-on-failure
8. Observe the error report.

Expected behavior
Tests passing.

Screenshots

Crash report

Translated Report (Full Report Below)
-------------------------------------

Process:               oqs_test_kems [17508]
Path:                  /Users/USER/*/oqs_test_kems
Identifier:            oqs_test_kems
Version:               ???
Code Type:             ARM-64 (Native)
Parent Process:        ctest [17500]
Responsible:           Terminal [983]
User ID:               501

Date/Time:             2023-03-27 15:39:40.6519 -0400
OS Version:            macOS 13.2.1 (22D68)
Report Version:        12
Anonymous UUID:        161C054B-E964-CDD3-5EBC-5A9DBE3E2AE2


Time Awake Since Boot: 66000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BREAKPOINT (SIGTRAP)
Exception Codes:       0x0000000000000001, 0x00000001a6283108

Termination Reason:    Namespace SIGNAL, Code 5 Trace/BPT trap: 5
Terminating Process:   exc handler [17508]

Application Specific Information:
BUG IN CLIENT OF LIBPLATFORM: Trying to recursively lock an os_once_t
Abort Cause 259


Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libsystem_platform.dylib      	       0x1a6283108 _os_once_gate_recursive_abort + 36
1   libsystem_platform.dylib      	       0x1a627f710 _os_once_gate_wait + 348
2   libsystem_pthread.dylib       	       0x1a624dd84 pthread_once + 100
3   libcrypto.3.dylib             	       0x1012bdfcc CRYPTO_THREAD_run_once + 12
4   libcrypto.3.dylib             	       0x1012d0ea4 ossl_obj_add_object + 236
5   gostprov.dylib                	       0x1015bf5b4 populate_gost_engine + 116
6   gostprov.dylib                	       0x1015bd478 OSSL_provider_init + 116
7   libcrypto.3.dylib             	       0x1012bbddc provider_activate + 260
8   libcrypto.3.dylib             	       0x1012bbc48 ossl_provider_activate + 56
9   libcrypto.3.dylib             	       0x1012ba93c provider_conf_init + 608
10  libcrypto.3.dylib             	       0x101212c4c CONF_modules_load + 856
11  libcrypto.3.dylib             	       0x101212ee8 CONF_modules_load_file_ex + 120
12  libcrypto.3.dylib             	       0x101213738 ossl_config_int + 68
13  libcrypto.3.dylib             	       0x1012b2400 ossl_init_config_ossl_ + 16
14  libsystem_pthread.dylib       	       0x1a624ddec __pthread_once_handler + 76
15  libsystem_platform.dylib      	       0x1a627d7e0 _os_once_callout + 32
16  libsystem_pthread.dylib       	       0x1a624dd84 pthread_once + 100
17  libcrypto.3.dylib             	       0x1012bdfcc CRYPTO_THREAD_run_once + 12
18  libcrypto.3.dylib             	       0x1012b2208 OPENSSL_init_crypto + 1104
19  libcrypto.3.dylib             	       0x1012d1098 obj_lock_initialise_ossl_ + 20
20  libsystem_pthread.dylib       	       0x1a624ddec __pthread_once_handler + 76
21  libsystem_platform.dylib      	       0x1a627d7e0 _os_once_callout + 32
22  libsystem_pthread.dylib       	       0x1a624dd84 pthread_once + 100
23  libcrypto.3.dylib             	       0x1012bdfcc CRYPTO_THREAD_run_once + 12
24  libcrypto.3.dylib             	       0x1012d0408 OBJ_sn2nid + 112
25  libcrypto.3.dylib             	       0x1012d02f4 OBJ_txt2obj + 216
26  libcrypto.3.dylib             	       0x1012d0944 OBJ_txt2nid + 20
27  libcrypto.3.dylib             	       0x1012bd048 core_obj_create + 36
28  oqsprovider.0.5.0-dev.dylib   	       0x100f73678 OSSL_provider_init + 292
29  libcrypto.3.dylib             	       0x1012bbddc provider_activate + 260
30  libcrypto.3.dylib             	       0x1012bbc48 ossl_provider_activate + 56
31  libcrypto.3.dylib             	       0x1012ba93c provider_conf_init + 608
32  libcrypto.3.dylib             	       0x101212c4c CONF_modules_load + 856
33  libcrypto.3.dylib             	       0x101212ee8 CONF_modules_load_file_ex + 120
34  libcrypto.3.dylib             	       0x1012af1a4 OSSL_LIB_CTX_load_config + 20
35  oqs_test_kems                 	       0x100de7420 main + 80
36  dyld                          	       0x1a5f27e50 start + 2544


Thread 0 crashed with ARM Thread State (64-bit):
    x0: 0x0000000000000103   x1: 0x000000016f019b90   x2: 0x00000001a624dda0   x3: 0x0000000000000103
    x4: 0x000000000000000a   x5: 0x0000000024200000   x6: 0x0000000000000000   x7: 0x0000000000000500
    x8: 0x0000000000000103   x9: 0x0000000000000103  x10: 0x0000000000000103  x11: 0x0000600000fb8000
   x12: 0x0000000000000010  x13: 0x00000000fffffcee  x14: 0x00000000000007fb  x15: 0x00000000a4188ffb
   x16: 0x00000001a627d760  x17: 0x00000002066400a0  x18: 0x0000000000000000  x19: 0x0000000101438d58
   x20: 0x0000000000000103  x21: 0x00000001a624dda0  x22: 0x000000016f019b90  x23: 0x0000000000000103
   x24: 0x0000000000000103  x25: 0x0000000000000000  x26: 0x0000000000000002  x27: 0x0000000000000002
   x28: 0x0000600000fa4000   fp: 0x000000016f019b80   lr: 0x00000001a627f710
    sp: 0x000000016f019b50   pc: 0x00000001a6283108 cpsr: 0x60001000
   far: 0x00000001ff2bc0b8  esr: 0xf2000001 (Breakpoint) brk 1

Environment (please complete the following information):

• OS: MacOS Ventura 13.2.1
• OpenSSL version 3.1.0
• This provider version: current master (0.5.0-dev)

Additional context
This is on MacBook Pro - Apple Silicon M2 chip. Similar problem on Intel-based iMac (used same process as above).

Note: commenting out, e.g., GOST provider in openssl.cnf did not help.

$ openssl version
OpenSSL 3.1.0 14 Mar 2023 (Library: OpenSSL 3.1.0 14 Mar 2023)
$ openssl list -providers
Providers:
  base
    name: OpenSSL Base Provider
    version: 3.1.0
    status: active
  default
    name: OpenSSL Default Provider
    version: 3.1.0
    status: active
  gost
    name: OpenSSL GOST Provider
    status: active
  legacy
    name: OpenSSL Legacy Provider
    version: 3.1.0
    status: active
  oqs
    name: OpenSSL OQS Provider
    version: 0.5.0-dev
    status: active
  pkcs11
    name: PKCS#11 Provider
    version: 3.1.0
    status: active
$ 

Unlike OpenSSL 1.1.1, OpenSSL3.0 permits EVP keys that do not have both public and private key material. Therefore, supporting only "half" PK key pairs as EVP keys makes sense. This also allows for loading (and storing) "true" private EVP OQS keys now. This in turn in in contrast with the oqs-openssl implementation of storing (persisting) public key material together with private key material.

This issue is about discussing whether we should (also) do this in oqs-provider, ensuring interoperability with oqs-openssl-generated (persistent) data or whether we want to be more efficient and only store private key material in oqs-provider EVP keys, but thus breaking interoperability between oqs-provider and oqs-openssl(1.1.1).

The question also will be how to match private and public keys via the provider: Simply comparing key material (as is possible now), doing actual sign/verify operations, or some "hybrid" (hashing) based approach. Example code currently here.

Already extending the feature set is that oqs-provider supports more hybrid KEM mechanisms. But for full TLS signature operations oqs-openssl will remain more relevant until openssl/openssl#10512 is resolved thus making interoperability between oqs-openssl and oqs-provider desirable for users (of most functionality and future-proof OpenSSL).

Thoughts welcome, @dstebila @christianpaquin

in https://datatracker.ietf.org/doc/draft-ietf-lamps-dilithium-certificates/

Enables:
openssl genpkey -algorithm $1 -out priv.pem
openssl req -x509 -new -key priv.pem -out ca.csr -subj "/CN=CN name"

Mirrors open-quantum-safe/openssl#333

Hi.

We are trying to deploy OpenSSL, liboqs & OQS provider on Windows 10 x64 for the very first time.

• OpenSSL 3.2.0 master branch built from source
• liboqs 0.7.2 built from source
• oqs-provider 0.4.0 built from source

When running the (unit) tests in oqs-provider, one of them fails.

D:\oqs-provider\bin>oqs_test_endecode.exe oqsprovider ..\etc\oqs.cnf

# INFO:  @ d:\oqs-provider-0.4.0\test\oqs_test_endecode.c:1078
# Generating keys...
1..216
# ERROR: (bool) 'OSSL_ENCODER_to_bio(ectx, mem_ser) == true' failed @ d:\oqs-provider-0.4.0\test\oqs_test_endecode.c:495
# false
# ERROR: (bool) 'encode_cb(file, line, &encoded, &encoded_len, pkey, selection, output_type, output_structure, pass, pcipher) == true' failed @ d:\oqs-provider-0.4.0\test\oqs_test_endecode.c:136
# false
# OPENSSL_TEST_RAND_ORDER=1666474314
not ok 1 - test_unprotected_dilithium2_via_DER

Due to our quite limited knowledge of this project, it is really hard for us to debug or troubleshoot this issue.
Any pointers are greatly appreciated.

Regards.

Hi,
we are currently developing a crypto api and found some memory leaks with valgrind during key creation with pqc algorithms.

The leaks we found:

• During EVP_PKEY_keygen_init in oqs_kmgmt.c, line 422
  gctx->oqs_name = OPENSSL_strdup(oqs_name);
• During EVP_PKEY_generate in oqsprov_keys.c, line 618
  ret->tls_name = OPENSSL_strdup(tls_name);
• During EVP_DigestSignInit from ASN1_item_i2d in oqs_sig.c
  Not able to fix it, but we assume its coming from line 156: ctx->aid_len = get_aid(&(ctx->aid), ctx->sig->tls_name);, from a not correctly freed aid.

We started with version 0.4.0 of the oqs-provider, when we saw that there are memory leaks fixed on commit, we switched to the current main version. The leaks listed above also occurred there.

Follow-up after #16:

• So far, the shared secret uses "Comb-Concat" as described in https://tools.ietf.org/html/draft-ietf-tls-hybrid-design-01#appendix-B.4.1. The same is done in OSSL111. This is suitable for inputting the shared secret to the TLS 1.3 key schedule. Compared to the implementation in OSSL111, the oqs-provider can also be used outside a TLS context. For this purpose, a method like "Comb-KDF" would be useful.
• Investigate using more OSSL3 API to allow more flexible combination of hybrid schemes:
  -> query algorithm parameters (secret-, ciphertext-, key-lengths) with provider API
  -> define individual algorithms for hybrid KEM in an array
  -> unify initialization of EVP-ECP and EVP-ECX code

Some test cases are dependent on the availability of the openssl test library. That is not available in binary-only installs of openssl. This issue is to ensure oqsprovider testing (with or without Debug config) can be executed without presence of a full openssl source and test environment.

Hi,
I followed the instructions in the README.md to install the 3 components, OpenSSL, liboqs, oqs-provider. I installed OpenSSL and liboqs separately then compiled oqs-provider pointing to local/lib as described. I can see my list of providers as below

LD_LIBRARY_PATH=./local/lib ./local/bin/openssl list -signature-algorithms -provider-path ./local/lib/ossl-modules  -provider oqsprovider
  oqs_sig_default @ oqsprovider
  dilithium2 @ oqsprovider
  dilithium3 @ oqsprovider
  dilithium5 @ oqsprovider
  dilithium2_aes @ oqsprovider
  dilithium3_aes @ oqsprovider
  dilithium5_aes @ oqsprovider
  falcon512 @ oqsprovider
  falcon1024 @ oqsprovider
  picnicl1full @ oqsprovider
  picnic3l1 @ oqsprovider
  rainbowIclassic @ oqsprovider
  rainbowVclassic @ oqsprovider
  sphincsharaka128frobust @ oqsprovider
  sphincssha256128frobust @ oqsprovider
  sphincsshake256128frobust @ oqsprovider

However, running

LD_LIBRARY_PATH=./local/lib ./local/bin/openssl genpkey -algorithm dilithium2 -provider-path ./local/lib/ossl-modules -provider oqsprovider -provider default

I get

Error writing key
8012DE77217F0000:error:1D800065:ENCODER routines:OSSL_ENCODER_to_bio:reason(101):crypto/encode_decode/encoder_lib.c:56:No encoders were found. For standard encoders you need at least one of the default or base providers available. Did you forget to load them?
8012DE77217F0000:error:04800073:PEM routines:do_pk8pkey:error converting private key:crypto/pem/pem_pk8.c:133:
0x55e19808a720:   0:OQSX_KEY

What am I missing?
Regards.

Openssl3.0, 3.1, 3.2/master have different provider API capabilities. This issue is to make sure oqsprovider builds against all variants. Most notably, pluggable signature functionality should be added automatically as/when OpenSSL supports that capability (most likely not in 3.0).

Some tests depend on ssltestlib, an OpenSSL internal test library. This dependency causes issues when testing in a setup without this code present (non-master source, binary-only).

This issue is to suggest removing this dependency and to develop independent tests providing similar functionality.

It would be nice to know where Open Quantum Safe is with providing interface compatibility for earlier versions of openssl.

Providing a way to extend the setup of an existing system with a small bundle would make for easy use, so it's a question visitors might arrive with.

Warning: This is a big issue. Anyone willing to tackle it should consider this requires substantial OpenSSL3 support (see for example openssl/openssl#10512). Also helpful to get started may be discussions documented here.

As the conclusion arrived at in #32 (de-emphasizing oqs-openssl111 maintenance and support) has been reverted in yesterday's team meeting, this issue is to track tasks to re-activate interop testing between oqs-openssl111 and oqs-provider.

Implications:

• Interop test scripts need to be parameterized to
  o permit switching to different docker images (oqs-openssl111-curl had been used but has changed in the meantime to oqs-provider-curl)
  o permit switching off RFC data encoding testing as that is not implemented in oqs-openssl111 (or are you willing to add that also to oqs-openssl111, @bhess?)
• create and maintain separate docker images for oqs-provider and oqs-openssl111 operations (at least for curl) -- or completely revert open-quantum-safe/oqs-demos#190 --opinions, @christianpaquin @dstebila ?

As open-quantum-safe/liboqs#1333 lands, oqsprovider must be able to cope/be built with subset of algorithms enabled. This also optimizes its size.

Pursuant open-quantum-safe/openssl#313

@bhess: When doing this (assumed simple) update I had to notice that the new NIDs you created "extra" now clash with those we had to make up in oqs-openssl to keep "s2n history"... Please check out #26 for a first cut at resolving this issue: Would you want to make a suggestion how to fix things now? Either "registering" your new hybrid NIDs via an amend-commit to open-quantum-safe/openssl#313 or amend #26 (change extra-NIDs) here. I'd prefer the latter (updating extra NIDs) to not make things more complicated in oqs-openssl.

The KEMs are currently tested with TLS group tests.

OpenSSL 3 also allows to use KEMs directly using the EVP API. See e.g. https://www.openssl.org/docs/manmaster/man3/EVP_PKEY_encapsulate.html

KEM tests can be added that directly uses the EVP KEM API.

Interop tests against https://test.openquantumsafe.org fail for p384_kyber90s768 and p256_oqs_kem_default

As per https://www.openssl.org/docs/manmaster/man7/provider-encoder.html

When trying to run TLS with the signature algorithms sphincssha256128frobust, sphincsharaka128frobust, sphincsharaka128fsimple, sphincsshake256128fsimple, I get the following errors.
Server error:

7C240000:error:0A000417:SSL routines:ssl3_read_bytes:ssl/tls alert illegal parameter:ssl\record\rec_layer_s3.c:839:SSL alert number 47

Client error:

142B0000:error:0A000098:SSL routines:read_state_machine:excessive message size:ssl\statem\statem.c:648:

Some further comments:

• I double-checked if the algorithms above were enabled by liboqs and indeed they are.
• TLS works with sphincssha256128ssimple and the other enabled signature algorithms.

I debugged into the openssl library.
In statem.c:

            if (s->s3.tmp.message_size > max_message_size(s)) {
                SSLfatal(s, SSL_AD_ILLEGAL_PARAMETER,
                         SSL_R_EXCESSIVE_MESSAGE_SIZE);
                return SUB_STATE_ERROR;
            }

max_message_size(s) is set by SSL3_RT_MAX_PLAIN_LENGTH = 16384. However, s->s3.tmp.message_size is larger than 16384 this resulting in the SSLfatal error.

#80 describes a problem triggered by errors occurred earlier. This issue is to review and suitably change code throughout the provider such as to not just raise error messages but to also effectively exit on them if they will cause "fatal" consequent errors.