elementsproject / libwally-core Goto Github PK

wally provides python bindings and provides binary version of these python bindings with wally - however IIRC these can't be uploaded to pip as they don't follow the anylinux build requirements. IIRC this is related to glibc versions.

Wally pip wheels should be anylinux or manylinux compatible - https://www.python.org/dev/peps/pep-0571/

A good approach could be a variant on wally_wif_to_address:

int wally_addr_from_bip32_key(const struct ext_key *hdkey, const char *addr_family,\
                              uint32_t flags, char **output)

Where flags indicate the address type (legacy, native segwit, p2sh-segwit).

Most of the implementation can be borrowed from c-lightning, in particular encode_pubkey_to_addr which uses a pubkey straight from ext_key.

If you pass a 32 byte sized array into mnemonic_from_bytes a 23 word phrase is created.

Looking at the first two lines:
size_t total_bits = bytes_len * 8u; /* bits in 'bytes' */
size_t total_mnemonics = total_bits / w->bits; /* Mnemonics in 'bytes' */

There's no way total_mnemonics could result in the correct value of 24 for a 32 byte parameter.

Similar to #103, but it would produce bytes containing the scriptPubKey.

This is useful when composing a transaction, to directly generate a change output. Otherwise you have to derive a change address with wally_bip32_key_to_address / wally_bip32_key_to_addr_segwit and then parse it with wally_addr_segwit_to_bytes / wally_address_to_scriptpubkey (#112)

The bip32 library only works with xpub/xprv.

If you want to use xpub/ypub/zpub etc. (see https://github.com/satoshilabs/slips/blob/master/slip-0132.md), the library gives you a hard time currently.

I currently work around it by using only the xprv/xpub versions, and then force overwriting the version bytes when serializing.

It would be great to keep the bip32 API flexible enough so that one can provide custom version bytes.

Bitcoin Core introduced Output Descriptors recently: https://github.com/bitcoin/bitcoin/blob/master/doc/descriptors.md

They're quite useful, but there's no BIP. So perhaps it's premature to implement. But if so, I'd like to see a method that takes a descriptor string and an index as input, and returns an address (like the deriveaddresses RPC).

E.g. from one of the Bitcoin Core examples:

Inputs:
* Descriptor: 
wsh(multi(1,xpub661MyMwAqRbcFW31YEwpkMuc5THy2PSt5bDMsktWQcFF8syAmRUapSCGu8ED9W6oDMSgv6Zz8idoc4a6mr8BDzTJY47LJhkJ8UB7WEGuduB/1/0/*,xpub69H7F5d8KSRgmmdJg2KhpAK8SR3DjMwAdkxj3ZuxV27CprR9LgpeyGmXUbC6wb7ERfvrnKZjXoUmmDznezpbZb7ap6r1D3tgFxHmwMkQTPH/0/0/*))#t2zpj2eu
* Index: 0

Output:
* Address: bc1qvjtfmrxu524qhdevl6yyyasjs7xmnzjlqlu60mrwepact60eyz9s9xjw0c

For better performance it could take an array of indices to calculate an array of addresses, or it could output a cache that the consuming application can hold on to (that's what Bitcoin Core does).

Without any way of knowing the length to allocate, how do we call wally_psbt_to_bytes?

For the moment, we allocate 1k and loop, doubling each time.
But worse, it returns WALLY_EINVAL for reasons OTHER than "you didn't give enough room".

There are two ways of fixing this:

1. Expose psbt_get_length(). This is inefficient.
2. Change the semantics of bytes_written to "bytes_needed", and set it to the number of bytes you want. Then return WALLY_ENOMEM to distinguish from unrelated errors.

The second allows a more optimal implementation in future.

In addition to bip32_key_get_fingerprint it's often useful to know the master key fingerprint. Reusing wally_key_origin_info from wally_psbt.h in wally_bip32.h would make sense to me.

I'm trying to create a transaction for issuing an asset using the Python wrapper of libwally.
For my test, I'm creating a transaction with just 1 input and 2 outputs like the following:

def build_tx(prev_tx_hash, prev_tx_index, seq, issuance_entropy, issuance_amount, ...):
   tx = wally.tx_init(2, 0, 1, 2)
   wally.tx_add_elements_raw_input(
            tx,
            prev_tx_hash,
            prev_tx_index,
            seq, # sequence
            None, # scriptsig, is calculated later
            None, # witness
            None, # nonce
            issuance_entropy, # entropy
            issuance_amount, # issuance_amount
            None, # inflation_keys
            None, # issuance_amount_rangeproof
            None, # issuance_rangeproof
            None, # pegin_witness
            0 # flags, must be zero
   )
   print(wally.hex_from_bytes(wally.tx_get_input_entropy(tx, 0)))

   # add outputs
   [ ... ]

   return wally.tx_to_hex(tx, 0)

When I test this function it actually prints the same entropy I'm passing for the input and returns the hex of the unsigned transaction.

However, when decoding the transaction with liquid-cli decoderawtransaction, I see from the JSON that the input misses the issuance part.

The same happens using libwally with tx_from_bytes(...) and then printing again wally.tx_get_input_entropy(tx, 0), now it's 0, thus I'm quite sure I'm using the wrong flag when returning the serialized transaction but I did not find any example of this in this repo.

There are some plans on adding some functions to help to organize the logical hierarchy of HD wallets with accounts/sub-accounts as in BIP44? Or this should be done in specific implementation outside libwally?

test_clear fails both on OS X El Capitan (Version 10.11.6) with Xcode Version 8.2.1 (8C1002), and macOS Sierra 10.12.5 (16F73) with Xcode Version 8.3.3 (8E3004b).

$ make check
Making check in src
Making check in secp256k1
/Applications/Xcode.app/Contents/Developer/usr/bin/make  check-TESTS
============================================================================
Testsuite summary for libsecp256k1 0.1
============================================================================
# TOTAL: 0
# PASS:  0
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
/Applications/Xcode.app/Contents/Developer/usr/bin/make  check-TESTS check-local
FAIL: test_clear
============================================================================
Testsuite summary for libwallycore 0.1
============================================================================
# TOTAL: 1
# PASS:  0
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See src/test-suite.log
============================================================================
make[4]: *** [test-suite.log] Error 1
make[3]: *** [check-TESTS] Error 2
make[2]: *** [check-am] Error 2
make[1]: *** [check-recursive] Error 1
make: *** [check-recursive] Error 1

src/test-suite.log:

$ cat src/test-suite.log
==========================================
   libwallycore 0.1: src/test-suite.log
==========================================

# TOTAL: 1
# PASS:  0
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: test_clear
================

Found bip39_mnemonic_to_bytes secret at stack position 523592
7f, 7f, 7f, 7f, 7f, 7f, 7f, 7f, 7f, 7f, 7f, 7f, 7f, 7f, 7f, 7f, 
test_bip39 clear() test failed!
FAIL test_clear (exit status: 1)

The user would start with wally_scriptpubkey_get_type. This returns the WALLY_SCRIPT_TYPE_ type. Then we could add individual parser functions for each type, or one generic one:

scriptpubkey_to_address(const unsigned char *bytes, size_t bytes_len, \
                        uint32_t scriptpubkey_type, unsigned char *bytes_out, size_t len)

When scriptpubkey_type is 0 it infers the type.

For SegWit this is uncessary; wally_addr_segwit_from_bytes works fine.

Hello it's very possible that I'm doing it wrong, but I have the impression that the mnemonic generation does not work at all

        std::string get_mnemonic() {
        std::array<unsigned char, WALLY_SECP_RANDOMIZE_LEN> data;
        assert(WALLY_OK == wally_secp_randomize(data.data(), data.size()));
        char* output;
        words* output_words;
        assert(WALLY_OK == bip39_get_wordlist(NULL, &output_words));
        assert(WALLY_OK == bip39_mnemonic_from_bytes(output_words, data.data(), data.size(), &output));
        assert(WALLY_OK == bip39_mnemonic_validate(output_words, output));
        return output;
}

And i have call wally_init and wally_cleanup correctly in the main function

I'm trying to reproduce tests from here and I can't obtain first child non-hardened key.

1. I get correct root key, which is

xprv9s21ZrQH143K31xYSDQpPDxsXRTUcvj2iNHm5NUtrGiGG5e2DtALGdso3pGz6ssrdK4PFmM8NSpSBHNqPqm55Qn3LqFtT2emdEXVYsCzC2U

2. Then I'm trying to compute first child using this key by

xpriv = bip32_key_from_parent(root_key, 0, BIP32_FLAG_KEY_PRIVATE)

I got

xprv9tu9RdvJDMiJvDuA1j9kpsKaa5Z3mRxytPF6W5GZpjxRDppHLL64Tvd2tmvDvyDJnDK8BVCoiWVYxNJC4T2m2TV28jeS5N9XWVtAxU32BtM

which is the same result i can obtain via

child_priv = bip32_key_from_parent_path(root_key, [0], BIP32_FLAG_KEY_PRIVATE)

but Test vector 2 for Chain m/0 has

xprv9vHkqa6EV4sPZHYqZznhT2NPtPCjKuDKGY38FBWLvgaDx45zo9WQRUT3dKYnjwih2yJD9mkrocEZXo1ex8G81dwSM1fwqWpWkeS3v86pgKt

which is obviously different. I can repeat it for hardened path with the same discrepancy. Why results from tests are different?

When I test Python 2.7.15+ and 32bit libwallycore.so, make check always fail.

======================================================================
FAIL: test_bip32 (__main__.BIP32Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "swig_python/contrib/bip32.py", line 56, in test_bip32
    bip32_key_from_parent_path(master, p, 0)
AssertionError: OverflowError not raised

----------------------------------------------------------------------

The following ./src/swig_python/contrib/bip32.py insists
(1) 0xffffffff is valid.
(2) -1, in other word, 0xffffffff is invalid.

the parameter 'p' looks defined as uint32_t in the c code.
Please kindly tell me the correct test standard.

 47         # Test our SWIG integer conversions for overflow etc in path derivation
 48         for p, valid in [([2**32-1],  True),   # 0xffffffff is valid
 49                          ([float(1)], False),  # We don't support float casting
 50                          ([2**32],    False),  # Overflow
 51                          ([-1],       False)]: # Underflow
 52             if valid:
 53                 bip32_key_from_parent_path(master, p, 0)
 54             else:
 55                 with self.assertRaises(OverflowError):
 56                     bip32_key_from_parent_path(master, p, 0)

It seems like Libwally supports the broken 2016 Schnorr implementation:

/** Indicates that a signature using EC-Schnorr-SHA256 is required */
#define EC_FLAG_SCHNORR 0x2

I assume this is totally incompatible with BIP-Schnorr, and that another flag would be required?

Hi!

It's possible to have some example on how to build and include the native binaries in an iOS/ObjectiveC app?

Thank you in advance

I'm using the python build to create and sig transaction. When I try to broadcast signed transaction the verify signature error occurs: non-mandatory-script-verify-flag (Signature must be zero for failed CHECK(MULTI)SIG operation) (code 64).

I created and broadcasted transaction with liquid CLI and decoding it I saw that only vin.txinwitness field is different from transaction build in python, so i think that the error is in my signing function, defined as a follow:

def sign_transaction(raw_tx, vins, privkey, pubkey):
    tx = wally.tx_from_bytes(raw_tx, wally.WALLY_TX_FLAG_USE_ELEMENTS | wally.WALLY_TX_FLAG_USE_WITNESS)

    for i, vin in enumerate(vins):
        prevout_script = bytearray([0x19, wally.OP_DUP, wally.OP_HASH160, 0x14]) + wally.hash160(pubkey) + bytearray([wally.OP_EQUALVERIFY, wally.OP_CHECKSIG])
        prevout_value = wally.tx_confidential_value_from_satoshi(vin["value"]);
        hash_to_sign = wally.tx_get_elements_signature_hash(
            tx,
            i,
            prevout_script,
            prevout_value,
            wally.WALLY_SIGHASH_ALL,
            wally.WALLY_TX_FLAG_USE_WITNESS
        )
        signature = wally.ec_sig_from_bytes(
            privkey,
            hash_to_sign,
            wally.EC_FLAG_ECDSA | wally.EC_FLAG_GRIND_R
        )

        der_signature = wally.ec_sig_to_der(signature) + bytearray([wally.WALLY_SIGHASH_ALL])
        
        witness_stack = wally.tx_witness_stack_init(2)
        wally.tx_witness_stack_add(witness_stack, der_signature)
        wally.tx_witness_stack_add(witness_stack, pubkey)

        script_sig = wally.witness_program_from_bytes(wally.hash160(pubkey), wally.WALLY_SCRIPT_AS_PUSH)
        wally.tx_set_input_script(tx, i, script_sig)
        wally.tx_set_input_witness(tx, i, witness_stack)

    return wally.tx_to_hex(tx, wally.WALLY_TX_FLAG_USE_WITNESS)

If you have a repository with examples or you'll notice something wrong with my script, I'd appreciate it.

This is useful, and we want it for c-lightning!

Please expose the internal one!

(You should also create wally_psbt_clone(), too...)

Not clear why the type was change to the (undefined!) struct secp256k1_context; did you mean just secp256k1_context which is a typedef?

On macOS 10.14.5 with Python 3.7.3 and Node v10.16.0. I think this fails because print "%s.%s.%s" % sys.version_info[:3]; is Python 2 syntax.

./configure --enable-debug --enable-export-all --enable-swig-python --enable-coverage --enable-js-wrappers
make

Fails with:

cd wrap_js && LIBWALLY_DIR=../.. npm_config_debug=1 yarn install
  CCLD     test_bech32
  CCLD     test_tx
yarn install v1.16.0
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 🔨  Building fresh packages...
$ node-gyp configure && node-gyp build
[##########################################################################################################################] 242/242gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | darwin | x64
gyp ERR! configure error 
gyp ERR! stack Error: Command failed: /Users/sjors/.pyenv/shims/python -c import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack   File "<string>", line 1
gyp ERR! stack     import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack                                ^
gyp ERR! stack SyntaxError: invalid syntax

Hi Team,

Summary

While fuzzing libwally-core using clang 6.0 with ASAN a stack-based buffer-overflow was observed in test_clear.c

Vulnerable code from test_clear.c:

    for (i = 0; i < PTHREAD_STACK_MIN - len - 1; ++i)
        if (!memcmp(gstack + i, search, len)) {
            if (caller) {
                printf("Found %s secret at stack position %ld\n", caller, (long)i);
                dump_mem(search, len);
            }
            return true; /* Found */
        }

Output from ASAN:

==97601==ERROR: AddressSanitizer: stack-buffer-underflow on address 0x7f5320c9a5a0 at pc 0x0000004aa159 bp 0x7f5320c9a590 sp 0x7f5320c99d40
READ of size 8 at 0x7f5320c9a5a0 thread T1
    #0 0x4aa158 in __interceptor_memcmp.part.273 (/home/zubin/Git/libwally-core/src/.libs/test_clear+0x4aa158)
    #1 0x50c17e in in_stack /home/zubin/Git/libwally-core/src/ctest/test_clear.c:73:14
    #2 0x50c17e in test_search /home/zubin/Git/libwally-core/src/ctest/test_clear.c:92
    #3 0x50c17e in run_tests /home/zubin/Git/libwally-core/src/ctest/test_clear.c:125
    #4 0x4df782 in __asan::AsanThread::ThreadStart(unsigned long, __sanitizer::atomic_uintptr_t*) (/home/zubin/Git/libwally-core/src/.libs/test_clear+0x4df782)
    #5 0x7f53205f86da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
    #6 0x7f531f95f88e in clone /build/glibc-OTsEL5/glibc-2.27/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

0x7f5320c9a5a0 is located 519584 bytes inside of 524288-byte region [0x7f5320c1b800,0x7f5320c9b800)
allocated by thread T0 here:
    #0 0x4d1cd0 in malloc (/home/zubin/Git/libwally-core/src/.libs/test_clear+0x4d1cd0)
    #1 0x50bc38 in checked_malloc /home/zubin/Git/libwally-core/src/ctest/test_clear.c:61:17
    #2 0x50bc38 in main /home/zubin/Git/libwally-core/src/ctest/test_clear.c:143
    #3 0x7f531f85fb96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310

Thread T1 created by T0 here:
    #0 0x4351c0 in __interceptor_pthread_create (/home/zubin/Git/libwally-core/src/.libs/test_clear+0x4351c0)
    #1 0x50bdc2 in main /home/zubin/Git/libwally-core/src/ctest/test_clear.c:154:11
    #2 0x7f531f85fb96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310

SUMMARY: AddressSanitizer: stack-buffer-underflow (/home/zubin/Git/libwally-core/src/.libs/test_clear+0x4aa158) in __interceptor_memcmp.part.273
Shadow bytes around the buggy address:
  0x0feae418b460: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0feae418b470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0feae418b480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0feae418b490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0feae418b4a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0feae418b4b0: 00 00 00 00[f1]f1 f1 f1 00 f3 f3 f3 00 00 00 00
  0x0feae418b4c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0feae418b4d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0feae418b4e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0feae418b4f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0feae418b500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==97601==ABORTING

Request team to have a look and validate.
Thanks.

The current bip32 API stores 20 bytes for the parent hash, but only ever uses the first four bytes as the fingerprint.

This makes functions like bip32_key_init_alloc() awkward to use, as it takes 20 bytes for the parent, but if you only have data from an xpub, there are only 4 bytes available.

It would also get rid of this inconsistent state:

Thoughts?

Currently python wheels built with swig 4.0 don't work, they fail with

ImportError: cannot import name 'libwallycore' from 'wallycore' (/home/username/venv3/lib/python3.7/site-packages/wallycore/init.py)

A Travis linter task should prevent issues like #110 and item 1 of @glslang feedback on #103.

It could run ./tools/uncrustify and fail if that touches any files.

The original Liquid deployment had a hand-rolled fedpegscript, and the decision at some point was to not tweak the "emergency" keys in the CSV clause. Moving forward with dynamic federations we will need to make this less templated, which means allowing these 33 byte pushes to be tweaked as well.

see: https://github.com/ElementsProject/elements/blob/master/src/pegins.cpp#L84

Can I use this library to have native Scrypt support in my android app?
If yes, what should be done to enable it as by default bitcoinjs native_library_loaded = loader.load("scrypt", true); returns false and uses slow java version of Scrypt when I have libscrypt.so replaced with libwallycore.so.

Thanks!

The new PSBT methods seem to have code documentation, but they're not showing up at https://wally.readthedocs.io/en/release_0.7.5

It might be a matter of adding "psbt" here:

cc @achow101

On macOS 10.14.5 with Python 3.7.3 and Swig 4.0.0 installed via Homebrew.

./configure --enable-debug --enable-export-all --enable-swig-python
make
make check

A bunch of tests pass and then:

Ran 1 test in 0.001s

OK
Traceback (most recent call last):
  File "swig_python/contrib/bip32.py", line 3, in <module>
    from wallycore import *
  File "/Users/sjors/dev/libwally-core/src/swig_python/wallycore/__init__.py", line 13, in <module>
    from . import libwallycore
ImportError: cannot import name 'libwallycore' from 'wallycore' (/Users/sjors/dev/libwally-core/src/swig_python/wallycore/__init__.py)
make[3]: *** [check-local] Error 1
make[2]: *** [check-am] Error 2
make[1]: *** [check-recursive] Error 1
make: *** [check-recursive] Error 1

I noticed some commits referring to Swig 3.0, so maybe that's the issue? Unfortunately if I use [email protected] then make fails with Error: SWIG is not installed.

This also means I can't generate coverage reports.

Hey,

It seems irresponsible to default to -O3 which is known to use "experimental" optimizations that sometimes even pessimize the build. Did you performance check if there is a significant difference between -O2 and -O3? Did you also check this for platforms like arm?

On the same note, our project is building libwally-core for embedded and we would like to use -Os. But this is explicitly not respected.

I think the motivation to disallow the correct optimizations is invalid. Optimization flags are always given on a project scope and not per library. You anyway can't control if the final binary is compiled with LTO for example.

My feature request is thus: Do not add optimization flags in case CFLAGS is set at configure time. I.e. I would want ./configure CFLAGS="-Os" to be respected.

When I ran "./configure" in mac,a error happened.
$ ./configure
checking build system type... x86_64-apple-darwin17.4.0
checking host system type... x86_64-apple-darwin17.4.0
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... tools/build-aux/install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking how to print strings... printf
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /Library/Developer/CommandLineTools/usr/bin/ld
checking if the linker (/Library/Developer/CommandLineTools/usr/bin/ld) is GNU ld... no
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 196608
checking how to convert x86_64-apple-darwin17.4.0 file names to x86_64-apple-darwin17.4.0 format... func_convert_file_noop
checking how to convert x86_64-apple-darwin17.4.0 file names to toolchain format... func_convert_file_noop
checking for /Library/Developer/CommandLineTools/usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @file support... no
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /bin/dd
checking how to truncate binary pipes... /bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for dsymutil... dsymutil
checking for nmedit... nmedit
checking for lipo... lipo
checking for otool... otool
checking for otool64... no
checking for -single_module linker flag... yes
checking for -exported_symbols_list linker flag... yes
checking for -force_load linker flag... yes
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... yes
checking for gcc option to produce PIC... -fno-common -DPIC
checking if gcc PIC flag -fno-common -DPIC works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/Library/Developer/CommandLineTools/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin17.4.0 dyld
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no
checking whether make supports nested variables... (cached) yes
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
checking for libtool... /usr/bin/libtool
checking for gcc-ar... (cached) /usr/bin/libtool
checking for ar... (cached) /usr/bin/libtool
checking for gcc-ranlib... no
checking for ranlib... /usr/bin/ranlib
checking for gsed... no
configure: error: gsed must be available to build this library

depth is declared as uint8_t in the struct:

https://github.com/ElementsProject/libwally-core/blob/master/include/wally_bip32.h#L47

bip32_key_init_alloc's arg is uint32_t:

https://github.com/ElementsProject/libwally-core/blob/master/src/bip32.c#L619

There is no check on the range and the struct field is set directly:

https://github.com/ElementsProject/libwally-core/blob/master/src/bip32.c#L659

Should the arg type be changed to uint8_t?

At least trezor and electrum support SLIP132 base58 prefixes for extended keys, which indicate the script type to be derived.

I think it could be worth adding a flags parameter to bip32_key_from_base58[,_alloc], otherwise could expose the behavior through a separate function and avoid the BC break?

I've been reviewing for common libraries that may not yet have segwit address support. I haven't seen anyone work on it yet, please add this soon!

https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki
See also: https://github.com/sipa/bech32/tree/master/ref/c

See ElementsProject/elements@0f28780

The bech32 address format has pretty neat error detection functions that could really improve a wallet user experience. See it in action here.

We could add something like:

`wally_validate_bech32(const char *addr, const char *addr_family, uint32_t flags, unsigned char *bytes_out, size_t len, size_t *written

where bytes_out is be an array with the position(s) of invalid characters.

I'm not sure how this would be implemented.

In particular, my tests complain about a memory leak because you allocate a secp ctx. Unf. that allocator isn't switchable, so I can't even hack it that way.

The BIP39 wordlists are selected so that the first 4 letters of each word are unique. The functions for working with mnemonics should accept them in that form, and expand them to the full words.

This is important because tools like the CryptoSteel / BillFodl metal key storage devices only accept the first four letters of each word, and making the user hand-expand that back to the whole word is painful and unnecessary.

Screwing around with the input in this way playing guess-what-I-mean is fine because of the checksum. As long as the checksum passes, we know we performed the correct expansion. (In light of this I think we should also offer user-friendly things like automatic casefolding to the same case as the dictionary. All of this should only be triggered after trying the entered string as-is and failing the checksum in that form.)

The code internally already has methods that detect serialization formats, but it is not exposed anywhere. It'd be nice to be able to peek at a serialized transaction and determine its serialization type.

Currently the only alternative is to know ahead of time, or first try one method, and if that fails try the other, which is a bit cumbersome.

For an example of this see this commit: we have to pass in is_elements in the wire message before parsing the transaction included in the same message so we can set that global variable and parse correctly.

Both clangs static analyzer and some broken gcc version get tripped up by the memory shuffling this flag requires, as it causes memcpys across members within the same struct.

Both should be fixed by copying the data out to a temporary array, poking the values back in the correct order and securely nuking the temp buffer.

See ElementsProject/lightning#1961 for an example of gcc b0rking on this.

In embedded, it's preferable to use stack vars over malloc.

See ElementsProject/lightning#695

I am unsure why 16 cosigners are allowed, as only up to 15 can be safely used in P2SH. Is this a typo? The unit tests do not cover the upper limits.

The function should limit it to 15, or drop the limit check and document that the user has to make sure to check the limits depending on the context the script is used in.

Some calls, e.g., wally_tx_get_length, will fail if we specify that we use elements when calling them, leading the caller to have to mask out the bits in the call.

An example of this can be found here:

https://github.com/ElementsProject/lightning/blob/22ea458298238dcf23dfb1dffae9ad8b9725b94b/bitcoin/tx.c#L325-L329

It'd be nice to perform this masking inside the function itself, in order to expose a more consistent interface. It could also verify that the flag matches with the internal detection of elements transactions and return an error if they don't match (though that'd be a breaking API change).

It's currently non-trivial to verify that the code in src/secp256k1 matches upstream. A git submodule or subtree is probably the simplest way to fix that. If you need any patches, have a script apply those.

See also #11 which goes further and argues for completely removing it in favor of a system library, but that doesn't seem practical.

There may be other solutions.

wrap_js does not install/build. This may be an issue with my configuration.

Install libwally-core as:

./tools/autogen.sh
./configure --enable-js-wrappers --disable-swig-python --disable-swig-java --enable-elements --enable-debug
make

When I require(..../libwally-core/src/wrap_js) and call via Node everything works fine. But when including this install in an npm package.json, or simply calling npm install libwally-core/src/wrap_js\, the following error occurs:

[email protected] install /Users/tomos/lava-wallet/libwally-core/src/wrap_js
> node-gyp configure && node-gyp build
 CC(target) Release/obj.target/deps/src/combined.o
../src/combined.c:2:10: fatal error: ‘internal.c’ file not found
#include “internal.c”
     ^~~~~~~~~~~~
1 error generated.
make: *** [Release/obj.target/deps/src/combined.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack   at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack   at ChildProcess.emit (events.js:321:20)
gyp ERR! stack   at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)
gyp ERR! System Darwin 19.3.0
gyp ERR! command “/usr/local/Cellar/node/13.8.0/bin/node” “/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js” “build”
gyp ERR! cwd /Users/tomos/lava-wallet/libwally-core/src/wrap_js
gyp ERR! node -v v13.8.0
gyp ERR! node-gyp -v v5.0.5
gyp ERR! not ok 
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node-gyp configure && node-gyp build`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR!   /Users/tomos/.npm/_logs/2020-02-13T12_06_21_382Z-debug.log

MacOs Catalina 10.15.3
Node v13.8.0
npm v6.13.6

I'm surprised it works with any gcc, in fact:

  CC                      = gcc-4.8
  CFLAGS                  = -g -W -std=c89 -pedantic -Wall -Wextra -Wcast-align -Wnested-externs -Wshadow -Wstrict-prototypes -Wno-unused-function -Wno-long-long -Wno-overlength-strings -fvisibility=hidden -O3

Note the -std=c89 (which AFAICT comes from libsecp):

Sure enough, libwally is not C89. For example:

../../libwally-core/src/address.c: In function ‘wally_bip32_key_to_address’:
../../libwally-core/src/address.c:25:5: warning: C++ style comments are not allowed in ISO C90 [enabled by default]
     // Catch known incorrect combinations of address type and version:
     ^
../../libwally-core/src/address.c:25:5: warning: (this will be reported only once per input file) [enabled by default]
../../libwally-core/src/elements.c: In function ‘wally_asset_pak_whitelistproof’:
../../libwally-core/src/elements.c:629:5: error: ‘for’ loop initial declarations are only allowed in C99 mode
     for (size_t i = 0; i < num_keys; ++i) {
     ^
../../libwally-core/src/elements.c:629:5: note: use option -std=c99 or -std=gnu99 to compile your code
../../libwally-core/src/sign.c: In function ‘wally_ec_sig_to_public_key’:
../../libwally-core/src/sign.c:324:5: warning: ISO C90 forbids mixed declarations and code [-Wpedantic]
     int recid = (sig[0] - 27) & 3;
     ^

It would be nice for libwally to support BIP 174 PSBTs.

HI! I built libwally for iOS/ObjC passing the --enable-elements flag using 0.6.8 commit.

Once static libs and header included in a iOS project, I cannot reach wally_confidential_addr_to_addr, wally_confidential_addr_to_ec_public_key, wally_confidential_addr_from_addr

If I try to run wally_is_elements_build it returns 1, saying elements have been included correctly.

What I'm doing wrong?

We currently manually manipulate the arrays to order the transactions, but it'd be nicer if we could add in the correct place, rather than always appending.