Giter Club home page Giter Club logo

cava's People

Contributors

atoulme avatar cleishm avatar happy0 avatar jrhea avatar ligi avatar schroedingerscode avatar vitorpy avatar zilm13 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cava's Issues

Periodic sodium OutOfMemory exception when consuming scuttlebutt streams

When consuming a scuttlebutt stream, there are periodic sodium OutOfMemory exception errors while decrypting the messages received as responses from the server for random messages received:

2019-04-02 09:37:20.086+0000 ERROR [localhost:8008] Sodium.sodium_malloc failed allocating 24
java.lang.OutOfMemoryError: Sodium.sodium_malloc failed allocating 24
	at net.consensys.cava.crypto.sodium.Sodium.malloc(Sodium.java:224)
	at net.consensys.cava.crypto.sodium.Sodium.dup(Sodium.java:262)
	at net.consensys.cava.crypto.sodium.Sodium.dup(Sodium.java:273)
	at net.consensys.cava.crypto.sodium.SecretBox$Nonce.fromBytes(SecretBox.java:230)
	at net.consensys.cava.crypto.sodium.SecretBox$Nonce.fromBytes(SecretBox.java:213)
	at net.consensys.cava.scuttlebutt.handshake.SecureScuttlebuttStream.decryptMessage(SecureScuttlebuttStream.java:107)
	at net.consensys.cava.scuttlebutt.handshake.SecureScuttlebuttStream.decrypt(SecureScuttlebuttStream.java:85)
	at net.consensys.cava.scuttlebutt.handshake.SecureScuttlebuttStream.readFromServer(SecureScuttlebuttStream.java:53)
	at net.consensys.cava.scuttlebutt.handshake.vertx.SecureScuttlebuttVertxClient$NetSocketClientHandler.handle(SecureScuttlebuttVertxClient.java:98)
	at io.vertx.core.net.impl.NetSocketImpl$DataMessageHandler.handle(NetSocketImpl.java:392)
	at io.vertx.core.streams.impl.InboundBuffer.handleEvent(InboundBuffer.java:225)
	at io.vertx.core.streams.impl.InboundBuffer.write(InboundBuffer.java:123)
	at io.vertx.core.net.impl.NetSocketImpl.handleMessage(NetSocketImpl.java:370)
	at io.vertx.core.net.impl.ConnectionBase.handleRead(ConnectionBase.java:386)
	at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:320)
	at io.vertx.core.impl.EventLoopContext.execute(EventLoopContext.java:43)
	at io.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:188)
	at io.vertx.core.net.impl.VertxHandler.channelRead(VertxHandler.java:174)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:579)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:496)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)

Steps to reproduce:

  1. Check out the branch underlying https://github.com/ConsenSys/cava/pull/202/files
  2. Modify the postMessageTest test in the PatchworkIntegrationText file in the mux test package to post 20,000 messages by changing the for loop to iterate more and run the test. I'm not sure the minimum number of items in a stream required to make the error happen.
  3. Run the streamTest test in the PatchworkIntegrationTest file in the mux package.
  4. Observe the above error is occasionally output in the log.

java.lang.IllegalStateException: BouncyCastleProvider is not available

Hi there,

I hava added cava to my project, add then I used the SECP256K1 class to sign my data, but some errors happened. Is there anyone met the problem and tells me why? thx

The error message is like this:

Exception in thread "main" java.lang.ExceptionInInitializerError
	at net.consensys.cava.crypto.SECP256K1$PublicKey.fromSecretKey(SECP256K1.java:527)
	at net.consensys.cava.crypto.SECP256K1$KeyPair.fromSecretKey(SECP256K1.java:684)
	at one.contentbox.boxd.samples.SignSample.main(SignSample.java:40)
Caused by: java.lang.IllegalStateException: BouncyCastleProvider is not available, see https://www.bouncycastle.org/wiki/display/JA1/Provider+Installation
	at net.consensys.cava.crypto.SECP256K1$Parameters.<clinit>(SECP256K1.java:111)
	... 3 more
Caused by: java.security.NoSuchProviderException: no such provider: BC
	at sun.security.jca.GetInstance.getService(GetInstance.java:83)
	at sun.security.jca.GetInstance.getInstance(GetInstance.java:206)
	at java.security.KeyPairGenerator.getInstance(KeyPairGenerator.java:279)
	at net.consensys.cava.crypto.SECP256K1$Parameters.<clinit>(SECP256K1.java:109)
	... 3 more

My code is like this:

String privKeyHex = "29fbf01166fc31c941cadc1659a5f684f81c22c1113e5aa5b0af28b7dd453269";
BigInteger privKeyBigInteger = new BigInteger(privKeyHex, 16);
String data = "46f7f67f515e7d053525459991c7a6b3e673950809af776324f46d91df9e600d";

 SECP256K1.Signature signature = SECP256K1
                .sign(Hex.decode(data), SECP256K1.KeyPair.
                        fromSecretKey(SECP256K1.SecretKey.fromInteger(privKeyBigInteger)));

byte[] data  = signature.bytes().toArray();

Support for java.util.List in SSZWriter in addition to varargs

Currently SSZWriter supports serializing a number of single values as well as an [object]... varags list. While this works for single elements and primitive arrays, a java.util.List must be converted into a primitive array first. Native support of java.util.List types would be a nice to have.

scuttlebutt-handshake implementation assumes 1 full message is read at once

I don't have steps to reproduce this, but I think it might be an issue (from reading the code.)

A scuttlebutt RPC message has a 34 bit header, followed by a variable length body. When bytes are received over the socket, the current implementation of scuttlebutt-handshake assumes that it is a complete message in the byte buffer handed over by Vertx.

https://github.com/ConsenSys/cava/blob/e8ea2b897a2ff0a0ed4ae9c47300583b0f0874dc/scuttlebutt-handshake/src/main/java/net/consensys/cava/scuttlebutt/handshake/vertx/SecureScuttlebuttVertxClient.java#L62

Depending on network conditions, application load, etc, this could theoretically be only part of a message in this buffer (or perhaps even 1 RPC response and a little bit of another.) Perhaps this could be re-implemented to make sure we buffer up full messages first before handing over the bytes to the receivedMessage method.

Expose SSZ#merkleHash to Allow Callers to Implement Signed Roots

Signed Roots allow calculating the merkle hash of a subset of a SSZ container, specifically to allow signing of a container while excluding signature fields.

While this definition signed_root(container, "signature") = merkle_hash([hash_tree_root(sub_object_1), ..., hash_tree_root(sub_object_N)]) has been deprecated in further versions of the Eth2 SSZ spec, I see no harm in allowing public callers to use SSZ#merkleHash directly for compatibility.

Mirror JCenter artifacts to Maven Central

Makes it easier for the non-Gradle crowd to use your library.

I put this off a long time for my personal projects because I believed it was a hassle, but finally I went though the steps recently and it's really not that bad. It's basically just creating a Jira account and filing some tickets to claim ownership of the artifact ID, then pushing the sync button on the JCenter site.

https://central.sonatype.org/pages/ossrh-guide.html

SSZ support for endianness

There is no support for setting endianness at this time in the SSZ library.
The spec was modified to allow endianness when serializing individual values.

Toml library fails its own tests

> Task :toml:test

net.consensys.cava.toml.TomlTest > shouldParseString(String, String)[3] FAILED
    org.opentest4j.AssertionFailedError at TomlTest.java:81

net.consensys.cava.toml.TomlTest > shouldParseString(String, String)[8] FAILED
    org.opentest4j.AssertionFailedError at TomlTest.java:81

net.consensys.cava.toml.TomlTest > shouldParseString(String, String)[23] FAILED
    org.opentest4j.AssertionFailedError at TomlTest.java:81

net.consensys.cava.toml.TomlTest > testHardExampleUnicode() FAILED
    org.opentest4j.AssertionFailedError at TomlTest.java:512

net.consensys.cava.toml.TomlTest > shouldParseArray(String, Object[])[8] FAILED
    org.opentest4j.AssertionFailedError at TomlTest.java:555

212 tests completed, 5 failed

> Task :toml:test FAILED

FAILURE: Build failed with an exception.

TOML: Missing dependency on Antlr4 runtime

Hi! Really nice TOML parsing library -- thanks for sharing it under a business-friendly license. I especially like the detailed messages for syntax validation errors. I thought I should report an issue I ran into while getting started.

In a project that uses net.consensys.cava:cava-toml:0.2.0, parsing a TOML config file fails with the following exception unless the Antlr4 runtime is added as an explicit dependency:

Exception in thread "main" java.lang.NoClassDefFoundError: org/antlr/v4/runtime/CharStream
	at com.couchbase.connector.elasticsearch.Config.main(Config.java:147)
Caused by: java.lang.ClassNotFoundException: org.antlr.v4.runtime.CharStream
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 1 more

The workaround is to manually add a dependency on the Antlr4 runtime. For gradle, this looks like

compile 'net.consensys.cava:cava-toml:0.2.0'
compile 'org.antlr:antlr4-runtime:4.7.1'

It would be nice if the Maven POM for the cava-toml library expressed a transitive dependency on the Antlr4 runtime. Better yet, embed the runtime in the cava-toml library as a shaded/shadowed dependency.

Support "tuples" and "lists" from SSZ Spec v0.5.1

@atoulme and/or @cleishm, if you wouldn't mind giving a little thought to the below proposal, I'd be most appreciative. I've spent a little bit mulling this over and plan to take a first stab at a PR shortly, but in the meantime an extra set of eyes doesn't hurt.

Description
The SSZ spec was rewritten as part of the eth2-specs v0.5.1 release to include both fixed-length and variable-length arrays/lists. Following the python notation used in the eth2-specs, and for the sake of clarity, fixed-length arrays/lists will be referred to as "tuples" and variable-length array/lists will be referred to as "lists".

(Not so) Quick Background

  • Firstly, the SSZ spec has a limited number of base types:
    • uintN (where N is in 8, 16, 32, 64, 128, or 256)
      a N bit unsigned integer
    • bool
      A Boolean (True or False)
    • byte == uint8
      byte is an alias for uint8
  • Composite types from these include:
    • tuple, represented as [type, N]
      a fixed-length homogenous collection of N values
    • list, represented as [type]
      a variable-length homogenous collection of values
  • Some derived values, therefore are:
    • bytes == ["byte"]
      A variable-length collection of byte types (uint8 types by extension)
    • bytesN == ["byte", N]
      A fixed-length collection of byte types with N length

The implications of the above are that "list" requires a length mixin, where as a "tuple" does not.

Proposed Changes

  • When Java bytes are serialized (i.e. when recursively serializing a Cava Bytes type), they should be serialized as a uint8.
  • A Cava Bytes type of length 1 should be serialized as a uint8. [still undecided on this one]
  • A Cava Bytes type of length >1 should be treated and serialized as a "list".
  • All fixed length Cava BytesN types (i.e. Bytes32) should be treated and serialized as a "tuple".
  • Support will be added for serializing Bytes... (varags list) as either a "list" or a "tuple"
    • The proposed method of achieving this is overloading the relevant functions with an additional length parameter.
  • Support will be added for serializing List<? extends Bytes> as either a "list" or a "tuple".
    • The proposed method of achieving this is overloading the relevant functions with an additional length parameter.

Additional Information
See https://github.com/ethereum/eth2.0-specs/blob/v0.5.1/specs/simple-serialize.md.

Cannot decrypt response header for scuttlebutt RPC request

Hi - thanks for working on these Java libraries for using the scuttlebutt protocol!

I have encountered a bug while attempting to use the scuttlebutt-handshake library to

  • Connect to a scuttlebot instance and do the secret handshake, using the same public and private key that the scuttlebot server is running with.
  • Send an RPC command to the server and receive a response. I chose the simplest command I know of, which is whoami which simply reports back the server's public key. I also tried createFeedStream by copying and pasting the body from the protocol docs and using a key for someone the scuttlebot instance is following.

Expected behaviour

  • My ClientHandler implementation's receivedMessage should be called back with the response bytes from the RPC request.

Actual behaviour

Steps I used to reproduce:

  • Check out commit c01f2e5745d8f2a1296027f9bda98ad9ac2aed3d of cava (which was the latest for me at the time - i didn't seem to be able to get the latest from the Maven repos.)

  • Run gradle build to create jar files for the libraries.

  • Init a gradle project and copy the libs into a 'lib' directory and add the libs directory as a dependency

  • Define a main class:

*
 * This Java source file was generated by the Gradle 'init' task.
 */
package cava.test;

import io.vertx.core.Vertx;
import net.consensys.cava.bytes.Bytes;
import net.consensys.cava.bytes.Bytes32;
import net.consensys.cava.concurrent.AsyncCompletion;
import net.consensys.cava.scuttlebutt.handshake.vertx.SecureScuttlebuttVertxClient;

import net.consensys.cava.crypto.sodium.Signature;
import net.consensys.cava.scuttlebutt.rpc.RPCCodec;
import net.consensys.cava.scuttlebutt.rpc.RPCFlag;

import java.util.Base64;

public class App {

    public static void main(String[] args) throws Exception {

        int port = 8008;

        String host = "localhost";

        String publicKeyBase64 = System.getenv("scuttle_public_key");
        String privateKeyBase64 = System.getenv("scuttle_private_key");

        if (publicKeyBase64 == null) {
            throw new Exception("scuttle_public_key environment variable missing");
        }

        if (privateKeyBase64 == null) {
            throw new Exception("scuttle_private_key environment variable missing");
        }

        byte[] publicKeyBytes = Base64.getDecoder().decode(publicKeyBase64);
        byte[] privateKeyBytes = Base64.getDecoder().decode(privateKeyBase64);

        Signature.PublicKey publicKey = Signature.PublicKey.fromBytes(publicKeyBytes);
        Signature.SecretKey privateKey = Signature.SecretKey.fromBytes(privateKeyBytes);

        Signature.KeyPair keyPair = new Signature.KeyPair(publicKey, privateKey);

        Vertx vertx = Vertx.vertx();

        // The default scuttlebutt network key (the one Patchwork uses.)
        String networkKeyBase64 = "1KHLiKZvAvjbY1ziZEHMXawbCEIM6qwjCDm3VYRan/s=";
        byte[] networkKeyBytes = Base64.getDecoder().decode(networkKeyBase64);

        Bytes32 networkKeyBytes32 = Bytes32.wrap(networkKeyBytes);

        SecureScuttlebuttVertxClient secureScuttlebuttVertxClient =
                new SecureScuttlebuttVertxClient(vertx, keyPair, networkKeyBytes32);

        AsyncCompletion onConnect = secureScuttlebuttVertxClient.connectTo(port, host, publicKey, (senderFn, stopFn) -> {

            // Tell the client handler how to send bytes to the server
            MyClientHandler clientHandler = new MyClientHandler(senderFn, stopFn);

            // An RPC command that just tells us our public key (like ssb-server whoami on the command line.)
            String rpcRequestBody = "{\n" +
                    "  \"name\": [\"whoami\"],\n" +
                    "  \"type\": \"async\" " +
                    "}";
            Bytes rpcRequest = RPCCodec.encodeRequest(rpcRequestBody, RPCFlag.BodyType.JSON);

            System.out.println("Attempting RPC request...");
            clientHandler.sendMessage(rpcRequest);

            // We hand over a ClientHandler so we get called back when new bytes arrive
            return clientHandler;
        });

    }
}

Define a ClientHandler implementation


package cava.test;

import net.consensys.cava.bytes.Bytes;
import net.consensys.cava.scuttlebutt.handshake.vertx.ClientHandler;

import java.util.function.Consumer;

public class MyClientHandler implements ClientHandler {

    private final Consumer<Bytes> sender;
    private final Runnable terminationFn;

    public MyClientHandler(Consumer<Bytes> sender, Runnable terminationFn) {
        this.sender = sender;
        this.terminationFn = terminationFn;
    }

    @Override
    public void receivedMessage(Bytes message) {

        System.out.println("We received a message?");

    }

    @Override
    public void streamClosed() {

        System.out.println("Stream closed?");

    }

    void sendMessage(Bytes bytes) {
        System.out.println("Sending message?");
        sender.accept(bytes);
    }

    void closeStream() {
        terminationFn.run();
    }
}
  • Set the public and private keys as the scuttle_public_key and scuttle_private_key environment variables by copying the base64 representations from the secret file in the .ssb directory (minus the .ed25519 suffix.)

  • Notice it doesn't get called back with bytes for the response, and step through debug to see it throwing a NullPointerException while trying to decrypt the header from the RPC response

I'm trying to debug the issue at the moment (i wonder if there's something strange going on with the keys), but I wondered if you had any ideas?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.