nymtech / nym Goto Github PK

Nym provides strong network-level privacy against sophisticated end-to-end attackers, and anonymous transactions using blinded, re-randomizable, decentralized credentials.

Home Page: https://nymtech.net

Rust 72.28% Shell 0.68% HTML 0.71% JavaScript 2.08% Go 3.06% Python 0.33% TypeScript 19.37% CSS 0.06% Dockerfile 0.01% Makefile 0.12% C 0.37% Roff 0.02% MDX 0.85% Handlebars 0.02% C++ 0.04%

nym's Introduction

The Nym Privacy Platform

The platform is composed of multiple Rust crates. Top-level executable binary crates include:

nym-mixnode - shuffles Sphinx packets together to provide privacy against network-level attackers.
nym-client - an executable which you can build into your own applications. Use it for interacting with Nym nodes.
nym-socks5-client - a Socks5 proxy you can run on your machine and use with existing applications.
nym-gateway - acts sort of like a mailbox for mixnet messages, which removes the need for direct delivery to potentially offline or firewalled devices.
nym-network-monitor - sends packets through the full system to check that they are working as expected, and stores node uptime histories as the basis of a rewards system ("mixmining" or "proof-of-mixing").
nym-explorer - a (projected) block explorer and (existing) mixnet viewer.
nym-wallet - a desktop wallet implemented using the Tauri framework.

Building

Platform build instructions are available on our docs site. Wallet build instructions are also available on our docs site.

Developing

There's a .env.sample-dev file provided which you can rename to .env if you want convenient logging, backtrace, or other environment variables pre-set. The .env file is ignored so you don't need to worry about checking it in.

For Typescript components, please see ts-packages.

Developer chat

We used to use Keybase for developer chats, but we have since migrated to Matrix and Discord. We no longer check the old nymtech.friends Keybase team.

You can chat to us in two places:

The #dev channel on Matrix
The various developer channels on Discord

Rewards

Node, node operator and delegator rewards are determined according to the principles laid out in the section 6 of Nym Whitepaper. Below is a TLDR of the variables and formulas involved in calculating the epoch rewards. Initial reward pool is set to 250 million Nym, making the circulating supply 750 million Nym.

Symbol	Definition
$R#gh-light-mode-only$ $\color{white}R#gh-dark-mode-only$	global share of rewards available, starts at 2% of the reward pool.
$R_{i}#gh-light-mode-only$ $\color{white}R_{i}#gh-dark-mode-only$	node reward for mixnode `i`.
$\sigma_{i}#gh-light-mode-only$ $\color{white}\sigma_{i}#gh-dark-mode-only$	ratio of total node stake (node bond + all delegations) to the token circulating supply.
$\lambda_{i}#gh-light-mode-only$ $\color{white}\lambda_{i}#gh-dark-mode-only$	ratio of stake operator has pledged to their node to the token circulating supply.
$\omega_{i}#gh-light-mode-only$ $\color{white}\omega_{i}#gh-dark-mode-only$	fraction of total effort undertaken by node `i`, set to `1/k`.
$k#gh-light-mode-only$ $\color{white}k#gh-dark-mode-only$	number of nodes stakeholders are incentivised to create, set by the validators, a matter of governance. Currently determined by the `reward set` size, and set to 720 in testnet Sandbox.
$\alpha#gh-light-mode-only$ $\color{white}\alpha#gh-dark-mode-only$	A Sybil attack resistance parameter - the higher this parameter is set, the stronger the reduction in competitiveness for a Sybil attacker.
$PM_{i}#gh-light-mode-only$ $\color{white}PM_{i}#gh-dark-mode-only$	declared profit margin of operator `i`, defaults to 10%.
$PF_{i}#gh-light-mode-only$ $\color{white}PF_{i}#gh-dark-mode-only$	uptime of node `i`, scaled to 0 - 1, for the rewarding epoch
$PP_{i}#gh-light-mode-only$ $\color{white}PP_{i}#gh-dark-mode-only$	cost of operating node `i` for the duration of the rewarding epoch, set to 40 NYMs.

Node reward for node i is determined as:

$R_{i}=PF_{i} \cdot R \cdot (\sigma^'_{i} \cdot \omega_{i} \cdot k + \alpha \cdot \lambda^'_{i} \cdot \sigma^'_{i} \cdot k)/(1 + \alpha)#gh-light-mode-only$

$\color{white}R_{i}=PF_{i} \cdot R \cdot (\sigma^'_{i} \cdot \omega_{i} \cdot k + \alpha \cdot \lambda^'_{i} \cdot \sigma^'_{i} \cdot k)/(1 + \alpha)#gh-dark-mode-only$

where:

$\sigma^'_{i} = min\{\sigma_{i}, 1/k\}#gh-light-mode-only$

$\color{white}\sigma^'_{i} = min\{\sigma_{i}, 1/k\}#gh-dark-mode-only$

and

$\lambda^'_{i} = min\{\lambda_{i}, 1/k\}#gh-light-mode-only$

$\color{white}\lambda^'_{i} = min\{\lambda_{i}, 1/k\}#gh-dark-mode-only$

Operator of node i is credited with the following amount:

$min\{PP_{i},R_{i})\} + max\{0, (PM_{i} + (1 - PM_{i}) \cdot \lambda_{i}/\delta_{i}) \cdot (R_{i} - PP_{i})\}#gh-light-mode-only$

$\color{white}min\{PP_{i},R_{i})\} + max\{0, (PM_{i} + (1 - PM_{i}) \cdot \lambda_{i}/\delta_{i}) \cdot (R_{i} - PP_{i})\}#gh-dark-mode-only$

Delegate with stake s receives:

$max\{0, (1-PM_{i}) \cdot (s^'/\sigma_{i}) \cdot (R_{i} - PP_{i})\}#gh-light-mode-only$

$\color{white}max\{0, (1-PM_{i}) \cdot (s^'/\sigma_{i}) \cdot (R_{i} - PP_{i})\}#gh-dark-mode-only$

where s' is stake s scaled over total token circulating supply.

Licensing and copyright information

This is a monorepo and components that make up Nym as a system are licensed individually, so for accurate information, please check individual files.

As a general approach, licensing is as follows this pattern:

applications and binaries are GPLv3
libraries and components are Apache 2.0 or MIT
documentation is Apache 2.0 or CC0-1.0

Again, for accurate information, please check individual files.

nym's People

Contributors

Stargazers

Watchers

Forkers

juliointhestudio narodnik sahith-narahari wissemriahi zatoichi-labs frankhorv hyperfekt gyrusdentatus keviinfoes timkuijsten inspectordidi sgeisler benk31 purplesparkle confio privyack ness-network stguller kqryptic lnceballosz arno01 legogris deyure2u immense055 habibrr huitseeker interlunar weiribao j56320 tidealhub ethancreeger evilebottnawi userid9708 mt2721 papyrus75 shenyezh rickau123 topcryptoinvestor tommyv1987 jason0bourne bamaao shark-o danielementary taras-miner ntwrkrcrt pasha030687 kundouzhishou 2g4y1 jerzybrzoska yupuitsuens jacobjohansen riccardomasutti martinyung taf-ins flightlesseagle alan45325181 tonycrypto44 lgs duong2024 phamhung8989 amit-chainflux joingithubstyle gophersecurity sts0mrg0 maxvoyakin errorist79 iamfittz nathanganser chengfai subgraph-pro kolyavasin15 membrane7 ckocaogullar compnetzero-ai dev-protocol hofmann25 tlietz tilslava testlool somecookie elliot-u410 chirag-bgh faretingolin vladrofl crypto-zone valyriaeth geniusxprime rachyandco satman81 notrustverify erkushka iamonuwa sagar-a16z kaisbaccour nishshoko kinalartur gdanezis monikacat witty66cc pinkdiamond1

nym's Issues

Add version number to presence

It would be nice to see what version each current mixnode is running.

Add version number to presence

Knowing what version of the provider is being run would be quite helpful in certain situations.

Messages returned by fetch are base64 encoded

Ping sent: {'type': 'send', 'message': 'ping', 'recipient_address': 'kauuj71-RPvETjz8FMQugnsNSDJ8033E4lNS_anMFD0='}
Ping address = kauuj71-RPvETjz8FMQugnsNSDJ8033E4lNS_anMFD0=
Pong sent: {'type': 'fetch'}
Pong received messages: []
Pong sent: {'type': 'fetch'}
Pong received messages: []
Pong sent: {'type': 'fetch'}
Pong received messages: ['cGluZw==']
Pong sent: {'type': 'fetch'}
Pong received messages: []
Pong sent: {'type': 'fetch'}
Pong received messages: []
Pong sent: {'type': 'fetch'}
Pong received messages: []
Pong sent: {'type': 'fetch'}

Code is here:

import asyncio
import json
import websockets

async def ping():
    pong_address = "kauuj71-RPvETjz8FMQugnsNSDJ8033E4lNS_anMFD0="
    uri = "ws://127.0.0.1:9001/mix"

    async with websockets.connect(uri) as websock:
        request_object = {
            "type": "ownDetails"
        }
        await websock.send(json.dumps(request_object))
        response = json.loads(await websock.recv())
        print("Ping address =", response["address"])

        request_object = {
            "type": "send",
            "message": "ping",
            "recipient_address": pong_address
        }
        await websock.send(json.dumps(request_object))
        print("Ping sent:", request_object)

async def pong():
    ping_address = "lzdJ2EkCHlPFVLdsBCkW_n6thaql2wImrGMG0gt_EX4="
    uri = "ws://127.0.0.1:9002/mix"

    async with websockets.connect(uri) as websock:
        request_object = {
            "type": "ownDetails"
        }
        await websock.send(json.dumps(request_object))
        response = json.loads(await websock.recv())
        print("Ping address =", response["address"])

        while True:
            request_object = {
                "type": "fetch"
            }
            await websock.send(json.dumps(request_object))
            print("Pong sent:", request_object)
            response = json.loads(await websock.recv())
            messages = response["messages"]
            print("Pong received messages:", messages)

            if not messages:
                await asyncio.sleep(1)

def main():
    asyncio.get_event_loop().run_until_complete(asyncio.gather(
        ping(), pong()
    ))

if __name__ == "__main__":
    main()

Message should just be a normal string. It should be up to the sender to base64 encode/decode binary data themselves if sending through websocket.

Port run_network.sh from old Go mixnet

It was quite handy to have a way to run a local network for development purposes, port the script from the old Go codebase. It'll need to run the directory locally (or die in spectacular fashion). There's a branch at feature/script/localnet which is partly done.

Make sure to fix the docs for it as well.

Check if RwLock on MixProcessingData is still needed

That data is only ever read - does it need the lock at all? The Arc alone should suffice, I think.

Check layer 1 connectivity at client start

When the Nym client starts up and gets a good topology from the directory server, there may still be a problem (i.e. if the layer 1 mixnodes are using IPv6 and the client can't speak that due to network constraints, packets will not route).

To work around this, at client start, do a simple TCP connectivity check to all layer 1 mixnodes and knock any failures out of the topology.

Provider needs to announce two of its addresses (+ issue of hardcoded port)

Right now we have taken the Go network down entirely, so we can finally deal with https://github.com/nymtech/nym/blob/develop/client/src/clients/provider.rs#L57

WASM version of the Sphinx packet

Solidify TCPSocket on client

Improve TCPSocket on the nym-client, it was done pretty quickly.

Error referring to mismatched types caused by topology

Running on Debian Stretch

When running the cargo test command in the nym directory, I receive this error.

error[E0308]: mismatched types
--> common/topology/src/lib.rs:69:30
|
69 | address: address_bytes,
| ^^^^^^^^^^^^^ expected struct sphinx::route::NodeAddressBytes, found array of 32 elements
|
= note: expected type sphinx::route::NodeAddressBytes
found type [u8; 32]

error: aborting due to previous error

For more information about this error, try rustc --explain E0308.
error: could not compile topology.
warning: build failed, waiting for other jobs to finish...
error: build failed

Stop panicking!

We've been mildly sloppy as we rush to get things running. Sanitize all uses of unwrap and expect to ensure we don't accidentally destroy anything in situations where we expect bad inputs, handling errors gracefully.

Persistent socket connection (Websocket with client)

Persistently store ledger with registered clients and their auth tokens

Check required sfw-provider args

It seems more difficult than I'd expect to initially start the sfw-provider. Should we mark more arguments as required? Can we provide some sensible defaults?

Nym-client refactor

Refactor nym-client to pull apart complex files into modules of code that follow the single-responsibility principle. In tandem, clean up our multi-threading now so that we don't grow a big scheissball.

scripts: run_local_network.sh doesn't die nicely

The CTRL-C trap isn't firing on my machine, so after trying to stop the local devnet, all the processes keep running. A minor impoliteness that it'd be nice to fix.

Replace args with proper config files

At the moment, mixnode keys change on every node restart. This is going to cause friction for clients (especially if they don't refresh periodically) because a restarted node will change its key, causing MacIntegrity errors as clients send un-decryptable packets. Ultimately this breaks the path, a big no-no.

So we might as well store some persistent information on disk.

make an init command for mixnodes and providers
create persistent mixnode keys that survive across restarts
we need to consider whether we need node human names for this

Clean shutdown for all processes

Catch signal interrupts to cleanly shut down client, provider, validators and mixnodes.

Fix incorrectly used Arcs

When we're processing client requests (say in provider), we keep processing data as Arc<some struct> this only happens to work because some struct implements Copy. This should be Arc<RefCell<some struct>> or <Arc<Cell<some struct>> instead.

Create config struct for mixnode (possibly also for client)

Currently only our provider does nice argument parsing to create a proper config (i.e. fn new_config(matches: &ArgMatches) -> provider::Config. It would be nice if mixnode did something similar and we just passed the config itself to create the mix instance.

Persistent socket connection with other mixes

Logging

Add proper loggers to all parts of the Nym codebase.

Nym client crashing and disconnecting local websocket with complaint about binary data

..Received a message from 127.0.0.1:36940: {"type": "fetch"}
.Received a message from 127.0.0.1:36940: {"type": "fetch"}
Received a message from 127.0.0.1:36940: Binary Data<length=4>
thread 'tokio-runtime-worker' panicked at 'Other types of messages are also unsupported!', nym-client/src/sockets/ws.rs:66:18
stack backtrace:
   0: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
   1: core::fmt::write
   2: std::io::Write::write_fmt
   3: std::panicking::default_hook::{{closure}}
   4: std::panicking::default_hook
   5: std::panicking::rust_panic_with_hook
   6: std::panicking::begin_panic
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panicking.rs:400
   7: <nym_client::sockets::ws::ClientRequest as core::convert::From<tungstenite::protocol::message::Message>>::from
             at nym-client/src/sockets/ws.rs:66
   8: <T as core::convert::Into<U>>::into
             at /build/rust/src/rustc-1.40.0-src/src/libcore/convert.rs:547
   9: nym_client::sockets::ws::handle_connection::{{closure}}
             at nym-client/src/sockets/ws.rs:173
  10: <std::future::GenFuture<T> as core::future::future::Future>::poll::{{closure}}
             at /build/rust/src/rustc-1.40.0-src/src/libstd/future.rs:43
  11: std::future::set_task_context
             at /build/rust/src/rustc-1.40.0-src/src/libstd/future.rs:79
  12: <std::future::GenFuture<T> as core::future::future::Future>::poll
             at /build/rust/src/rustc-1.40.0-src/src/libstd/future.rs:43
  13: tokio::task::core::Core<T>::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/core.rs:128
  14: tokio::task::harness::Harness<T,S>::poll::{{closure}}::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/harness.rs:120
  15: core::ops::function::FnOnce::call_once
             at /build/rust/src/rustc-1.40.0-src/src/libcore/ops/function.rs:227
  16: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panic.rs:317
  17: std::panicking::try::do_call
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panicking.rs:287
  18: __rust_maybe_catch_panic
  19: std::panicking::try
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panicking.rs:265
  20: std::panic::catch_unwind
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panic.rs:396
  21: tokio::task::harness::Harness<T,S>::poll::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/harness.rs:101
  22: tokio::loom::std::causal_cell::CausalCell<T>::with_mut
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/loom/std/causal_cell.rs:41
  23: tokio::task::harness::Harness<T,S>::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/harness.rs:100
  24: tokio::task::raw::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/raw.rs:162
  25: tokio::task::raw::RawTask::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/raw.rs:113
  26: tokio::task::Task<S>::run
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/mod.rs:371
  27: tokio::runtime::thread_pool::worker::GenerationGuard::run_task
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/worker.rs:446
  28: tokio::runtime::thread_pool::worker::GenerationGuard::process_available_work
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/worker.rs:304
  29: tokio::runtime::thread_pool::worker::GenerationGuard::run
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/worker.rs:269
  30: tokio::runtime::thread_pool::worker::Worker::run::{{closure}}::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/worker.rs:158
  31: std::thread::local::LocalKey<T>::try_with
             at /build/rust/src/rustc-1.40.0-src/src/libstd/thread/local.rs:262
  32: std::thread::local::LocalKey<T>::with
             at /build/rust/src/rustc-1.40.0-src/src/libstd/thread/local.rs:239
  33: tokio::runtime::thread_pool::worker::Worker::run::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/worker.rs:134
  34: tokio::runtime::thread_pool::current::set::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/current.rs:47
  35: std::thread::local::LocalKey<T>::try_with
             at /build/rust/src/rustc-1.40.0-src/src/libstd/thread/local.rs:262
  36: std::thread::local::LocalKey<T>::with
             at /build/rust/src/rustc-1.40.0-src/src/libstd/thread/local.rs:239
  37: tokio::runtime::thread_pool::current::set
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/current.rs:29
  38: tokio::runtime::thread_pool::worker::Worker::run
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/worker.rs:130
  39: tokio::runtime::thread_pool::Workers::spawn::{{closure}}::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/mod.rs:113
  40: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/blocking/task.rs:30
  41: tokio::task::core::Core<T>::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/core.rs:128
  42: tokio::task::harness::Harness<T,S>::poll::{{closure}}::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/harness.rs:120
  43: core::ops::function::FnOnce::call_once
             at /build/rust/src/rustc-1.40.0-src/src/libcore/ops/function.rs:227
  44: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panic.rs:317
  45: std::panicking::try::do_call
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panicking.rs:287
  46: __rust_maybe_catch_panic
  47: std::panicking::try
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panicking.rs:265
  48: std::panic::catch_unwind
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panic.rs:396
  49: tokio::task::harness::Harness<T,S>::poll::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/harness.rs:101
  50: tokio::loom::std::causal_cell::CausalCell<T>::with_mut
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/loom/std/causal_cell.rs:41
  51: tokio::task::harness::Harness<T,S>::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/harness.rs:100
  52: tokio::task::raw::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/raw.rs:162
  53: tokio::task::raw::RawTask::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/raw.rs:113
  54: tokio::task::Task<S>::run
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/mod.rs:371
  55: tokio::runtime::blocking::pool::run_task
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/blocking/pool.rs:290
  56: tokio::runtime::blocking::pool::Inner::run
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/blocking/pool.rs:206
  57: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/blocking/pool.rs:186
  58: tokio::runtime::context::enter
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/context.rs:72
  59: tokio::runtime::handle::Handle::enter
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/handle.rs:33
  60: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/blocking/pool.rs:185
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread 'tokio-runtime-worker' panicked at 'Failed to forward request: TrySendError { kind: Disconnected }', src/libcore/result.rs:1165:5
stack backtrace:
   0: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
   1: core::fmt::write
   2: std::io::Write::write_fmt
   3: std::panicking::default_hook::{{closure}}
   4: std::panicking::default_hook
   5: std::panicking::rust_panic_with_hook
   6: std::panicking::continue_panic_fmt
   7: rust_begin_unwind
   8: core::panicking::panic_fmt
   9: core::result::unwrap_failed
  10: core::result::Result<T,E>::expect
             at /build/rust/src/rustc-1.40.0-src/src/libcore/result.rs:960
  11: nym_client::sockets::ws::accept_connection::{{closure}}
             at nym-client/src/sockets/ws.rs:232
  12: <std::future::GenFuture<T> as core::future::future::Future>::poll::{{closure}}
             at /build/rust/src/rustc-1.40.0-src/src/libstd/future.rs:43
  13: std::future::set_task_context
             at /build/rust/src/rustc-1.40.0-src/src/libstd/future.rs:79
  14: <std::future::GenFuture<T> as core::future::future::Future>::poll
             at /build/rust/src/rustc-1.40.0-src/src/libstd/future.rs:43
  15: tokio::task::core::Core<T>::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/core.rs:128
  16: tokio::task::harness::Harness<T,S>::poll::{{closure}}::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/harness.rs:120
  17: core::ops::function::FnOnce::call_once
             at /build/rust/src/rustc-1.40.0-src/src/libcore/ops/function.rs:227
  18: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panic.rs:317
  19: std::panicking::try::do_call
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panicking.rs:287
  20: __rust_maybe_catch_panic
  21: std::panicking::try
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panicking.rs:265
  22: std::panic::catch_unwind
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panic.rs:396
  23: tokio::task::harness::Harness<T,S>::poll::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/harness.rs:101
  24: tokio::loom::std::causal_cell::CausalCell<T>::with_mut
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/loom/std/causal_cell.rs:41
  25: tokio::task::harness::Harness<T,S>::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/harness.rs:100
  26: tokio::task::raw::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/raw.rs:162
  27: tokio::task::raw::RawTask::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/raw.rs:113
  28: tokio::task::Task<S>::run
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/mod.rs:371
  29: tokio::runtime::thread_pool::worker::GenerationGuard::run_task
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/worker.rs:446
  30: tokio::runtime::thread_pool::worker::GenerationGuard::process_available_work
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/worker.rs:304
  31: tokio::runtime::thread_pool::worker::GenerationGuard::run
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/worker.rs:269
  32: tokio::runtime::thread_pool::worker::Worker::run::{{closure}}::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/worker.rs:158
  33: std::thread::local::LocalKey<T>::try_with
             at /build/rust/src/rustc-1.40.0-src/src/libstd/thread/local.rs:262
  34: std::thread::local::LocalKey<T>::with
             at /build/rust/src/rustc-1.40.0-src/src/libstd/thread/local.rs:239
  35: tokio::runtime::thread_pool::worker::Worker::run::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/worker.rs:134
  36: tokio::runtime::thread_pool::current::set::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/current.rs:47
  37: std::thread::local::LocalKey<T>::try_with
             at /build/rust/src/rustc-1.40.0-src/src/libstd/thread/local.rs:262
  38: std::thread::local::LocalKey<T>::with
             at /build/rust/src/rustc-1.40.0-src/src/libstd/thread/local.rs:239
  39: tokio::runtime::thread_pool::current::set
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/current.rs:29
  40: tokio::runtime::thread_pool::worker::Worker::run
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/worker.rs:130
  41: tokio::runtime::thread_pool::Workers::spawn::{{closure}}::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/thread_pool/mod.rs:113
  42: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/blocking/task.rs:30
  43: tokio::task::core::Core<T>::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/core.rs:128
  44: tokio::task::harness::Harness<T,S>::poll::{{closure}}::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/harness.rs:120
  45: core::ops::function::FnOnce::call_once
             at /build/rust/src/rustc-1.40.0-src/src/libcore/ops/function.rs:227
  46: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panic.rs:317
  47: std::panicking::try::do_call
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panicking.rs:287
  48: __rust_maybe_catch_panic
  49: std::panicking::try
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panicking.rs:265
  50: std::panic::catch_unwind
             at /build/rust/src/rustc-1.40.0-src/src/libstd/panic.rs:396
  51: tokio::task::harness::Harness<T,S>::poll::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/harness.rs:101
  52: tokio::loom::std::causal_cell::CausalCell<T>::with_mut
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/loom/std/causal_cell.rs:41
  53: tokio::task::harness::Harness<T,S>::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/harness.rs:100
  54: tokio::task::raw::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/raw.rs:162
  55: tokio::task::raw::RawTask::poll
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/raw.rs:113
  56: tokio::task::Task<S>::run
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/task/mod.rs:371
  57: tokio::runtime::blocking::pool::run_task
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/blocking/pool.rs:290
  58: tokio::runtime::blocking::pool::Inner::run
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/blocking/pool.rs:206
  59: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/blocking/pool.rs:186
  60: tokio::runtime::context::enter
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/context.rs:72
  61: tokio::runtime::handle::Handle::enter
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/handle.rs:33
  62: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at /home/nar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.9/src/runtime/blocking/pool.rs:185
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
...........

Just run this example for a couple of minutes:

import asyncio
import json
import websockets

async def ping():
    pong_address = "kauuj71-RPvETjz8FMQugnsNSDJ8033E4lNS_anMFD0="
    uri = "ws://127.0.0.1:9001/mix"

    async with websockets.connect(uri) as websock:
        request_object = {
            "type": "ownDetails"
        }
        await websock.send(json.dumps(request_object))
        response = json.loads(await websock.recv())
        print("Ping address =", response["address"])

        request_object = {
            "type": "send",
            "message": "ping",
            "recipient_address": pong_address
        }
        await websock.send(json.dumps(request_object))
        print("Ping sent:", request_object)

async def pong():
    ping_address = "lzdJ2EkCHlPFVLdsBCkW_n6thaql2wImrGMG0gt_EX4="
    uri = "ws://127.0.0.1:9002/mix"

    async with websockets.connect(uri) as websock:
        request_object = {
            "type": "ownDetails"
        }
        await websock.send(json.dumps(request_object))
        response = json.loads(await websock.recv())
        print("Ping address =", response["address"])

        while True:
            request_object = {
                "type": "fetch"
            }
            await websock.send(json.dumps(request_object))
            print("Pong sent:", request_object)
            response = json.loads(await websock.recv())
            messages = response["messages"]
            print("Pong received messages:", messages)

            if not messages:
                await asyncio.sleep(0.1)

def main():
    asyncio.get_event_loop().run_until_complete(asyncio.gather(
        ping(), pong()
    ))

if __name__ == "__main__":
    main()

Client API consistency

Make APIs between TCPSocket and WebSocket consistent. This could also be a good time to look into @narodnik's async client API idea (mentioned in chat).

Travis releases for Nym

Websocket text fix

Make the websocket always return plain text rather than base64.

Health Checker inside Validator

As the testnet has launched and is now effectively beyond the direct control of any single entity, we need to be able to measure network health in non-adversarial situations.

The validator node should be able to send a bunch of packets through the mixnet to get a view of whether packets are being routed effectively by the available mixnodes. Later, we'll expand this to form the basis of the mix-mining system detailed by Claudia and Aggelos, so we should write this code with that goal in mind.

If a thread blows at startup, panic the entire application

At the moment, if a subsidiary thread explodes during startup, the main application thread just keeps on trucking. Ideally the whole node application startup process would fail, so that node administrators don't think startup succeeded.

Reuse TCP socket connection between client and mixnodes

Persistent socket connection (TCP Socket with client)

Persistent socket connection (Websocket with client)

Make Electron app work with new Rust mixnet client

Combine Rust repositories into a single master repo containing multiple crates

Convert to non-url-safe base64 (everywhere)

Periodic client refresh

Get the client to refresh its topology periodically, so that it detects and reacts to any changes in the topology.

Split provider/mod.rs

The file is getting quite big with a lot of different types and different sets of functionalities - it should be split!

Database for gateway

Gateway should really be using some kinda of database for payload storage rather than raw storage on disk as that is kinda crude + thread unsafe.

More secured auth token - some signature on request

Currently the service provider gives out authentication tokens to everyone. If you forgot yours - it just gives you back what you should have had (so anyone could learn your token this way);
The request would need some signature to ensure that the provider is talking with the right client.

Change how topology is obtained

Currently topology is created via Topology::new() that creates new directory_client every single call. It should be able to reuse it.

Reuse TCP socket connection between mixnodes and providers

Currently when the nodes are sending messages between each other, they create a new TCP connection between them - why not reuse them instead considering they will be always talking to each other.

Once implementation is available, wherever appropriate, replace `futures::lock::Mutex` with `futures::lock::RwLock`

Basically wait a week or so for 0.6.0 release of https://github.com/asomers/futures-locks that will use tokio 0.2.0.
Why we want to change it to RwLock rather than Mutex? As there are way many more reads of the data compared to writes. It is read every single client request (for EVERY client) while it is written to only once per single client - during registration. So that would be a significant performance gain.

Edit: This issue refers to uses in sfw-provider as well as in nym-client

Simplify concurrency with a proper actor framework

Currently we have semi-unintentionally implemented our own 'actor' system. It'd be better to get rid of all that complexity and use something like actix or Riker.

However, that requires some additional research whether we should even do it - I will admit I only had a very quick glance at the concept itself.

Make websocket not crash on ping or pong messages

Persistent socket connection (TCP Socket with provider)

Restore nym-client lib in crate

Take version numbers into account when picking routes

It will certainly be the case that not all clients upgrade simultaneously - but probably not all nodes will either. We may be able to use this to our advantage by taking version numbers into account when the client picks its route. Old clients can generate routes using old nodes (if compatible versions are running). New clients can take advantage of new features, better performance, etc of new mixnode versions, so there are functional incentives to upgrade.

nym-mixnode mandatory host option

currently the doc states:

nym-mixnode run --layer 1 will start the mixnode in layer 1

but this errors with

error: The following required arguments were not provided:
--host

so either the README should state

nym-mixnode run --layer 1 --host localhost will start the mixnode in layer 1 on localhost

or (maybe preferred option) the host option should default to localhost