Giter Club home page Giter Club logo

roadmap's Introduction

Drand - A Distributed Randomness Beacon Daemon

Drand (pronounced "dee-rand") is a distributed randomness beacon daemon written in Golang.

Linked drand nodes collectively produce publicly verifiable, unbiased and unpredictable random values at fixed intervals using bilinear pairings and threshold cryptography.

Drand was first developed within the DEDIS organization, and as of December 2019, is now under the drand organization.

Table of Contents

Goal and Overview

The need for digital randomness is paramount in multiple digital applications ([e]voting, lottery, cryptographic parameters, embedded devices bootstrapping randomness, blockchain systems etc) as well in non-digital such as statistical sampling (used for example to check results of an election), assigning court cases to random judges, random financial audits, etc. However, constructing a secure source of randomness is anything but easy: there are countless examples of attacks where the randomness generation was the culprit (static keys, non-uniform distribution, biased output, etc). drand aims to fix that gap by providing a Randomness-as-a-Service network (similar to NTP servers for time, or Certificate Authority servers for CAs verification), providing continuous source of randomness which is:

  • Decentralized: drand is a software ran by a diverse set of reputable entities on the Internet and a threshold of them is needed to generate randomness, there is no central point of failure.
  • Publicly verifiable & unbiased: drand periodically delivers publicly verifiable and unbiased randomness. Any third party can fetch and verify the authenticity of the randomness and by that making sure it hasn't been tampered with.

A drand network is operated by a group of organizations around the world that includes Cloudflare, EPFL, Kudelski Security, Protocol Labs, Celo, UCL, and UIUC. You can learn more by visiting the League of Entropy website, where you can also see the random values being generated by the network in real time.

Public Randomness

Generating public randomness is the primary functionality of drand. Public randomness is generated collectively by drand nodes and publicly available. The main challenge in generating good randomness is that no party involved in the randomness generation process should be able to predict or bias the final output. Additionally, the final result has to be third-party verifiable to make it actually useful for applications like lotteries, sharding, or parameter generation in security protocols.

A drand randomness beacon is composed of a distributed set of nodes and has two phases:

  • Setup: Each node first generates a long-term public/private key pair. Then all of the public keys are written to a group file together with some further metadata required to operate the beacon. After this group file has been distributed, the nodes perform a distributed key generation (DKG) protocol to create the collective public key and one private key share per server. The participants NEVER see/use the actual (distributed) private key explicitly but instead utilize their respective private key shares for the generation of public randomness.
  • Generation: After the setup, the nodes switch to the randomness generation mode. Any of the nodes can initiate a randomness generation round by broadcasting a message which all the other participants sign using a t-of-n threshold version of the Boneh-Lynn-Shacham (BLS) signature scheme and their respective private key shares. Once any node (or third-party observer) has gathered t partial signatures, it can reconstruct the full BLS signature (using Lagrange interpolation). The signature is then hashed using SHA-256 to ensure that there is no bias in the byte representation of the final output. This hash corresponds to the collective random value and can be verified against the collective public key.

Installation

Official release

Please go use the latest drand binary in the release page.

Manual installation

Drand can be installed via Golang or Docker. By default, drand saves the configuration files such as the long-term key pair, the group file, and the collective public key in the directory $HOME/.drand/.

The docker image can also be built manually by running docker build --build-arg version=$(git describe --tags) --build-arg gitCommit=$(git rev-parse HEAD) -t drandorg/go-drand:latest . in the project root folder Additional instructions for running a node or network using docker can be found in the docker directory

Via Golang

Make sure that you have a working Golang installation and that your GOPATH is set.

Then install drand via:

git clone https://github.com/drand/drand
cd drand
make install

Via Docker

The setup is explained in docker/README.md.

Usage

Run Drand locally

To run a local demo, you can simply run:

make demo

The script spins up a few drand local processes, performs resharing and other operations and will continue to print out new randomness every Xs (currently 6s). For more information, look at the demo README.

A drand beacon provides several public services to clients. A drand node exposes its public services on a gRPC endpoint as well as a REST JSON endpoint, on the same port. The latter is especially useful if one wishes to retrieve randomness from a JavaScript application. Communication is meant to be protected through TLS by using a reverse-proxy to perform TLS termination.

Create a Drand deployment

Consult full instructions at DEPLOYMENT

Fetching Public Randomness

To get the latest public random value, run

drand get public --round <i> <group.toml>

where <group.toml> is the group identity file of a drand node. You can specify the round number when the public randomness has been generated. If not specified, this command returns the most recent random beacon.

The JSON-formatted output produced by drand is of the following form:

{
  "round": 367,
  "signature": "b62dd642e939191af1f9e15bef0f0b0e9562a5f570a12a231864afe468377e2a6424a92ccfc34ef1471cbd58c37c6b020cf75ce9446d2aa1252a090250b2b1441f8a2a0d22208dcc09332eaa0143c4a508be13de63978dbed273e3b9813130d5",
  "previous_signature": "afc545efb57f591dbdf833c339b3369f569566a93e49578db46b6586299422483b7a2d595814046e2847494b401650a0050981e716e531b6f4b620909c2bf1476fd82cf788a110becbc77e55746a7cccd47fb171e8ae2eea2a22fcc6a512486d",
  "randomness": "d7aed3686bf2be657e6d38c20999831308ee6244b68c8825676db580e7e3bec6"
}

Here Signature is the threshold BLS signature on the previous signature value Previous and the current round number. Randomness is the hash of Signature, to be used as the random value for this round. The field Round specifies the index of Randomness in the sequence of all random values produced by this drand instance. The message signed is therefore the concatenation of the round number treated as a uint64 and the previous signature. At the moment, we are only using BLS signatures on the bls12-381 curves and the signature is made over G1.

(Note that this command expects access to a drand group member, this won't work with the current League of Entropy nodes, since they are not exposing their GRPC endpoints directly.)

Using HTTP endpoints

This is the recommended way of using drand randomness, but don't forget to validate the beacons' signatures against the group public key.

One may want to get the distributed key or public randomness by issuing a GET to a HTTP endpoint instead of using a gRPC client. Here is a basic example on how to do so with curl.

To get the distributed key, you can use:

curl <address>/group

Similarly, to get the latest round of randomness from the drand beacon, you can use

curl <address>/public/latest

JavaScript client

To facilitate the use of drand's randomness in JavaScript-based applications, we provide drand-client.

For more details on the procedure and instructions on how to use it, refer to the readme.

Documentation

Here is a list of all documentation related to drand:

As well, here is a list of background readings w.r.t to the cryptography used in drand:

Note that drand was originally a DEDIS-owned project that is now spinning off on its own Github organization. For related previous work on public randomness, see DEDIS's academic paper Scalable Bias-Resistant Distributed Randomness.

What's Next?

Although being already functional, drand is still at an early development stage and there is a lot left to be done. The list of opened issues is a good place to start. On top of this, drand would benefit from higher-level enhancements such as the following:

  • Implement a more failure-resilient DKG protocol or an approach based on verifiable succinct computations (zk-SNARKs, etc).
  • Use / implement a faster pairing based library in JavaScript
  • Add more unit tests
  • Add a systemd unit file

Feel free to submit feature requests or, even better, pull requests ;)

Development

If you want to contribute to Drand, head over to our Development documentation.

Acknowledgments

Thanks to @herumi for providing support on his optimized pairing-based cryptographic library used in the first version.

Thanks to Apostol Vassilev for its interest in drand and the extensive and helpful discussions on the drand design.

Thanks to @Bren2010 and @grittygrease for providing the native Golang bn256 implementation and for their help in the design of drand and future ideas.

Finally, a special note for Bryan Ford from the DEDIS lab for letting me work on this project and helping me grow it.

Coverage

Tracing

In the ./docker folder, you can use the docker-compose.tracing.yaml to spin up the necessary components for monitoring a drand binary in-depth. To run tracing, you will need to pass the --traces command line flag with the endpoint of the tool running in the docker-compose file (or another OpenTelemetry endpoint). You can optionally pass the --traces-probability flag to configure how many calls you wish to sample for telemetry.

License

The drand project is dual-licensed under Apache 2.0 and MIT terms:

roadmap's People

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

roadmap's Issues

Milestone: Bring Timelock on FVM natively

eta: 2023-12-31

description: Integrating Timelock Encryption on FVM can give a big advantage to applications and smart contracts supported by the FVM, giving a competitive advantage against other systems. This milestone includes the development that needs to be done from the drand side to make timelock encryption compatible with FVM natively.

Children:

Milestone: ProtoSchool Tutorial on drand

eta: 2023-09-30

description: publish one proto-school tutorial to demonstrate: randomness consumption/verification in client integrations with L1 blockchains as well as smart contracts/dApps, dynamic NFTs (for example see this Chainlink VRF blog); using randomness in non-blockchain systems such as MP games, lottery systems.

Milestone: Implement native FVM operations to optimise gas cost

eta: 2023-12-31

description: In order to optimise gas cost when using drand or timelock on FVM, we need to make sure we have native actors supporting the necessary operations (BLS verification and time-unlock). Once we have these natively, timelock will officially be natively supported on FVM.

Milestone: FIP to switch Filecoin to the unchained network

eta: 2022-12-31

description: We have recently rolled out new features for the drand network which transform the drand service to an unchained sequence of random numbers (currently rolled out only on drand testnet). With this FIP, we aim to propose that Filecoin switches to the new unchained and higher frequency network. This will bring several benefits such as timelock encryption on the FVM, among others.

Milestone: Unchained drand mainnet

eta: 2023-03-31

description: We have recently rolled out new features for the drand network which transform the drand service to an unchained sequence of random numbers (currently rolled out only on drand testnet). This milestone targets the deployment of the new features on the drand mainnet network.

Theme: Drive community participation and expand client-base

eta: 2023-09-30

description: The objective of this effort is to expand the drand community both in terms of developers contributing to drand, and in terms of clients/users of drand as a service. The measurable items that we plan to pursue in order to achieve the objective are included in subsequent milestones of this Theme.

children:

Milestone: merge FRC for timelock

eta: 2023-08-31

To engage the community and achieve consensus on the best practices for timelock encryption on FVM, we must release a Filecoin Request for Comment and get it approved

Milestone: Optimise storage of beacons and the DB

eta: 2022-12-31

description: Currently, every node participating in the LoE drand network has to store the entire history of beacons in boltdb. This makes it inefficient and presents scaling challenges, especially as we roll out higher-frequency networks.
With this milestone, we want to:

  • support a more efficient and scalable database (e.g., postgres or MySQL)
  • store snapshots of the beacon history in external systems (e.g., IPFS)

Theme: drand & Timelock on FVM

eta: 2023-09-30

description: Timelock Encryption is an extremely promising feature that can be supported when Milestone 3 lands. Further, integrating Timelock Encryption on FVM can give a big advantage to applications and smart contracts supported by the FVM, giving a competitive advantage against other systems. This milestone includes the development that needs to be done from the drand side to make timelock encryption compatible with FVM.

children:

Milestone: Ship drand-v2.0

eta: 2023-09-29

description: We're working on a number of important updates to the drand codebase, with the refactoring of the DKG being the most important one of them. With this milestone we want to group all of these changes together under a common Milestone. The children issues below constitute the changes we're working on.

children:

Theme: Incentives for the League of Entropy (LoE)

eta: 2023-06-30

description: The LoE has been contributing resources to run drand nodes as a public good. Although the commitment is not huge in terms of monetary cost or maintenance effort, having a way to support LoE partners will increase momentum. Depending on the amount of funding that will be made available, development contribution from LoE partners will increase momentum and foster team work across partners.

Milestone: FIP to expose drand values to FVM

eta: 2023-06-30

description: The drand values are currently stored in the Filecoin block headers, however it seems they are currently not available to be used by FVM code. We will need to raise a FIP in order to make these accessible through FVM easily. Notice that this is much more useful than just for Timelock support, while it does enable timelock support, it also enables FVM code to do lotteries or other custom randomness derivation based on the unpredictable drand randomness.

Milestone: release stable tlock actor to Filecoin mainnet

eta: 2023-10-31
After we (and the community) have decided whether tlock should be a native or built-in actor, and have built and tested an MVP in Calibnet (and/or on the experimental lotus fork), we must release it to the filecoin mainnet

Theme: Network Maintenance and Upgrades

eta: 2023-03-31

description: This line of work focuses on the necessary maintenance tasks that are required to keep the codebase in good health. It also includes recent and future feature updates that we want to do on drand in order to meet our long term targets and vision.

children:

Task: Refactor public APIs for drand-v2.0

eta: 2023-08-31

description: Our work to refactor the DKG and ship drand-v2.0 will break the current APIs, which will need to be refactored as a consequence. This task is part of this effort.

Milestone: release tooling for foolproof hybrid encryption

eta: 2023-09-30

Given the resource limitations of the FVM, it does not make sense to perform symmetric decryption on content stored in it. To that end, we must create tooling to make it seamless and bug-free for developers building applications with timelock encryption on FVM.

Milestone: Proof of Concept BLS/Timelock working on FVM

eta: 2023-06-31

description: Create a basic Proof of Concept to verify BLS signatures on FVM, do pairing operations, or even do Timelock in FVM (with FEVM or WASM, depending on FVM's status). This is an important step in order to be able to have timelock on the FVM.

Task: Tear out the Relays and Clients

eta: 2023-07-31

Description: This is about streamlining our drand daemon and making sure that the relay and client code can't cause issues with the daemon. This also allows us to reduce the number of dependencies.

Milestone: Unchained drand mainnet

eta: 2023-03-31

description: The new features mentioned above are rolled out on drand mainnet as a new network that is set up and runs in parallel to the legacy mainnet (that Filecoin currently uses). This update includes swapping G1 and G2 curves to save space in headers.

Task: Release drand-v2.0

eta: 2023-09-29

description: This task will be completed with the code-freeze and release of drand-v2.0.

Milestone: tlock actor using experimental lotus fork

eta: 2023-08-14
Custom native actors will not be available on filecoin officially until the M2 release. However, there is an experimental branch of lotus that can run them for testing purposes. We must develop an MVP on this system to test its viability for release to production

Milestone: FIP to get required native FVM operations

eta: 2023-09-30

description: In order to optimise gas cost when using drand or timelock on FVM, we need to make sure we have native actors supporting the necessary operations (BLS verification and time-unlock). Once we have these natively, timelock will officially be natively supported on FVM. This Milestone is about getting a FIP about this in.

Milestone: release tlock-rs library

eta: 2023-07-31
We must release a tlock rust library that compiles to WASM and doesn't use AGE wrapping, in order to implement a native actor on FVM effectively

Task: Refactor DKG and resharing

eta: 2023-03-31

description: The current implementation of the Distributed Key Generation used in drand has served us well for the initial stages of the drand service and growth, but cannot support our plans for expansion of the League of Entropy, or async and more frequent resharing ceremonies. A refactored DKG with simpler and more straightforward CLI interactions can make the operator’s job easier.
Example challenges we’ve identified include:

  • nodes restarting during a resharing ceremony end up in the wrong state and effectively block the DKG from completing
  • some messages are sent in sync, but some others are gossiped, which results in some of them being delayed and makes things difficult to monitor and debug.

Apart from implementing the features to deal with the above-mentioned challenges, successful completion of this Milestone will be marked by async mainnet ceremonies.

You can read the DKG specification in this Notion doc: DKGv2 Spec.

Milestone: Host the 2023 Randomness Summit

eta: 2023-03-31

description: This will be the follow-up event after the Randomness Summit of 2020. It will be a physical event and will take place alongside the Real World Crypto (RWC) conference, in Tokyo, in March 27-29.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.