Giter Club home page Giter Club logo

sandstorm's Introduction

Sandstorm

sandstorm

Cairo prover powered by miniSTARK

license stability-experimental

Sandstorm is a Cairo prover built on top of miniSTARK. The prover was built by reverse engineering StarkWare's open source verifier and was used to submit the first independent proof to StarkWare's Ethereum verifier (see tweet here).

Demo - proving Cairo programs

Generating a proof Verifying a proof
Generating the proof Verifying the proof

In this demo, the prover has a Cairo program that appears to sum the values of an array. The prover runs the program with cairo-run to generate trace.bin (stores register values at each VM cycle) and memory.bin (stores memory address value pairs). The prover then runs sandstorm prove which builds a STARK execution trace and proof from trace.bin, memory.bin and the compiled program.

The verifier, supplied with this proof and the original code, can run sandstorm verify to assert the program was executed correctly without having to run the program themselves. This is a small program for demonstration purposes and it'd probably be faster for the verifier to run the program themselves. Sandstorm is capable of generating proofs for much larger programs, where proof verification would run orders of magnitude faster than running the program. To run this demo locally:

# 1. (optional) install Cairo and activate the venv
# https://www.cairo-lang.org/docs/quickstart.html
source ~/cairo_venv/bin/activate

# 2. (optional) compile and run the Cairo program
cairo-compile example/array-sum.cairo --proof_mode --output example/array-sum.json
cairo-run --program example/array-sum.json \
          --air_private_input example/air-private-input.json \
          --air_public_input example/air-public-input.json \
          --trace_file example/trace.bin \
          --memory_file example/memory.bin \
          --min_steps 128 \
          --layout recursive \
          --proof_mode

# 3. generate the proof
cargo +nightly run -p sandstorm-cli -r -F parallel -- \
    --program example/array-sum.json \
    --air-public-input example/air-public-input.json \
    prove --air-private-input example/air-private-input.json \
          --output example/array-sum.proof

# 4. verify the proof
cargo +nightly run -p sandstorm-cli -r -F parallel -- \
    --program example/array-sum.json \
    --air-public-input example/air-public-input.json \
    verify --proof example/array-sum.proof
Proving Cairo programs with Goldilocks field

Work in Progress - Proving Cairo programs with Goldilocks field

The goldilocks field is a magical 64-bit prime field that has very fast arithmetic. This field was discovered after StarkWare built their Solidity verifier for Cairo programs. As a result Cairo uses a much larger 252-bit prime field by default. Arithmetic in this 252-bit field is slow and it can be hard to practically utilize the storage provided by each field element.

Sandstorm recently supported proving Cairo programs with the 64-bit Goldilocks field instead of StarkWare's default 252-bit field. On a M1 Max proof generation is 5x faster using the 64-bit Goldilocks field and only uses 1/4 of the overall memory when compared against Cairo's default 252-bit field. To run and prove with Goldilocks field locally:

# 1. install Cairo and activate the venv
# https://www.cairo-lang.org/docs/quickstart.html
source ~/cairo_venv/bin/

# 2. compile the Cairo program with Goldilocks field
cairo-compile example/array-sum.cairo \
        --prime 18446744069414584321 \
        --output example/array-sum.json \
        --proof_mode

# 3. modify the Cairo runner to support Goldilocks
# there are a few overly protective asserts that need to be commented out to get 
# things working. The location of these files is based on where you installed Cairo.
# For me they were in `~/cairo_venv/lib/python3.9/site-packages/starkware/cairo/`.
# Remove or comment out the following asserts:
# - lang/vm/relocatable.py line 84 `assert value < 2 ** (8 * n_bytes - 1)`
# - lang/compiler/encode.py line 38 `assert prime > 2 ** (3 * OFFSET_BITS + 16)`

# 4. run the Cairo program
cairo-run --program example/array-sum.json \
        --trace_file example/trace.bin \
        --memory_file example/memory.bin \
        --min_steps 128 \
        --proof_mode

# 5. generate the proof
cargo +nightly run -r -F parallel,asm -- \
    --program example/array-sum.json \
    prove --trace example/trace.bin \
        --memory example/memory.bin \
        --output example/array-sum.proof

# 6. verify the proof
cargo +nightly run -r -F parallel,asm -- \
    --program example/array-sum.json \
    verify --proof example/array-sum.proof
How Sandstorm works

How Sandstorm works

Those curious about the inner workings of Sandstorm can read the comments in air.rs. The comments expect some understanding of how STARK proofs are generated - if you need some background on this then Anatomy of a STARK (part 4) by Alan Szepieniec is a great resource. The pseudo code in section 4.5 of the Cairo whitepaper provides a nice high level overview of how some pieces fit together.

Troubleshooting

Running on ARM64

When running on ARM64 (i.e. Mac M1/M2), remove the asm feature from the example cli calls.

sandstorm's People

Contributors

andrewmilson avatar lucidluckylee avatar tarrencev avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

sandstorm's Issues

unresolved import `ministark::TraceInfo`

I have been trying (several nightly versions from 1.68 till 1.71) to generate the proof (third step) on a Debian VM without the GPU flag, currently I am using the following command:

cargo +nightly run -r -F parallel,asm -- --program example/array-sum.json prove --trace example/trace.bin --memory example/memory.bin --output example/array-sum.proof

However I am getting an error from the ministark dependency, related with TraceInfo.
Is there something I am missing?
Output details below.

Thank you!

cargo +nightly run -r -F parallel,asm --     --program example/array-sum.json     prove --trace example/trace.bin           --memory example/memory.bin           --output example/array-sum.proof
    Updating crates.io index
    Updating git repository `https://github.com/andrewmilson/algebra`
    Updating git repository `https://github.com/andrewmilson/ministark`
   Compiling autocfg v1.1.0
   Compiling libc v0.2.147
   Compiling version_check v0.9.4
   Compiling proc-macro2 v1.0.66
   Compiling unicode-ident v1.0.11
   Compiling cfg-if v1.0.0
   Compiling syn v1.0.109
   Compiling crossbeam-utils v0.8.16
   Compiling memoffset v0.9.0
   Compiling typenum v1.16.0
   Compiling quote v1.0.32
   Compiling crossbeam-epoch v0.9.15
   Compiling num-traits v0.2.16
   Compiling generic-array v0.14.7
   Compiling scopeguard v1.2.0
   Compiling getrandom v0.2.10
   Compiling num-integer v0.1.45
   Compiling rayon-core v1.11.0
   Compiling rand_core v0.6.4
   Compiling crossbeam-deque v0.8.3
   Compiling crossbeam-channel v0.5.8
   Compiling num_cpus v1.16.0
   Compiling num-bigint v0.4.3
   Compiling either v1.9.0
   Compiling ppv-lite86 v0.2.17
   Compiling rand_chacha v0.3.1
   Compiling crypto-common v0.1.6
   Compiling block-buffer v0.10.4
   Compiling digest v0.10.7
   Compiling rayon v1.7.0
   Compiling rand v0.8.5
   Compiling syn v2.0.28
   Compiling paste v1.0.14
   Compiling ark-serialize-derive v0.4.2
   Compiling ark-std v0.4.0
   Compiling ark-serialize v0.4.2
   Compiling derivative v2.2.0
   Compiling ark-ff-asm v0.4.2
   Compiling itertools v0.10.5
   Compiling ark-ff-macros v0.4.2
   Compiling ahash v0.8.3
   Compiling once_cell v1.18.0
   Compiling serde v1.0.183
   Compiling serde_derive v1.0.183
   Compiling hashbrown v0.13.2
   Compiling cc v1.0.82
   Compiling proc-macro-error-attr v1.0.4
   Compiling doc-comment v0.3.3
   Compiling zeroize_derive v1.4.2
   Compiling heck v0.4.1
   Compiling sha2-asm v0.6.3
   Compiling proc-macro-error v1.0.4
   Compiling ministark-gpu v0.1.0 (https://github.com/andrewmilson/ministark#34e0b9c9)
   Compiling rustversion v1.0.14
   Compiling zeroize v1.6.0
   Compiling ark-ff v0.4.2
   Compiling snafu-derive v0.7.5
   Compiling ministark v0.1.0 (https://github.com/andrewmilson/ministark#34e0b9c9)
   Compiling unicode-segmentation v1.10.1
   Compiling unicode-width v0.1.10
   Compiling cpufeatures v0.2.9
   Compiling ruint-macro v1.1.0
   Compiling bincode v1.3.3
   Compiling ruint v1.10.1
   Compiling sha2 v0.10.7
   Compiling textwrap v0.11.0
   Compiling heck v0.3.3
   Compiling snafu v0.7.5
   Compiling atty v0.2.14
   Compiling ansi_term v0.12.1
   Compiling serde_json v1.0.104
   Compiling vec_map v0.8.2
   Compiling strsim v0.8.0
   Compiling bitflags v1.3.2
   Compiling clap v2.34.0
   Compiling strum_macros v0.24.3
   Compiling structopt-derive v0.4.18
   Compiling ryu v1.0.15
   Compiling strum v0.24.1
   Compiling lazy_static v1.4.0
   Compiling itoa v1.0.9
   Compiling structopt v0.3.26
   Compiling pollster v0.2.5
   Compiling ark-poly v0.4.2
   Compiling ark-ff-optimized v0.4.1
   Compiling sandstorm-binary v0.1.0 (/home/azureuser/sandstorm/binary)
   Compiling ark-poly v0.4.0 (https://github.com/andrewmilson/algebra?branch=vec-allocator#5d7964ea)
   Compiling sandstorm-layouts v0.1.0 (/home/azureuser/sandstorm/layouts)
error[E0432]: unresolved import `ministark::TraceInfo`
  --> layouts/src/layout6/trace.rs:32:5
   |
32 | use ministark::TraceInfo;
   |     ^^^^^^^^^^^^^^^^^^^^ no `TraceInfo` in the root

error[E0438]: const `NUM_BASE_COLUMNS` is not a member of trait `Trace`
   --> layouts/src/layout6/trace.rs:251:5
    |
251 |     const NUM_BASE_COLUMNS: usize = NUM_BASE_COLUMNS;
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not a member of trait `Trace`

error[E0438]: const `NUM_EXTENSION_COLUMNS` is not a member of trait `Trace`
   --> layouts/src/layout6/trace.rs:252:5
    |
252 |     const NUM_EXTENSION_COLUMNS: usize = NUM_EXTENSION_COLUMNS;
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not a member of trait `Trace`

Some errors have detailed explanations: E0432, E0438.
For more information about an error, try `rustc --explain E0432`.
error: could not compile `sandstorm-layouts` (lib) due to 3 previous errors
warning: build failed, waiting for other jobs to finish...

Generating memory and trace for the Bootloader

Hello, I wanted to try the example of the Bootloader. But I get a KeyError: hashedSupportedCairoVerifiers on line 89 of the bootloader when trying to run it. Is there any steps I can make to replicate the generation of the memory and trace files ?

Proof compatibility between Stone verifier and Sandstorm

I'm trying to verify Sandstorm proofs with Stone prover to validate that they are compatible. For the same program and layout they seem to be quite different in size, and not really similar. Maybe some parameter is wrong.

I've tried with the following ones:

Layout: recursive

    "field": "PrimeField0",
    "stark": {
        "fri": {
            "fri_step_list": [
                0,
                2,
		2,
		2,
		2,
		2,
		2,
		2,
		2,
		2
            ],
            "last_layer_degree_bound": 1,
            "n_queries": 18,
            "proof_of_work_bits": 24
        },
        "log_n_cosets": 2
    },
    "use_extension_field": false
}

And in sandstorm I'm using:

cargo +nightly run -p sandstorm-cli -r -F parallel -- \
            --program fibonacci_compiled.json \
             --air-public-input fibonacci_public_input.json \
             prove --air-private-input fibonacci_private_input.json \
             --output proof.proof --fri-max-remainder-coeffs 1 --lde-blowup-factor 4 --fri-folding-factor 4 --num-queries 18 --proof-of-work-bits 24

Default ProofParams on sandstorm and Starkware Verifier expected ProofParams

The proof generation on sandstorm uses the following parameters listed below as default:

#[structopt(long, default_value = "65")]
num_queries: u8,
#[structopt(long, default_value = "2")]
lde_blowup_factor: u8,
#[structopt(long, default_value = "16")]
proof_of_work_bits: u8,
#[structopt(long, default_value = "8")]
fri_folding_factor: u8,
#[structopt(long, default_value = "16")]
fri_max_remainder_coeffs: u8,

While checking starkex smart contracts, the proofParams expected on the smart contract method verifyProofAndRegister of Starkware GpsStatementVerifier seems to be the related with the following:

PROOF_PARAMS_N_QUERIES_OFFSET
PROOF_PARAMS_LOG_BLOWUP_FACTOR_OFFSET
PROOF_PARAMS_PROOF_OF_WORK_BITS_OFFSET
PROOF_PARAMS_FRI_LAST_LAYER_LOG_DEG_BOUND_OFFSET
PROOF_PARAMS_N_FRI_STEPS_OFFSET
PROOF_PARAMS_FRI_STEPS_OFFSET

I can see a relation between the first three to the default ones used by sandstorm, but even looking at sandstorm and comparing with starkex smart contracts, couldn't figure yet how I can get the other parameters (PROOF_PARAMS_FRI_LAST_LAYER_LOG_DEG_BOUND_OFFSET, PROOF_PARAMS_N_FRI_STEPS_OFFSET, PROOF_PARAMS_FRI_STEPS_OFFSET), taking into account on sandstorm there is the fri_folding_factor and fri_max_remainder_coeffs.

I believe there might be a way, maybe not trivial, can some insight be given on this matter? @andrewmilson
Thank you!

Bad parameter leads to an infinite loop

Executive Summary

The Prover doesn't check the folding_factor before using it, leading to a potential infinite loop.

Steps to Reproduce

  1. Reproduce the command lines in the demo of Sandstorm adding the following line:

        --fri-folding-factor 1

    between the line

        --air-private-input example/air-private-input.json \

    and the line

       --output example/array-sum.proof

    Like that:

         # 1. (optional) install Cairo and activate the venv
         # https://www.cairo-lang.org/docs/quickstart.html
         source ~/cairo_venv/bin/activate
    
         # 2. (optional) compile and run the Cairo program
         cairo-compile example/array-sum.cairo --proof_mode --output example/array-sum.json
         cairo-run --program example/array-sum.json \
                 --air_private_input example/air-private-input.json \
                 --air_public_input example/air-public-input.json \
                 --trace_file example/trace.bin \
                 --memory_file example/memory.bin \
                 --min_steps 128 \
                 --layout recursive \
                 --proof_mode
    
         # 3. generate the proof
         cargo +nightly run -p sandstorm-cli -r -F parallel -- \
             --program example/array-sum.json \
             --air-public-input example/air-public-input.json \
             prove --air-private-input example/air-private-input.json \
                 --fri-folding-factor 1 \
                 --output example/array-sum.proof

Root Cause Analysis

In the main function of Sandstorm, at line 76, the parameters of the command are charged from the command lines, and so is the fri-folding-factor, equal to 1.

This factor is used in the num_layers function,
itself called by build_layers (here),
itself called by default_prove (here),
itself called by the function prove called in the main file at line 201.

Normally the fri-folding-factor, (i.e. the folding_factor) should not be anything other than 2, 4, 8, 16, but, in the build_layers function, the num_layers function is called before checking the factor (line 185). In another case than ours, this is not a problem, because exiting the num_layers function leads to the verifications (with the match use) in the build_layers function, but in our case, since the folding_factor is 1, we never go out the num_layers function (since we divide the domain_size by 1 at each iteration), and so we have an infinite loop.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.