Giter Club home page Giter Club logo

arbitrary's Introduction

Arbitrary

The trait for generating structured data from arbitrary, unstructured input.

GitHub Actions Status

About

The Arbitrary crate lets you construct arbitrary instances of a type.

This crate is primarily intended to be combined with a fuzzer like libFuzzer and cargo-fuzz or AFL, and to help you turn the raw, untyped byte buffers that they produce into well-typed, valid, structured values. This allows you to combine structure-aware test case generation with coverage-guided, mutation-based fuzzers.

Documentation

Read the API documentation on docs.rs!

Example

Say you're writing a color conversion library, and you have an Rgb struct to represent RGB colors. You might want to implement Arbitrary for Rgb so that you could take arbitrary Rgb instances in a test function that asserts some property (for example, asserting that RGB converted to HSL and converted back to RGB always ends up exactly where we started).

Automatically Deriving Arbitrary

Automatically deriving the Arbitrary trait is the recommended way to implement Arbitrary for your types.

Automatically deriving Arbitrary requires you to enable the "derive" cargo feature:

# Cargo.toml

[dependencies]
arbitrary = { version = "1", features = ["derive"] }

And then you can simply add #[derive(Arbitrary)] annotations to your types:

// rgb.rs

use arbitrary::Arbitrary;

#[derive(Arbitrary)]
pub struct Rgb {
    pub r: u8,
    pub g: u8,
    pub b: u8,
}

Customizing single fields

This can be particular handy if your structure uses a type that does not implement Arbitrary or you want to have more customization for particular fields.

#[derive(Arbitrary)]
pub struct Rgba {
    // set `r` to Default::default()
    #[arbitrary(default)]
    pub r: u8,

    // set `g` to 255
    #[arbitrary(value = 255)]
    pub g: u8,

    // Generate `b` with a custom function of type
    //
    //    fn(&mut Unstructured) -> arbitrary::Result<T>
    //
    // where `T` is the field's type.
    #[arbitrary(with = arbitrary_b)]
    pub b: u8,

    // Generate `a` with a custom closure (shortuct to avoid a custom function)
    #[arbitrary(with = |u: &mut Unstructured| u.int_in_range(0..=64))]
    pub a: u8,
}

fn arbitrary_b(u: &mut Unstructured) -> arbitrary::Result<u8> {
    u.int_in_range(64..=128)
}

Implementing Arbitrary By Hand

Alternatively, you can write an Arbitrary implementation by hand:

// rgb.rs

use arbitrary::{Arbitrary, Result, Unstructured};

#[derive(Copy, Clone, Debug)]
pub struct Rgb {
    pub r: u8,
    pub g: u8,
    pub b: u8,
}

impl<'a> Arbitrary<'a> for Rgb {
    fn arbitrary(u: &mut Unstructured<'a>) -> Result<Self> {
        let r = u8::arbitrary(u)?;
        let g = u8::arbitrary(u)?;
        let b = u8::arbitrary(u)?;
        Ok(Rgb { r, g, b })
    }
}

License

Licensed under dual MIT or Apache-2.0 at your choice.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

arbitrary's People

Contributors

00xc avatar aljoschameyer avatar benluelo avatar blt avatar c410-f3r avatar cobaltcause avatar dtolnay avatar e00e avatar fitzgen avatar frewsxcv avatar glandium avatar greyblake avatar jameysharp avatar jcaesar avatar maackle avatar manishearth avatar matklad avatar maurer avatar michaelsproul avatar nagisa avatar nullr0ute avatar parkmycar avatar rcvalle avatar sky9x avatar soenkehahn avatar tarcieri avatar thebluematt avatar thomasdenh avatar wackbyte avatar xiretza avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

arbitrary's Issues

Improving support for generating recursive types

Hi!

Would it be possible to update the API to allow implementors of Arbitrary to plumb some context through which would be useful when manually implementing Arbitrary?

I most often need this to implement a recursion limit when handling arbitrary recursive types, e.g. a tree. There is a workaround involving globals (using lazy_static) but it would be much easier to do if the trait supported a method like Arbitrary::with_context() which took a struct of a user-defined type so I could pass along the depth limit.

Here's some abstracted code for what I currently have (with a wrapper struct for making the depth tracking less verbose), if it helps:

enum Expression {
    Constant(f64),
    X,
    Y,
    Add(Box<Expression>, Box<Expression>),
}

lazy_static! {
    static ref ARBITRARY_RECURSION_GUARD: AtomicUsize = AtomicUsize::new(16);
}

impl<'a> Arbitrary<'a> for Expression {
    fn arbitrary(u: &mut Unstructured<'a>) -> arbitrary::Result<Self> {
        let depth = ARBITRARY_RECURSION_GUARD.fetch_sub(1, Ordering::SeqCst);
        if (depth == 0) {
           ARBITRARY_RECURSION_GUARD.fetch_add(1, Ordering::SeqCst);
           Ok(Expression::X)
        } else {
            let expr = // generate stuff here, can involve recursive calls to Expression::arbitrary()
            ARBITRARY_RECURSION_GUARD.fetch_add(1, Ordering::SeqCst);
            Ok(expr)
       }
    }
}

This would be simpler if the Expression::arbitrary() method could take an argument that it could pass down to the recursive calls.

If extending the API in this way isn't possible, do you have suggestions for how best to encode this pattern?

Test failures on 32 bits arches

On both i686 and armv7hl, one test fails:

 /usr/bin/env CARGO_HOME=.cargo RUSTC_BOOTSTRAP=1 /usr/bin/cargo test -j6 -Z avoid-dev-deps --release --no-fail-fast
   Compiling arbitrary v0.2.0 (/builddir/build/BUILD/arbitrary-0.2.0)
     Running `/usr/bin/rustc --crate-name arbitrary src/lib.rs --color never --emit=dep-info,link -C opt-level=3 --test -C metadata=2b9ebc5100c56330 -C extra-filename=-2b9ebc5100c56330 --out-dir /builddir/build/BUILD/arbitrary-0.2.0/target/release/deps -L dependency=/builddir/build/BUILD/arbitrary-0.2.0/target/release/deps -Copt-level=3 -Cdebuginfo=2 -Clink-arg=-Wl,-z,relro,-z,now -Ccodegen-units=1 --cap-lints=warn`
    Finished release [optimized] target(s) in 5.04s
     Running `/builddir/build/BUILD/arbitrary-0.2.0/target/release/deps/arbitrary-2b9ebc5100c56330`
running 3 tests
test test::finite_buffer_fill_buffer ... ok
test test::ring_buffer_fill_buffer ... ok
test test::ring_buffer_container_size ... FAILED
failures:
---- test::ring_buffer_container_size stdout ----
thread 'test::ring_buffer_container_size' panicked at 'assertion failed: `(left == right)`
  left: `10`,
 right: `9`', src/lib.rs:596:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
failures:
    test::ring_buffer_container_size
test result: FAILED. 2 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

See build.log on https://koji.fedoraproject.org/koji/taskinfo?taskID=39452113

Stack overflow for recursive type

The following code

use arbitrary::{Arbitrary, Unstructured};

#[derive(Debug, Arbitrary)]
enum Nat {
    Succ(Box<Nat>),
    Zero,
}

fn main() {
    Nat::arbitrary(&mut Unstructured::new(&[])).ok();
}

Results in a stack overflow

thread 'main' has overflowed its stack
fatal runtime error: stack overflow
fish: Job 1, 'cargo run' terminated by signal SIGABRT (Abort)

I guess there isn't any data for the discriminant to go on, so it choses the first one. Upon which it needs to construct a Succ, so it needs to construct a Nat. And there's the infinite recursion.

This issue is similar to #30.
I have no practical use for this type, I just encountered it while helping someone on a Matrix channel. I know the issue can be side-stepped by reordering the variants.

Arbitrary generates mostly empty strings

Hello, I'd like to use Arbitrary to create some Strings. However, I noticed that Arbitrary produces a lot of empty strings. Am I missing something? Thank you.

Test program:

use arbitrary::{Arbitrary, Unstructured};
use rand::{thread_rng, Rng};

fn main() {
    let mut seed = [0u8; 2048 * 2048];
    thread_rng().fill(&mut seed[..]);
    let mut generator = Unstructured::new(&seed);

    let t = (1..50)
        .map(|_| String::arbitrary(&mut generator).ok())
        .flatten()
        .collect::<Vec<String>>();

    println!("{:?}", t);
}

Output (ran several times) (mostly empty strings):

matus@pine arbit_test (master)> cargo -q r
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/arbit_test`
["", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", ""]
matus@pine arbit_test (master)> cargo -q r
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/arbit_test`
["", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", ""]
matus@pine arbit_test (master)> cargo -q r
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/arbit_test`
["u\r", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", ""]
matus@pine arbit_test (master)> cargo -q r
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/arbit_test`
["", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", ""]
matus@pine arbit_test (master)> cargo -q r
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/arbit_test`
["ูกศฃ\u{5}", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", ""]
matus@pine arbit_test (master)> cargo -q r
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/arbit_test`
["", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", ""]
matus@pine arbit_test (master)> cargo -q r
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/arbit_test`
["X", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", ""]
matus@pine arbit_test (master)> cargo -q r
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/arbit_test`
["", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", ""]
matus@pine arbit_test (master)> cargo -q r
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/arbit_test`
["", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", ""]
matus@pine arbit_test (master)> cargo -q r
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/arbit_test`
["v", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", ""]

Allow getting an arbitrary length with a maximum upper bound

Sometimes you don't want more than N things generated because, for example, it makes the fuzz target too slow and run up against libfuzzer timeouts.

A solution to this is pretty similar to a solution to #35 in that they both want access to (or something closer to) the underlying int_in_range from the end of the byte string.

impl Unstructured<'_> {
    pub fn arbitrary_len_with_max<ElementType>(&mut self, max: usize) -> Result<usize>
    where
        ElementType: Arbitrary,
    {
        ...
    }
}

Arbitrary derive fails with lifetimes in some cases

Hello! Bear with me as I am new to Arbitrary.

I have this type from juniper:

pub struct Arguments<'a, S> {
    pub items: Vec<(Spanning<&'a str>, Spanning<InputValue<S>>)>,
}

Spanning and InputValue successfully derive(arbitrary::Arbitrary)). When I try to derive on Arguments, I get the following error:

error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
  --> juniper/src/ast.rs:59:42
   |
59 | #[cfg_attr(feature = "arbitrary", derive(arbitrary::Arbitrary))]
   |                                          ^^^^^^^^^^^^^^^^^^^^
   |
note: first, the lifetime cannot outlive the lifetime `'a` as defined here...
  --> juniper/src/ast.rs:60:22
   |
60 | pub struct Arguments<'a, S> {
   |                      ^^
note: ...so that the types are compatible
  --> juniper/src/ast.rs:59:42
   |
59 | #[cfg_attr(feature = "arbitrary", derive(arbitrary::Arbitrary))]
   |                                          ^^^^^^^^^^^^^^^^^^^^
   = note: expected `<&'a str as Arbitrary<'_>>`
              found `<&str as Arbitrary<'_>>`
note: but, the lifetime must be valid for the lifetime `'arbitrary` as defined here...
  --> juniper/src/ast.rs:59:42
   |
59 | #[cfg_attr(feature = "arbitrary", derive(arbitrary::Arbitrary))]
   |                                          ^^^^^^^^^^^^^^^^^^^^
note: ...so that the types are compatible

Manually implenting it works:

impl<'a, S> arbitrary::Arbitrary<'a> for Arguments<'a, S>
where
    S: arbitrary::Arbitrary<'a>,
{
    fn arbitrary(u: &mut arbitrary::Unstructured<'a>) -> arbitrary::Result<Self> {
        let items: Vec<(Spanning<&'a str>, Spanning<InputValue<S>>)> = u.arbitrary()?;
        Ok(Self { items })
    }
}

Even though I am unblocked, I figured I'd file as one would expect the derive to work in this case.

Enum discriminants should be taken from the end of the input string, like lengths

The same rationale for why we take lengths from the end of the input byte string applies to discriminants: depending on the discriminant chosen, it changes which subsequent arbitrary calls are made and how many bytes are needed. It generally seems best to have "control flow"-y choices that affect shape be taken from the end of the input string and "filler" data taken from the start.

To do this we'll need to expose our internal int_in_range that takes from the end. Not sure what to call it, or how to explain why there are two variants in the documentation well...

#![no_std] Support

The Arbitrary crate isn't #[no_std], which prevent fuzzing #![no_std] crates. For now can be worked around by using #![cfg_attr(not(feature = "fuzzing), no_std)] at the top of the fuzzed crate instead of #![no_std], but it would be better not to need this workaround.

Although I don't really see a use case of doing the actual fuzzing in a #![no_std] environment, is would be nice not to need any reference to std in the fuzzed crate.

AFAIK, the arbitrary crate doesn't really rely on std features and pretty much everything required can be fetched from the alloc create.

I forked the crate and started to try to make the change. The first roadbump I encountered is the quote crate used in arbitrary-derive, so I might need to open a simillar issue over there.

Derive failing for module path for Enum

What is the issue

This is the same as #39 but for enum's instead of structs. This exists in version 0.4.3.

Example

use arbitrary;

#[derive(arbitrary::Arbitrary)]
pub enum Wrapper {
    TestA,
    TestB
}

fn main() {
    let a = Wrapper::TestA;
    match a {
        Wrapper::TestA => println!("a"),
        _ => println!("not a"),
    }
}

Has error:

error[E0433]: failed to resolve: use of undeclared type or module `Arbitrary`
 --> src/main.rs:3:10
  |
3 | #[derive(arbitrary::Arbitrary)]
  |          ^^^^^^^^^^^^^^^^^^^^ use of undeclared type or module `Arbitrary`
  |
  = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

error: aborting due to previous error

For more information about this error, try `rustc --explain E0433`.
error: could not compile `checker`.

To learn more, run the command again with --verbose.

Derive `Arbitrary` for types with lifetimes

derive(Arbitrary) currently won't derive Arbitrary if the type has a lifetime parameter in it. Most of the time this is fine, because Arbitrary: 'static. However, it seems like it would be possible to implement Arbitrary if all of the type parameters were set to 'static. This is somewhat inconsistent with Cow<'static, impl Arbitrary>: Arbitrary.

I haven't looked into how hard this would be, but I'm willing to mail a change adding this (maybe behind a Cargo feature?) if maintainers would consider it a useful addition.

Allow `Unstructured` to be backed by an iterator of bytes instead of a byte slice?

In our use case, we are not using Arbitrary for fuzzing, but simply for creating arbitrary fixture values in tests. Currently we create a 10MB static Vec<u8> of random noise and use that as our Unstructured data. However this is annoying since it requires 10MB of memory overhead, and sometimes even then we run out of bytes.

I am wondering if it would be valid to have two flavors of Unstructured, one backed by a byte slice, presumably for fuzzing, and one backed by an infinite iterator of bytes which can never be exhausted. I experimented with a PR for doing this and got some basic tests passing, but don't know how valid it is in the grand scheme. I did find that some functionality is indeed dependent on the fixed byte slice, so at the very least some extra UX effort would have to be made to provide slightly different interfaces for different Unstructured flavors.

I understand if this wouldn't be worth the effort but I guess I am primarily wondering from a motivational standpoint why Unstructured is backed by a byte array instead of an iterator, and only secondarily asking for some feedback on the feasibility of using infinite iterators. Note: I know next to nothing about fuzzing.

Allow custom arbitrary methods for fields in the custom derive

Sometimes a field of a struct doesn't implement arbitrary and it is either impossible to do (because it is from another crate, for example) or undesired.

We should support some kind of attribute to either use the Default::default implementation for the type, when we "don't care" about that field, or supply an arbitrary function that has the same signature as Arbitrary::arbitrary to be used instead.

This is all possible to do when implementing Arbitrary by hand, in the sense that you can always stuff whatever code inside the impl. But this seems common enough and simple enough that we may want to support it in the custom derive.

Straw person syntax:

#[derive(Arbitrary)]
struct MyThing {
    // This doesn't actually matter, just a diagnostic message, so don't
    // waste input bytes on it.
    #[arbitrary(default)]
    diagnostic: String,

    // This is a type defined by a foreign crate and doesn't implement
    // Arbitrary. Because of orphan rules, we can't implement Arbitrary
    // for it, so instead we have a one-off function.
    #[arbitrary(arbitrary = arbitrary_foreign_type)]
    foreign: foreign_crate::ForeignType,
}

fn arbitrary_foreign_type(u: &mut Unstructured)
    -> Result<foreign_crate::ForeignType>
{
    // ...
}

The open question in my mind is how to deal with the other, optional Arbtirary methods? e.g. arbitrary_take_rest, shrink, and size_hint. Should we assume the default implementations in this case? Should we also allow providing custom implementations for them?

Any thoughts @Manishearth?

+cc @bnjbvr

Can Arbitrary be biased?

I have a question that might be clarified in the documentation. Is Arbitrary supposed to be exactly uniformly distributed? Or is it supposed to be quick? For example, I can generate a number between 0 and 10 by dividing the value of a u8 by 26. With extra computations, the distribution could be made uniform, but is that necessary or desirable?

Reversed arbitrary

Hello,

I've been toying with custom mutators in cargo-fuzz with libfuzzer and it seems to me to support changes to typed data instead of bytes, the reversed arbrirtary operation would be really helpful. Here's the interface I have implemented so far:

/// Define a custom fuzz mutator.
///
/// If `$bytes` exceeds `$max_size`, it will be silently truncated.
///
/// ## Example
/// ```no_run
/// #![no_main]
/// use libfuzzer_sys::{fuzz_target, fuzz_mutator, llvm_fuzzer_mutate};
///
/// fuzz_target!(|data: &[u8]| {
///     let _ = std::str::from_utf8(data);
/// });
///
/// fuzz_mutator!(|data: &mut [u8], max_size: usize| {
///     println!("custom mutator called with data len = {} and max_size = {}", data.len(), max_size);
///     /* call wrapper function of libfuzzer's default mutator */
///     llvm_fuzzer_mutate(data, max_size)
/// });
/// ``` 

The mutator with typed data would be the equivalent:

fuzz_mutator!(|data: &mut T, max_size: usize | {
   loop {
   /* perform changes on `data` */

   if data.arbitrary_size() <= max_size { /* where arbitrary_size is some method from the Trait to calculate size in bytes */
     break;
  }
}
 });

Does this sound like a reasonable approach to you?

SIGSEGV on certain values

I think this is an issue with arbitrary, but let me know where I should move it if not.

I'm observing an issue where derived arbitrary types trigger a SIGSEGV when running cargo-fuzz. Here's a minimal example:

#![no_main]

use libfuzzer_sys::fuzz_target;
use arbitrary::Arbitrary;

#[derive(Arbitrary, Clone, Debug, PartialEq)]
pub enum Op {
    A(A),
    B(B),
}

#[derive(Arbitrary, Clone, Debug, PartialEq)]
pub enum A {
    Leaf,
    B(Box<B>),
}

#[derive(Arbitrary, Clone, Debug, PartialEq)]
pub enum B {
    Leaf,
    A(Box<A>),
}

fuzz_target!(|ops: Vec<Op>| {
});

Running this on the latest nightly (both mac and linux) triggers a SIGSEGV:

INFO: Seed: 3703294504
INFO: Loaded 1 modules   (17415 inline 8-bit counters): 17415 [0x10f5ef708, 0x10f5f3b0f), 
INFO: Loaded 1 PC tables (17415 PCs): 17415 [0x10f5f3b10,0x10f637b80), 
INFO:       59 files found in /Users/yusuf/Desktop/sandbox/fuzz-bug/fuzz/corpus/fuzz_target_1
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€

Error: Fuzz target exited with signal: 11

Any of these changes will remove the segfault:

  1. Changing the fuzz target's input from Vec<Op> to just Op.
  2. Removing the B enum variant from A
  3. Removing the A enum variant from B

Auto-deriving Arbitrary

Similar to what I did for quickcheck, it should be easy enough to write a derive macro for Arbitrary, perhaps even with some shrinking support for simple types. Should I do it in another crate or send a PR your way to integrate it?

Version 1.0

What does a 1.0 release look like? Personally, I feel like this crate is pretty feature complete as-is that it could be close to a 1.0 release.

Best way to generate float values in range

Hi folks, in technocreatives/dbc-codegen#12 we're generating implementations of Arbitrary for structs where we know some values have to be in a specific range. For ints, this is not a problem as we just use Unstructured::int_in_range, but I'm unsure how to proceed for floats. (We currently just go with always setting the float field to the lower bound.)

My initial idea was to keep generating floats until we get one in ourr range, but that seems awfully slow. Is there a good strategy for generating floats that are in a specific range? Anything I should look up? All I can find assumes an existing random number library is in placeโ€ฆ which I'm hesitant to add just for this.

Move into rust-fuzz organization?

I think @fitzgen is planning on putting more work into fuzzing so it would be nice if he could work on and publish to this crate, and in general we should be collaborating more on this amongst the fuzzing community.

Deriving `Arbitrary` on a field containing a `Cow<'static, str>` fails.

For example, the following code (using version 1.0.1) fails to compile.

use std::borrow::Cow;

#[derive(arbitrary::Arbitrary)]
pub struct WithACow {
    cow: Cow<'static, str>,
}

fn main() {
    println!("Hello, world!");
}

With the error message

error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
 --> src/main.rs:3:10
  |
3 | #[derive(arbitrary::Arbitrary)]
  |          ^^^^^^^^^^^^^^^^^^^^
  |
note: first, the lifetime cannot outlive the lifetime `'arbitrary` as defined on the impl at 3:10...
 --> src/main.rs:3:10
  |
3 | #[derive(arbitrary::Arbitrary)]
  |          ^^^^^^^^^^^^^^^^^^^^
note: ...so that the expression is assignable
 --> src/main.rs:3:10
  |
3 | #[derive(arbitrary::Arbitrary)]
  |          ^^^^^^^^^^^^^^^^^^^^
  = note: expected `&mut Unstructured<'_>`
             found `&mut Unstructured<'arbitrary>`
  = note: but, the lifetime must be valid for the static lifetime...
note: ...so that the expression is assignable
 --> src/main.rs:3:10
  |
3 | #[derive(arbitrary::Arbitrary)]
  |          ^^^^^^^^^^^^^^^^^^^^
  = note: expected `Cow<'static, _>`
             found `Cow<'_, _>`
  = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

But something like this does compile:

use std::borrow::Cow;

use arbitrary::Arbitrary;

pub struct WithACow {
    cow: Cow<'static, str>,
}

impl<'a> Arbitrary<'a> for WithACow {
    fn arbitrary(u: &mut arbitrary::Unstructured<'a>) -> arbitrary::Result<Self> {
        Ok(WithACow {
            cow: Arbitrary::arbitrary(u).map(Cow::Owned)?,
        })
    }
}

fn main() {
    println!("Hello, world!");
}

Tracking issue for 0.3.0 release

So we have some nice improvements to this crate, and I think everything has technically been compatible with a 0.2.X release but I think it is a good time to revisit some of the public API and trait designs and make a 0.3.0 release. I figured it would be good to open an issue to talk about what we might want in it.

TODO

  • Fill out public API doc comments, and add examples to each of them
  • Better Arbitrary for String implementation (#17)
  • Should Arbitrary have Debug as a super trait? (#7)
  • Should Arbitrary have Clone as a super trait? This would allow easier/better shrinking implementations (e.g. for various collections)
  • Refactor Unstructured to look more like FuzzedDataProvider? In particular, maybe we should get lengths from the end of the buffer like that does.
  • Add Arbitrary::arbitrary_take_rest methods to consume the rest of an Unstructured. (#18)
  • Add Arbitrary::size_hint and refactor Unstructured::container_size into Unstructured::collection_len, as discussed in #18.
  • Add a fn arbitrary<A: Arbitrary>(&mut self) -> Result<A, Self::Error> helper method to Unstructured to shorten invoking nested-arbitrary calls, so you can just do let foo = u.arbitrary()?; and let type inference figure things out for you?

As we surface more things to be done for 0.3.0 in the comments, I'll add them to this list, so that we have a single place for everything.


+cc @Manishearth: what do you think? Anything we should add to or remove from this list?

Questions about `Arbitrary`

Hi! A few notes to preface issue:

  • I asked @Manishearth this question a few weeks back and said that I should open an issue.
  • This is not meant to be judgement of the decisions made in this crate, especially since I am not aware of the constraints that y'all might have.

The question: why did y'all opt for a QuickCheck-style, trait-based approach over a Hypothesis/Proptest-style, explicit shrinking object/struct approach? For context, a Hypothesis/Proptest-style checker would have a shrinking strategy that returns a tree. This tree would values and shrinking/expansion strategies. I've noticed that the biggest win of Hypothesis/Proptest-style approaches is that shrinkers/generators can be implemented for completely arbitrary (no pun intended!) types so that orphan rules don't apply to generated code.

That being said, I suspect that fuzzers might have different constraints than property testing, so this entire question could be taken with a grain of salt.

If I recall correctly, @fitzgen mentioned to me that they preferred Quickcheck to Proptest-style systems at Strangeloop.

Derive failing in v0.4.2

What is the issue

The derive macro seems to have build issues when called upon via the crate in version 0.4.2

Example

Cargo.toml

[dependencies]
arbitrary = { version = "=0.4.2", features = ["derive"] }

main.rs

use arbitrary;

#[derive(arbitrary::Arbitrary)]
pub struct Wrapper {
    pub x: u8,
}

fn main() {
    let a = Wrapper {
        x: 3,
    };
    println!("{}", a.x);

Version 0.4.1

This was working in v0.4.1 and can be seen by changing the Cargo.toml in the above example to

arbitrary = { version = "=0.4.1", features = ["derive"] }

And re-running main.rs from above.

libfuzzer-sys

I'm not sure the exact cause of the issue but it will impact libfuzzer_sys v0.3.2 which
re-exports arbitrary::Arbitrary

Work around for libfuzzer_sys v0.3.2

I noticed there is a work around by doing
Also have arbitrary v0.4.2 as a dependency with features = ["derive'].
Then importing Arbitrary directly from arbitrary

e.g. for a fuzzer

use libfuzzer_sys::fuzz_target;
use arbitrary::Arbitrary;

as opposed to how it would previously be done

use libfuzzer_sys::{fuzz_target, arbitrary::Arbitrary};

fill_buffer doesn't match doc - never returns NotEnoughData so can't re-fill arbitrary

The fill_buffer documentation specifies:

/// If this `Unstructured` does not have enough data to fill the whole
/// `buffer`, an error is returned.

However the actual implementation is:

pub fn fill_buffer(&mut self, buffer: &mut [u8]) -> Result<()> {
let n = std::cmp::min(buffer.len(), self.data.len());
buffer[..n].copy_from_slice(&self.data[..n]);
for byte in buffer[n..].iter_mut() {
*byte = 0;
}
self.data = &self.data[n..];
Ok(())
}

Which silent-fails and never returns an error.

This in turn implies that code such as:

let line: SomeStructThatDerivesArbitrary = match unstructured.arbitrary() {
	Ok(ok) => ok,
	Err(arbitrary::Error::NotEnoughData) => {
		std::mem::drop(unstructured);
		random_data.clear();
		let mut trng = rand::thread_rng();
		random_data.extend(
			(0..1000)
				.map(|_| trng.sample::<u8, _>(rand::distributions::Standard)),
		);
		unstructured = arbitrary::Unstructured::new(&random_data);
		unstructured.arbitrary()?
	}
	Err(e) => return Err(e.into()),
};

actually always generates the same SomeStructThatDerivesArbitrary once the initial arbitrary::Unstructured is empty. (Which in my case is right at the start.)

fill_buffer should match the doc and return NotEnoughData when too small instead of silent-fail-ing, otherwise there's a huge risk that the user will actually explore a much smaller space of arbitrary values than they expected.

I note this was initially the case, but was changed in 1d2f653.

This commit makes the assumption that fuzzers will reduce their inputs by reducing the number of input bytes, and couldn't perform reduction by turning some input bytes into zeroes - which I think would probably be the correct thing for them to do when using arbitrary.

Arbitrary impl for char is slow

The current implementation, when it hits an invalid character, loops downward by one bit each time. This is slow, all of the invalid char values are between 0xD800 and 0xDFFF, or greater than 0x10ffff (already dealt with with the mask)

This means that we spend a lot of time just looping here, especially when creating strings. We should instead just replace these with null or decrement the fourth nibble.

Issue re-strutcturing Vec<u8> from a libfuzz test

I've got a simple fuzz test

fuzz_target!(|data: Vec<u8>| {
    if data.len() > 1 {
        panic!("asfa");
    }
});

launching with cargo fuzz and libfuzzer I've got

SUMMARY: libFuzzer: deadly signal
...
Base64: BB8fHx8qKv//
...

fuzz/artifacts/test/crash-327090b23ecd21b1ed2a745974e0df92e975c3ee

Output of `std::fmt::Debug`:

        [
            186,
            186,
            186,
            10,
            186,
            10,
            0,
            0,
         ]

If I want to use this data in a test

use arbitrary::Arbitrary;
        let a = base64::decode("BB8fHx8qKv//").unwrap();
        let mut b = arbitrary::Unstructured::new(&a[..]);
        let c = <Vec<u8>>::arbitrary(&mut b);
        dbg!(c);

I've got

[src/fuzz.rs:90] c = Ok(
    [],
)

same if I use

        let a = include_bytes!("../fuzz/artifacts/test/crash-327090b23ecd21b1ed2a745974e0df92e975c3ee");

This doesn't happen without using Vec

arbitrary version is v0.4.7 on the base project and in the fuzz project

one_lifetime and two_lifetimes tests fail on 32 bit.

Hi

rust-arbitrary's tests are failing on Debian's QA infrastruture when built with the "derive" feature, specifically.

---- one_lifetime stdout ----
thread 'one_lifetime' panicked at 'assertion failed: (left == right)
left: 4,
right: 8', tests/derive.rs:167:5
stack backtrace:
0: rust_begin_unwind
at /usr/src/rustc-1.56.0/library/std/src/panicking.rs:517:5
1: core::panicking::panic_fmt
at /usr/src/rustc-1.56.0/library/core/src/panicking.rs:101:14
2: core::panicking::assert_failed_inner
3: core::panicking::assert_failed
at /usr/src/rustc-1.56.0/library/core/src/panicking.rs:140:5
4: derive::one_lifetime
at ./tests/derive.rs:167:5
5: derive::one_lifetime::{{closure}}
at ./tests/derive.rs:160:1
6: core::ops::function::FnOnce::call_once
at /usr/src/rustc-1.56.0/library/core/src/ops/function.rs:227:5
7: core::ops::function::FnOnce::call_once
at /usr/src/rustc-1.56.0/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace.

---- two_lifetimes stdout ----
thread 'two_lifetimes' panicked at 'assertion failed: (left == right)
left: 8,
right: 16', tests/derive.rs:186:5
stack backtrace:
0: rust_begin_unwind
at /usr/src/rustc-1.56.0/library/std/src/panicking.rs:517:5
1: core::panicking::panic_fmt
at /usr/src/rustc-1.56.0/library/core/src/panicking.rs:101:14
2: core::panicking::assert_failed_inner
3: core::panicking::assert_failed
at /usr/src/rustc-1.56.0/library/core/src/panicking.rs:140:5
4: derive::two_lifetimes
at ./tests/derive.rs:186:5
5: derive::two_lifetimes::{{closure}}
at ./tests/derive.rs:178:1
6: core::ops::function::FnOnce::call_once
at /usr/src/rustc-1.56.0/library/core/src/ops/function.rs:227:5
7: core::ops::function::FnOnce::call_once
at /usr/src/rustc-1.56.0/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace.

I'm guessing this is a testsuite issue and that the values should be changed to be the size of a pointer and twice the size of a pointer respectively rather than hardcoded values of 8 and 16 but I'd like confirmation from someone who actually understands the code.

Cannot use u.choose because it panics

Hello I'm having an issue when I want to use choose method on an Unstructured struct. Here is the error:

INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3281183050
INFO: Loaded 1 modules   (77436 inline 8-bit counters): 77436 [0x5651ff1fc2d0, 0x5651ff20f14c),
INFO: Loaded 1 PC tables (77436 PCs): 77436 [0x5651ff20f150,0x5651ff33d910),
INFO:     1521 files found in /home/bnj/rust/apollo-rs/fuzz/corpus/parser
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
doc =====================

========================
INFO: seed corpus: files: 1521 min: 1b max: 727b total: 242498b rss: 37Mb
available type ------------------ []
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: NotEnoughData',

Do you know what I'm doing wrong ?

Thanks a lot.

Operating system: Ubuntu 21.10
Rust version: v1.60.0-nightly

Replace arbitrary_array macro with const generics (after stabilization)

arbitrary/src/lib.rs

Lines 716 to 797 in ab4adcf

macro_rules! arbitrary_array {
{$n:expr, ($t:ident, $a:ident) $(($ts:ident, $as:ident))*} => {
arbitrary_array!{($n - 1), $(($ts, $as))*}
impl<T: Arbitrary> Arbitrary for [T; $n] {
fn arbitrary(u: &mut Unstructured<'_>) -> Result<[T; $n]> {
Ok([
Arbitrary::arbitrary(u)?,
$(<$ts as Arbitrary>::arbitrary(u)?),*
])
}
#[allow(unused_mut)]
fn arbitrary_take_rest(mut u: Unstructured<'_>) -> Result<[T; $n]> {
$(let $as = $ts::arbitrary(&mut u)?;)*
let last = Arbitrary::arbitrary_take_rest(u)?;
Ok([
$($as,)* last
])
}
#[inline]
fn size_hint(depth: usize) -> (usize, Option<usize>) {
crate::size_hint::and_all(&[
<$t as Arbitrary>::size_hint(depth),
$( <$ts as Arbitrary>::size_hint(depth) ),*
])
}
#[allow(unused_mut)] // For the `[T; 1]` case.
fn shrink(&self) -> Box<dyn Iterator<Item = Self>> {
let mut i = 0;
let mut shrinkers = [
self[i].shrink(),
$({
i += 1;
let t: &$ts = &self[i];
t.shrink()
}),*
];
Box::new(iter::from_fn(move || {
let mut i = 0;
Some([
shrinkers[i].next()?,
$({
i += 1;
let t: $ts = shrinkers[i].next()?;
t
}),*
])
}))
}
}
};
($n: expr,) => {};
}
impl<T: Arbitrary> Arbitrary for [T; 0] {
fn arbitrary(_: &mut Unstructured<'_>) -> Result<[T; 0]> {
Ok([])
}
fn arbitrary_take_rest(_: Unstructured<'_>) -> Result<[T; 0]> {
Ok([])
}
#[inline]
fn size_hint(_: usize) -> (usize, Option<usize>) {
crate::size_hint::and_all(&[])
}
fn shrink(&self) -> Box<dyn Iterator<Item = Self>> {
Box::new(iter::from_fn(|| None))
}
}
arbitrary_array! { 32, (T, a) (T, b) (T, c) (T, d) (T, e) (T, f) (T, g) (T, h)
(T, i) (T, j) (T, k) (T, l) (T, m) (T, n) (T, o) (T, p)
(T, q) (T, r) (T, s) (T, u) (T, v) (T, w) (T, x) (T, y)
(T, z) (T, aa) (T, ab) (T, ac) (T, ad) (T, ae) (T, af)
(T, ag) }

Implement an MSRV policy

I noticed that this crate currently doesn't have an explicit minimum supported Rust version policy, nor does it check the MSRV in CI. With #98, this now depends on ControlFlow, which was introduced in Rust 1.55. I know many crates try to keep the MSRV to at least 6 months back, which would be 1.54 right now. (I'm using arbitrary for Quinn, which is currently checking for 1.53 support.) Since ControlFlow is now part of the public API for 1.x, I suppose reverting this change (or using a local copy of ControlFlow) is not a very attractive option for now, but it would be nice to put something in place to this at least gets noticed in the future.

int_in_range can return out-of-range values

I was able to get 0 out of int_in_range(1..=i32::MAX), which is less than the minimum specified value of 1!

Here's what I think is happening in int_in_range_impl:

  • Suppose when we mod our result we get i32::MAX - 1
  • We then do a wrapping_add of start (1) to give i32::MAX
  • T::from_widest then internally mods by i32::MAX to give 0

I'm not too sure of the general fix here, but I worked around this issue by instead using int_in_range(0..=i32::MAX - 1) + 1.

Avoid returning errors where possible

1d2f653

Next breaking release that we make, I think we can remove the fallibility from
the Arbitrary trait and all the Unstructured methods.

This wasn't addressed in the discussion about 1.0 #62 .

Is this still something we want to do? Are you open for a PR implementing it?

Generate collections via a "continuation byte" rather than reading a length

Rather than doing

let mut collection = Collection::new();
let n = u.arbitrary_len()?;
for _ in 0..n {
    collection.insert(u.arbitrary()?);
}

to create a collection, we should consider doing

let mut collection = Collection::new();
loop {
    let continue = u.arbitrary::<bool>();
    if !continue {
        break;
    }
    collection.insert(u.arbitrary()?);
}

This paper found this second approach to make test case reduction much more effective.

We might not want to do this for strings though, since the fuzzer likely has some smarts around utf-8 sequences that we might break.

Also, we might want to use the size hint to still put an upper limit on how many elements we parse.

Revisit making `String`'s `Arbitrary` implementation fallible

At least during test case reduction, it would be better to use from_utf8_lossy, because we would rather have some replacement chars in a smaller string than fail to generate a Self.

Possibly we could set an environment variable from cargo fuzz tmin that changes this behavior, if we still want test case generation to be fallible rather than use replacement chars.

cc @Manishearth

Arbitrary impl for String isn't great

Even with #8 fixed, we're not generating strings ideally. The current impl treats it as a vec of chars, which is isomorphic to a string but not typically how a fuzzer will think about a string. libFuzzer will try to mutate inputs using strings and other patterns from the program, so it's important that on feeding the fuzz target "banana" in bytes, it sees the same string. Currently you'd need to pass the string \0\0\0\0b\0\0\0\0a\0\0\0\0n... along with a container size at the beginning (which we should ideally fix, see #18)

We should do something more UTF8-aware (but, like the char impl, able to swiftly fall back)

int_in_range_impl can panic on valid ranges

let delta = end - start;

This line can panic if end - start doesn't fit in the range of the number type, such as if start is near i32::MIN and end is near i32::MAX.

I discovered this via my fuzzing in CI (which uses such ranges) starting to fail after the release of arbitrary 1.1.4 a few hours ago.

size_hint() has unclear semantics

The exact semantics of size_hint() are not clear from looking at the documentation and the existing impls. The lower bound specifically could have two different meanings:

  • A hard limit. If there is less data available than specified, arbitrary() is expected to fail, don't even try to call it.
  • A soft limit. All unique outputs can be achieved using larger inputs, so there is no advantage to using inputs shorter than the lower bound - though it may still work.

Additionally, it's unclear if the bounds apply only to arbitrary() or also to arbitrary_take_rest(). For more complex types, these implementations can differ significantly, where arbitrary() often needs some additional metadata bytes (increasing the lower bound), but arbitrary_take_rest() just passes the entire data on to its inner types.

Here are a few examples:

  • Integer types are (size_of::<T>(), Some(size_of::<T>())). However, they are constructed using Unstructured::fill_buffer(), which if it "does not have enough underlying data to fill the whole buffer, it pads the buffer out with zeros.". Thus, it's possible to construct integers even from completely empty buffers, making this a soft limit.
  • Vec<T> is (size_of::<usize>(), None). This is completely incorrect, but assuming usize should be bool, it does describe a soft limit for arbitrary(): the empty vector can be achieved both using the empty input as well as a single false continuation byte, so the empty input is redundant. However, for arbitrary_take_rest(), this does not hold: the only way to get an empty Vec<u8> is from the empty input - the size hint has to be (0, None), which is also the hard limit for arbitrary().
  • &[u8] is (size_of::<usize>(), Some(size_of::<usize>())), which is even more wrong than Vec, but again (size_of::<u8>(), None) would be a correct soft limit for arbitrary() (the empty input is equivalent to a single length byte), but (0, None) is required for arbitrary_take_rest() or as an arbitrary() hard limit.

Integrated Shrinking

In short, Hypothesis provides integrated shrinking without needing to implement a separate shrinker per type by shrinking the fuzzed byte input, rather than the generated value.

I've got an implementation of this in my suppositions crate, and I'm sure there's something similar in proptest too.

"Fill all available space" solution for container size

Broadly, there are two kinds of things Arbitrary generates. Fixed-size objects that map to a fixed set of integers, and variable size objects, which can use something like container_size().

Thing is, unlike, say, quickcheck, we already have a finite number of bytes as input. It would be nice if Arbitrary could be told "hey, I only have X bytes of data and you're my last consumer" as opposed to "slurp as much data as you want, I'll split the difference amongst the next consumers!"

This seems pretty tricky to design in the general case, but we could initially settle for types like String and Vec just filling directly.

Use same `Arbitrary` trait as quickcheck?

It doesn't look like this has been brought up before in this crate. Using a single trait for both cargo-fuzz and quickcheck would be super, but right now it requires two trait implementations that are very similar.

Consider adding a lifetime parameter to Arbitrary

If we tweaked the Arbitrary definition into this:

pub trait Arbitrary<'a>: Sized {
    fn arbitrary(u: &mut Unstructured<'a>) -> Result<Self>;

    // ...
}

Then we would allow implementing Arbitrary for things that borrow from the input string:

struct HasBorrow<'a> {
    borrow: &'a [u8],
}

impl<'a> Arbitrary<'a> for HasBorrow<'a> {
    fn arbitrary(u: &mut Unstructured<'a>) -> arbitrary::Result<HasBorrow<'a>> {
        let size = u.arbitrary_byte_size()?;
        let borrow = u.get_bytes(size)?;
        Ok(HasBorrow { borrow })
    }
}

There are two concerns, off the top of my head:

  1. This is a breaking change, and shouldn't be done lightly. Perhaps delayed until we have more breaking changes that we can roll up into one release.

  2. Right now, Arbitrary requires that types that implement it be 'static and I can't remember why. There sure are a bunch of errors when I remove this bound though. Anyways, this would also have to be removed.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.