Giter Club home page Giter Club logo

rust-smallvec's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rust-smallvec's Issues

Release a new version with stable alloc

Heyo, would it be possible to get a new release with the removed feature check around the alloc feature #159 ? Otherwise the alloc feature isn't usable on stable compilers. Thanks :).

Possible UB from use of uninitialized `[&T; N]`

Creating an array with mem::uninitialized may be UB if the array's elements contain non-null types (like &T) or uninhabited types (like !) or other types that have invalid values.

This can result in undefined behavior even if you never read the elements directly while they are uninitialized. For example, this program prints "true" if optimizations are enabled, when built with current rustc:

fn main() {
    let x: Option<[&u8; 1]> = unsafe { Some(std::mem::uninitialized()) };
    println!("{}", x.is_none());
}

SmallVec isn't eligible for null pointer optimization, so it isn't affected by this. However, it's possible that a future Rust compiler could add other optimizations that break if SmallVec uses mem::uninitialized to create arrays of such types.

The ideal solution is to switch from uninitialized to the new MaybeUninit union (rust-lang/rfcs#1892) when it becomes stable.

Dropping SmallVecN is unsound

For example, this segfaults:

SmallVec4::<Box<u32>>::new();

SmallVec4<T> contains a [T; 4] field directly, which is initialized in new with std::mem::zeroed(). When the vector is dropped, the destructor for T is run for each of the 4 Ts, even if there isn’t actually a T there (i.e. if the vector’s length is less than 4 by the time it is dropped).

https://github.com/bluss/arrayvec works around this issue by having (simplified):

enum Array<T> {
    Alive([T, 4]),
    Dropped,
}

with a destructor that resets to Dropped before the recursive destructors are run implicitly.

In SmallVecN, the second variant could instead contain the pointer and capacity for a spilled vector (reset to null/zero during destruction).

Move always move the whole `SmallVec` object.

A SmallVec<[f64; 100]> with a capacity() larger than 100 allocates the elements on the heap but moving the SmallVec generates a memcpy over the whole SmallVec object including the unused storage for the 100 f64 elements and not just 3*size.

This seems to be a general problem with enums but I couldn't find a rustc bug to track this, so I don't know if this is intended or even fixable. Anyhow, maybe SmallVec can workaround this behavior somehow?

`cargo bench` broken on Travis

From travis:

     Running `rustc --crate-name bench benches/bench.rs --emit=dep-info,link -C opt-level=3 --test -C metadata=6041378d75069d53 -C extra-filename=-6041378d75069d53 --out-dir /home/travis/build/servo/rust-smallvec/target/release/deps -L dependency=/home/travis/build/servo/rust-smallvec/target/release/deps --extern smallvec=/home/travis/build/servo/rust-smallvec/target/release/deps/libsmallvec-ad2736f846965482.rlib`

error[E0554]: #[feature] may not be used on the stable release channel

 --> benches/bench.rs:1:1

  |

1 | #![feature(test)]

  | ^^^^^^^^^^^^^^^^^

error: aborting due to previous error

error: Could not compile `smallvec`.

Caused by:

  process didn't exit successfully: `rustc --crate-name bench benches/bench.rs --emit=dep-info,link -C opt-level=3 --test -C metadata=6041378d75069d53 -C extra-filename=-6041378d75069d53 --out-dir /home/travis/build/servo/rust-smallvec/target/release/deps -L dependency=/home/travis/build/servo/rust-smallvec/target/release/deps --extern smallvec=/home/travis/build/servo/rust-smallvec/target/release/deps/libsmallvec-ad2736f846965482.rlib` (exit code: 101)

Build failed, waiting for other jobs to finish...

error: build failed

The command "cargo build --verbose &&

cargo build --features=heapsizeof --verbose &&

cargo test --verbose &&

cargo test --features=heapsizeof --verbose &&

([ $TRAVIS_RUST_VERSION != nightly ] || cargo bench --verbose bench)

" exited with 101.

SmallVec<[T; N]> is invariant over T

SmallVec<[T; N]> is invariant over T.
Although [T; N] is covariant over T as expected.

This is due to SmallVec<A> having field of type <A as Array>::Item in underlying union.
I propose to change that field type to A and remove A: Array bound from type declaration.

change drain to be consistent with `Vec`

drain has a different signature than Vec, in case this is something that is worth fixing, I would be glad to post a PR.

SmallVec: pub fn drain(&mut self) -> Drain<A::Item>
Vec: pub fn drain<R>(&mut self, range: R) -> Drain<T> where R: RangeBounds<usize>

Prevent memory unsafety when array offset values are >isize::MAX

From #28:

All of these casts to isize when offsetting are unsafe, since they could become negative numbers and cause us to write outside of the bounds of the array. We should either use to_isize.unwrap() and panic if that occurs, or use a conversion strategy that yields a value that will gives us worse performance but correct behaviour.
At a quick readthrough, it looks like push, pop, truncate, remove, insert all suffer from the same issue if len > isize::MAX.

It looks like one might be able to manipulate that situation in a call to insert() when len == isize::MAX (then the len would get set to size::MAX + 1).

use-after-free when growing to the same size

Attempting to call grow on a spilled SmallVec with a value equal to the current capacity causes it to free the existing data.

let mut v: SmallVec<[u8; 2]> = SmallVec::new();
v.push(0);
v.push(1);
// Cause it to spill.
v.push(2);
assert_eq!(v.capacity(), 4);
// grow with the same capacity
v.grow(4);
// The backing storage is now freed, here be dragons.
println!("{:?}", v);

In the grow method it falls through to deallocate.

I believe something like the following is the fix:

diff --git a/lib.rs b/lib.rs
index 5af32ba..b81b35a 100644
--- a/lib.rs
+++ b/lib.rs
@@ -664,6 +664,8 @@ impl<A: Array> SmallVec<A> {
                 if unspilled {
                     return;
                 }
+            } else {
+                return;
             }
             deallocate(ptr, cap);
         }

Is there a way to convert a SmallVec<A> into an A?

I have need for a variable-length array that is almost always exactly length 32 -- so I'd like to represent it as a SmallVec<[T; 32]>. I'd like to convert it to a [T; 32] without copying. It would be cool to add a method that can try doing the conversion if possible and returns the original SmallVec if it didn't work out:

fn into_array(self) -> Result<A, Self>

What do you think?

Constant generic

Constant generic is becoming more mature and I would like to know if is someone porting the existing code to use it behind a feature flag. If not, I might do it myself if you guys don't mind it.

Cursor<SmallVec<[u8; N]>> does not implement Write

I'd be pretty surprised if this weren't implemented for no good reason, but why is SmallVec not a Read? I'd like to be able to use it with the byteorder create like this:

    let mut cursor = std::io::Cursor::new(SmallVec::<[u8; 8]>::new());
    cursor.write_u16::<LittleEndian>(rand::random()).unwrap();
    cursor.write_u16::<LittleEndian>(rand::random()).unwrap();
    cursor.write_u16::<LittleEndian>(rand::random()).unwrap();
    cursor.write_u16::<LittleEndian>(rand::random()).unwrap();

    cursor.set_position(0);

    cursor.read_f64::<LittleEndian>().unwrap()

but this requires that SmallVec implement the Read trait.

Should we try to be more formal about safety?

e.g. I could envision a fn upholds_invariants() -> bool method that could be debug_assert!ed on all methods that use unsafe, thereby allowing us to fuzz-check for mistakes we may have made w.r.t. memory safety. Alas, there are some invariants we cannot check for now (namely around uninitialized memory), but we could at least document them.

We could also add more comments describing why what we do really should be safe, so others can understand – and try to poke holes into – our thinking.

.into_iter() doesn't work

The following code:

extern crate smallvec;

fn main() {
    let mut vec: smallvec::SmallVec<[u8; 8]> = smallvec::SmallVec::new();

    vec.push(5);

    println!("{:?}", vec.len());

    for num in vec.into_iter() {
        println!("{:?}", num);
    }
}

Prints:

1

Expected:

1
5

This is probably because into_iter() sets the length to 0, and then only calls iter_mut(), therefore getting an iterator of length 0.

Release new Cargo version

Is it possible to release a new version on cargo? Having the last two PRs would be helpful (#12, #15).

Thanks, and thanks for the package!

Understanding deserialization performance of SmallVec vs Vec

I was profiling my serde deserialization library and I went down the rabbit hole of benchmarking different KV container storage types. Got down to Vec<(&'a[u8], &'a[u8]> as the fastest and I wanted to see if my code would be faster by putting those string pointers on the stack. So I tried out SmallVec, was actually a little bit slower (20%) than vec. Should this be expected? I did not consider memory gains but the deserialization change doesn't make sense to me.

I extract my benchmark to run inside of rust-smallvec:


#[macro_use]
extern crate smallvec;
extern crate test;
extern crate bincode;
extern crate serde;
#[macro_use] extern crate serde_derive;


use self::test::Bencher;
use smallvec::{ExtendFromSlice, SmallVec};

#[bench]
fn smallvec_i32_benchmark(b: &mut Bencher) {

    use self::bincode::{serialize, deserialize};
    let data = {
        let tinyvec : SmallVec<[i32; 5]> = smallvec![1,2,3,4,5];
        let sv = tinyvec;
        serialize(&sv).unwrap()
    };

    b.iter(|| {
        let tinyvec_2 : SmallVec<[i32; 5]> = deserialize(&data[..]).unwrap();
    });
}

#[bench]
fn vec_i32_benchmark(b: &mut Bencher) {
    use self::bincode::{serialize, deserialize};
    let data = {
        let tinyvec : Vec<i32> = vec![1,2,3,4,5];
        let sv = tinyvec;
        serialize(&sv).unwrap()
    };

    b.iter(|| {
        let tinyvec_2 : Vec<i32> = deserialize(&data[..]).unwrap();
    });
}

#[bench]
fn smallvec_tuple_benchmark(b: &mut Bencher) {

    use self::bincode::{serialize, deserialize};
    let data = {
        let k1 = "hey";
        let v1 = "now";
        let k2 = "you're";
        let v2 = "an";
        let k3 = "all";
        let v3 = "star";
        let k4 = "get";
        let v4 = "your";
        let k5 = "game";
        let v5 = "one";
        let tinyvec : SmallVec<[(&str, &str); 5]> = smallvec![(k1, v1), (k2, v2), (k3, v3), (k4, v4), (k5, v5)];
        let sv = tinyvec;
        serialize(&sv).unwrap()
    };

    b.iter(|| {
        let tinyvec_2 : SmallVec<[(&str,&str); 5]> = deserialize(&data[..]).unwrap();
    });
}

#[bench]
fn vec_tuple_benchmark(b: &mut Bencher) {
    use smallvec::{ExtendFromSlice, SmallVec};

    use self::bincode::{serialize, deserialize};
    let data = {
        let k1 = "hey";
        let v1 = "now";
        let k2 = "you're";
        let v2 = "an";
        let k3 = "all";
        let v3 = "star";
        let k4 = "get";
        let v4 = "your";
        let k5 = "game";
        let v5 = "one";
        let regvec : Vec<(&str, &str)> = vec![(k1, v1), (k2, v2), (k3, v3), (k4, v4), (k5, v5)];
        let v = regvec;
        serialize(&v).unwrap()
    };

    b.iter(|| {
        let tinyvec_2 : Vec<(&str,&str)> = deserialize(&data[..]).unwrap();
    });
}

on my laptop this spits out these numbers:

running 4 tests
test smallvec_i32_benchmark   ... bench:          84 ns/iter (+/- 10)
test smallvec_tuple_benchmark ... bench:         222 ns/iter (+/- 23)
test vec_i32_benchmark        ... bench:          45 ns/iter (+/- 4)
test vec_tuple_benchmark      ... bench:         186 ns/iter (+/- 20)

test result: ok. 0 passed; 0 failed; 0 ignored; 4 measured; 0 filtered out

smallvec! macro requires T to be Copy

The smallvec! macro, when using the smallvec![a,b,c] form, requires that T: Copy.

This is because the macro expands to use SmallVec::from_slice.

    ($($x:expr),*) => ({
        SmallVec::from_slice(&[$($x),*])
    });

Perhaps it could instead expand into something like...

let s = SmallVec::new();
s.push(a);
s.push(b);
s.push(c);
s

... and remove the need for T: Copy?

from_elem is unsound when clone can panic

If clone panics during SmallVec::from_elem then uninitialized memory is dropped. [Original report by dbaupp on Reddit.]

This bug was introduced by #93 which is not yet included in the latest published release (0.6.2).

The suggested solution is to use something like SetLenOnDrop to make sure the length is set correctly when destructors run, without inhibiting optimizations.

Option<SmallVecN<T>> breaks the non-zero optimization

This currently fails:

assert!(Some(SmallVec4::<&u32>::new()).is_some());

This is because SmallVec<&u32> contains a [&u32; 4] field directly, and the compiler choses to use one of those &u32 (which are non-zero) to set to zero to represent None in Option<SmallVec<&u32>>. However, new initializes the array with std::mem::zeroed(), which conflicts with this optimization.

https://github.com/bluss/arrayvec works around this by using #[repr(u8)] on an enum. (The one used to make drop sound, see #4.)

CC @pcwalton

Dedup functionality

I'd like to .sort() and .dedup() the SmallVecs so I can see which values are unique. Failing that, I guess SmallHashSet would work too.

error when updating rustc

I'm trying to update rust-lang/rust to the latest smallvec (0.6.7 ➙ 0.6.9), and an running into an issue where rustc fails to compile:

error: internal compiler error: src/librustc/ty/relate.rs:773: impossible case reached: can't relate: '_#0r with Type(RustaceansAreAwesome)

I have narrowed down the change that breaks this to #137.

I don't know anything about smallvec and very little about rustc, so if anyone can help figure out what's wrong, I'd appreciate it. I'll keep digging a little, but it is unlikely that I'll be able to do more. This can be reproduced with cargo update -p smallvec; ./x.py build (on master, 021a5033098ff0e3f7126acc7ac35149d325f16d).

backtrace
thread 'rustc' panicked at 'Box', src/librustc_errors/lib.rs:637:9
stack backtrace:
   0: std::sys_common::backtrace::print
   1: std::panicking::default_hook::{{closure}}
   2: std::panicking::default_hook
   3: rustc::util::common::panic_hook
   4: std::panicking::rust_panic_with_hook
   5: std::panicking::begin_panic
   6: rustc_errors::Handler::bug
   7: rustc::util::bug::opt_span_bug_fmt::{{closure}}
   8: rustc::ty::context::tls::with_opt::{{closure}}
   9: rustc::ty::context::tls::with_context_opt
  10: rustc::ty::context::tls::with_opt
  11: rustc::util::bug::opt_span_bug_fmt
  12: rustc::util::bug::bug_fmt
  13: ::relate
  14: <&mut I as core::iter::traits::iterator::Iterator>::next
  15:  as core::iter::traits::collect::FromIterator<::Item>>::from_iter
  16:  as rustc::ty::context::InternIteratorElement>::intern_with
  17: rustc::ty::relate::relate_substs
  18: ::relate
  19: rustc::infer::higher_ranked::::higher_ranked_sub
  20: rustc::infer::InferCtxt::commit_if_ok
  21: rustc::traits::select::SelectionContext::match_poly_trait_ref
  22: rustc::infer::InferCtxt::probe
  23: rustc::traits::select::SelectionContext::assemble_candidates
  24: rustc::traits::select::SelectionContext::candidate_from_obligation_no_cache
  25: rustc::dep_graph::graph::DepGraph::with_anon_task
  26: rustc::traits::select::SelectionContext::candidate_from_obligation
  27: rustc::traits::select::SelectionContext::evaluate_stack
  28: rustc::dep_graph::graph::DepGraph::with_anon_task
  29: rustc::traits::select::SelectionContext::evaluate_predicate_recursively
  30: rustc::traits::select::SelectionContext::evaluate_predicates_recursively
  31: rustc::infer::InferCtxt::probe
  32: rustc::traits::select::SelectionContext::evaluate_stack
  33: rustc::dep_graph::graph::DepGraph::with_anon_task
  34: rustc::traits::select::SelectionContext::evaluate_predicate_recursively
  35: rustc::infer::InferCtxt::probe
  36: rustc::traits::select::SelectionContext::evaluate_obligation_recursively
  37: rustc::ty::context::GlobalCtxt::enter_local
  38: rustc_traits::evaluate_obligation::evaluate_obligation
  39: rustc::ty::query::__query_compute::evaluate_obligation
  40: rustc::ty::query::::compute
  41: rustc::dep_graph::graph::DepGraph::with_task_impl
  42: rustc::ty::query::plumbing::::get_query
  43: rustc::traits::query::evaluate_obligation::::evaluate_obligation
  44: rustc::traits::query::evaluate_obligation::::evaluate_obligation_no_overflow
  45: rustc::ty::context::GlobalCtxt::enter_local
  46: rustc::traits::object_safety::::virtual_call_violation_for_method
  47:  as core::iter::traits::iterator::Iterator>::next
  48:  as alloc::vec::SpecExtend>::from_iter
  49: rustc::traits::object_safety::::object_safety_violations_for_trait
  50:  as core::iter::traits::iterator::Iterator>::next
  51:  as alloc::vec::SpecExtend>::from_iter
  52: rustc::traits::object_safety::is_object_safe_provider
  53: rustc::ty::query::::compute
  54: rustc::dep_graph::graph::DepGraph::with_task_impl
  55: rustc::ty::query::plumbing::::get_query
  56: rustc_data_structures::obligation_forest::ObligationForest::process_obligations
  57: ::select_where_possible
  58: ::select_all_or_error
  59: rustc_typeck::check::FnCtxt::select_all_obligations_or_error
  60: rustc::ty::context::GlobalCtxt::enter_local
  61: rustc_typeck::check::wfcheck::check_item_well_formed
  62: rustc::ty::query::__query_compute::check_item_well_formed
  63: rustc::ty::query::::compute
  64: rustc::dep_graph::graph::DepGraph::with_task_impl
  65: rustc::ty::query::plumbing::::get_query
  66: ::visit_item
  67: __rust_maybe_catch_panic
  68: rustc_data_structures::sync::par_for_each_in
  69: __rust_maybe_catch_panic
  70: rustc::hir::Crate::par_visit_all_item_likes
  71: rustc::util::common::time
  72: rustc_typeck::check_crate
  73: rustc_interface::passes::analysis
  74: rustc::ty::query::__query_compute::analysis
  75: rustc::dep_graph::graph::DepGraph::with_task_impl
  76: rustc::ty::query::plumbing::::get_query
  77: rustc::ty::context::tls::enter_global
  78: rustc_interface::passes::BoxedGlobalCtxt::access::{{closure}}
  79: rustc_interface::passes::create_global_ctxt::{{closure}}
  80: rustc_interface::interface::run_compiler_in_existing_thread_pool
  81: std::thread::local::LocalKey::with
  82: scoped_tls::ScopedKey::set
  83: syntax::with_globals

Using `grow` to shrink can cause corruption.

If grow is given a size that is within the inline size after a SmallVec has been spilled, the resulting value is corrupted.

let mut v: SmallVec<[u8; 4]> = SmallVec::new();
v.push(1);
v.push(2);
v.push(3);
v.push(4);
// Cause it to spill.
v.push(5);
assert!(v.spilled());
v.clear();
// Shrink to inline.
v.grow(2);
// This should crash with 'entered unreachable code'.
println!("after grow {:?} len={} cap={}", v, v.len(), v.capacity());

This is admittedly unusual, most code would use shrink_to_fit.

I think the following should fix it.

diff --git a/lib.rs b/lib.rs
index 5af32ba..3767e3b 100644
--- a/lib.rs
+++ b/lib.rs
@@ -654,6 +654,7 @@ impl<A: Array> SmallVec<A> {
                 }
                 self.data = SmallVecData::from_inline(mem::uninitialized());
                 ptr::copy_nonoverlapping(ptr, self.data.inline_mut().ptr_mut(), len);
+                self.capacity = len;
             } else if new_cap != cap {
                 let mut vec = Vec::with_capacity(new_cap);
                 let new_alloc = vec.as_mut_ptr();

release 0.6?

It seems that crates.io still has old version.

0.6.9 doesn't compile: #![feature] may not be used on the stable release channel

Just tried to cargo install racer, which tried to install smallvec v0.6.9, and it failed with this error:

error[E0554]: #![feature] may not be used on the stable release channel
  --> /home/nic/.cargo/registry/src/github.com-1ecc6299db9ec823/smallvec-0.6.9/lib.rs:33:32
   |
33 | #![cfg_attr(feature = "union", feature(untagged_unions))]
   |                                ^^^^^^^^^^^^^^^^^^^^^^^^

error[E0554]: #![feature] may not be used on the stable release channel
  --> /home/nic/.cargo/registry/src/github.com-1ecc6299db9ec823/smallvec-0.6.9/lib.rs:35:37
   |
35 | #![cfg_attr(feature = "may_dangle", feature(dropck_eyepatch))]

I have barely any idea what this even means, but am I building wrong, or is this a real error?

Performance benchmarks

Do you have performance benchmarks showing if a SmallVec is actually faster than a normal vec? The benchmarks I came up with are showing that Vec is actually faster, or at least comparable.

#[bench]
fn bench_normal(b: &mut Bencher) {
    b.iter(|| {
        for _ in 0..10000 {
            let mut v = Vec::with_capacity(32);
            for _ in 0..32 {
                v.push(1)
            }
        }
    });
}

#[bench]
fn bench_smallvec(b: &mut Bencher) {
    b.iter(|| {
        for _ in 0..10000 {
            let mut v = SmallVec32::new();
            for _ in 0..32 {
                v.push(1)
            }
        }
    });
}

test bench_normal   ... bench:    421725 ns/iter (+/- 17460)
test bench_smallvec ... bench:    448904 ns/iter (+/- 18151)
#[bench]
fn bench_normal(b: &mut Bencher) {
    b.iter(|| {
        for _ in 0..10000 {
            let mut v = Vec::with_capacity(32);
            for _ in 0..33 {
                v.push(1)
            }
        }
    });
}

#[bench]
fn bench_smallvec(b: &mut Bencher) {
    b.iter(|| {
        for _ in 0..10000 {
            let mut v = SmallVec32::new();
            for _ in 0..33 {
                v.push(1)
            }
        }
    });
}

test bench_normal   ... bench:    749297 ns/iter (+/- 24251)
test bench_smallvec ... bench:   1295246 ns/iter (+/- 112177)

Add to_smallvec for the slice primitive type

The slice primitive type has a to_vec method which makes it convenient to convert a slice to a vector. It would be great if this crate could define an extension trait that added a method called to_smallvec (or some other name) which did the same thing for SmallVec.

This makes it particularly convenient to port code to using SmallVec since you can just import that extension trait instead of converting all the calls to to_vec. 😃

Properly disclose #148 and #149

Recently reported #148 and #149 are memory safety issues and may be exploitable. They need to be properly disclosed:

  1. Identify the affected crate versions
  2. Yank vulnerable versions from crates.io
  3. File an advisory in https://github.com/RustSec/advisory-db

I have tried to track down the commits that introduced these bugs:

  • #148 is caused by 4d02e41 and affects versions 0.6.5 onwards until the fix is landed in 0.6.10
  • My testcase points to #149 being introduced in b24b3d2 but that doesn't make sense to me. If I am correct, that would make 0.6.3 and later vulnerable; possibly 0.6.2 as well but there is no 0.6.2 tag in git, so cannot check that.

The testcase I'm using to check for #149 is as follows:

let mut v: SmallVec<[u8; 2]> = SmallVec::new();
v.push(1);
v.push(2);
v.push(3);
assert!(v.spilled());
v.clear();
// Shrink to inline.
v.grow(2);
assert_eq!(v.capacity(), 2);

[nitpicking] The `Array` trait should be unsafe

It is possible to make the SmallVec segfault with only safe code by writing a wrong implementation of the Array trait:

impl Array for Foo {
    type Item = u8;
    fn size() -> usize { 8 }
    fn ptr(&self) -> *const u8 { std::ptr::null() }
    fn ptr_mut(&mut self) -> *mut u8 { std::ptr::null_mut() }
}

Panic safety in drop

@Stebalien discovered panic safety issue bluss/arrayvec#3 and it applies to smallvec as well.

My understanding is: SmallVec::drop first attempts to drop every element, then it inhibits the drop of the inner array. The panic safety issue is that a panic during drop of an element means the inhibition is never reached, so the inner data can be dropped again.

Testcase adapted from @Stebalien

    #[test]
    #[should_panic]
    fn test_drop_panic_smallvec() {
        // This test should only panic once, and not double panic,
        // which would mean a double drop
        struct DropPanic;

        impl Drop for DropPanic {
            fn drop(&mut self) {
                panic!("drop");
            }
        }

        let mut v = SmallVec::<[_; 1]>::new();
        v.push(DropPanic);
    }

Using "mem::unitialized()" can lead to "Attempted to instantiate uninhabited type"

The following code panics:

fn main() {
    enum Void {}
    let sv = smallvec::SmallVec::<[Void; 8]>::new();
}

With:

thread 'main' panicked at 'Attempted to instantiate uninhabited type [main::Void; 8]', /rustc/ceb2512144d1fc26330e85fb9d41c22ba1866259/src/libcore/mem.rs:630:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:70
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:58
             at src/libstd/panicking.rs:200
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:215
   4: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:478
   5: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:385
   6: rust_begin_unwind
             at src/libstd/panicking.rs:312
   7: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
   8: core::panicking::panic
             at src/libcore/panicking.rs:49
   9: core::mem::uninitialized
             at /rustc/ceb2512144d1fc26330e85fb9d41c22ba1866259/src/libcore/mem.rs:630
  10: <smallvec::SmallVec<A>>::new
             at /home/pierre/.cargo/registry/src/github.com-1ecc6299db9ec823/smallvec-0.6.8/lib.rs:402
  11: playpen::main
             at src/main.rs:3
  12: std::rt::lang_start::{{closure}}
             at /rustc/ceb2512144d1fc26330e85fb9d41c22ba1866259/src/libstd/rt.rs:64
  13: std::panicking::try::do_call
             at src/libstd/rt.rs:49
             at src/libstd/panicking.rs:297
  14: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:92
  15: std::rt::lang_start_internal
             at src/libstd/panicking.rs:276
             at src/libstd/panic.rs:388
             at src/libstd/rt.rs:48
  16: std::rt::lang_start
             at /rustc/ceb2512144d1fc26330e85fb9d41c22ba1866259/src/libstd/rt.rs:64
  17: main
  18: __libc_start_main
  19: _start

The culprit is here:

data: SmallVecData::from_inline(mem::uninitialized()),

"Why would someone create a SmallVec of Void?" you may ask.
In my situation, the type (Void here) is a template parameter of one of my structs.
When the user uses a Void, the SmallVec will of course always remain empty. However creating an empty SmallVec shouldn't itself lead to a panic.

Add opt-in `may_dangle` support

A couple of times now I've tried to use SmallVec as a drop-in replacement for Vec within rustc, and it hasn't worked because SmallVec doesn't have #[may_dangle] annotations. (rust-lang/rust#55525 is one case; rust-lang/rust#56269 is the other.) That annotation is unstable so I think it would have to be behind a feature flag, but that would suffice for my purposes.

I'm not sure about the right way to do this, though. I tried changing this line:

impl<A: Array> Drop for SmallVec {

to this:

unsafe impl<#[may_dangle] A: Array> Drop for SmallVec {

For the current code I'm working on (rust-lang/rust#56269), where Vec works but SmallVec doesn't, this reduced the number of borrow checker errors from 4 to 2. So it helped, but didn't solve things completely. Maybe there's a better way to do it.

`SmallVec::insert_many` is unsound

Gist here. A solution to this would be to set len = index before iterating. Obviously this would cause leaks but we're already leaking data.

(Sorry for the very verbose explanation in the linked gist but I wanted to link it as part of an article whose audience might not immediately understand the problem).

[meta] Wishlist for smallvec 1.0

This is a list of breaking changes that we would like to make before stabilizing the API for smallvec 1.0.

Increases the minimum Rust version from 1.20:

  • Use the alloc crate for no_std compatibility (#159, Requires Rust 1.36).
  • Replace mem::uninitialized with MaybeUninit (#126, Requires Rust 1.36).
  • Use NonNull<T> instead of *mut T (#171, Requires Rust 1.25).
  • Replace the unreachable crate with std::hint::unreachable_unchecked (#128, Requires Rust 1.27).

Breaking API changes:

  • Change drain method to take a RangeBounds argument (#142, Requires Rust 1.28).

Deprecations:

  • Remove deprecated SmallVecN type aliases.
  • Remove deprecated push_all_move method.
  • Remove deprecated VecLike trait.

Still depends on unstable Rust features. Postpone until 2.0:

  • Use const generics to change SmallVec<[T; N]> to SmallVec<T, N>. Remove the Array trait. (rust-lang/rust#44580).
  • Support custom allocators (#55, rust-lang/rust#32838).
  • Use specialization to optimize extend, insert_many, and from for slices of Copy types. Deprecate and remove extend_from_slice, insert_from_slice, and from_slice (rust-lang/rust#31844).
  • Use the union representation by default and remove the optional union feature added in #94 (rust-lang/rust#32836).

Use in constant expressions

It would be useful to be able to use SmallVec in constants (long shot - having BigInt with associated constants for ONE, ZERO, etc). This would require exposing SmallVecData enum. Any concerns/objections?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.