Thread safe way to check existence of items about parallel-hashmap HOT 8 CLOSED

Good to know! And thanks again for this awesome work and all your help!

from parallel-hashmap.

Yes, if the map is not modified, it is perfectly safe to check whether it contains keys from multiple threads.

from parallel-hashmap.

Thanks Greg! Another question please: if I have a lot of (tens of millions maybe) simple <int, int> pairs to insert into a hash map, is it a good idea to use a parallel_flat_hash_map and insert using multiple threads? My concern is that the number of data to insert is huge, and a single insert operation is lightweight, which may make the mutex lock a overhead.

from parallel-hashmap.

Sure, it still would be much faster to use a parallel_flat_hash_map and multiple threads, just make the N template parameter larger than the default 4, maybe 10 or something, and there will be very little mutex contention.

Another possibility. Where are all those pairs you want to insert coming from? If you can iterate very quickly over them, you can actually insert in a parallel_flat_hash_map without locking at all.

from parallel-hashmap.

Another possibility. Where are all those pairs you want to insert coming from? If you can iterate very quickly over them, you can actually insert in a parallel_flat_hash_map without locking at all.

Hmm... I have an array storing m offsets of a file. What I want to do is iterate over the array, read some bytes starting at the offset into memory, then store the pair <offset, index> in the map. So I don't think it can be done quickly. But how can iterating quickly make the insertion lock-free anyway?

from parallel-hashmap.

Actually that would work well. The parallel-flat-hash internally has an array of submaps (When N=4 you have 16 submaps).
What you would do is start 16 threads, and each thread would populate its own submap (thread # 0 populate submap 0, etc.....
So each thread would:

• iterate over all the offsets in the file.
• for each offset, check which submap it would go to (using the submap function).
  If not its target submap, it would do nothing.
  If it is, it would read some bytes starting at the offset into memory, then store the pair <offset, index> in the map.

that's it, no locking necessary.

This is what the bench does here. I really should write a better example.

from parallel-hashmap.

That's cool! By the way, when using a lock-free parallel hash map, is there a difference among insertion methods, like operator[]/insert/emplace?

from parallel-hashmap.

If you use the method I indicated above, use emplace_with_hash so you pass the hashval and it is not recomputed. Otherwise it doesn't make much difference when you insert a pair of integers.

from parallel-hashmap.