Giter Club home page Giter Club logo

threadsafedicts.jl's Introduction

CI Coverage Status

ThreadSafeDicts.jl

A thread-safe Dict type for Julia programming

Structs and Functions

struct ThreadSafeDict{K, V} <: AbstractDict{K, V}
    dlock::Threads.SpinLock
    d::Dict{K, V}
    ThreadSafeDict{K, V}() where V where K = new(Threads.SpinLock(), Dict{K, V}())
    ThreadSafeDict{K, V}(itr) where V where K = new(Threads.SpinLock(), Dict{K, V}(itr))
end
ThreadSafeDict(d::Dict{K, V}) where V where K = ThreadSafeDict{K, V}(d)
ThreadSafeDict() = ThreadSafeDict{Any,Any}()

Struct and constructor for ThreadSafeDict. There is one lock per Dict struct. All functions lock this lock, pass arguments to the d member Dict, unlock the spinlock, and then return what is returned by the Dict.

If there are going to be a large number of threads competing to update the Dict, causing most of the threads to be blocked at any given time, you may be better off keeping a Dict in a separate thread which accepts updates via a Channel of Pairs. YMMMV.

getindex(dict::ThreadSafeDict, k)
setindex!(dict::ThreadSafeDict, k, v)
haskey(dict::ThreadSafeDict, k)
get(dict::ThreadSafeDict, k, v)
get!(dict::ThreadSafeDict, k, v)
pop!(dict::ThreadSafeDict)
empty!(dict::ThreadSafeDict)
delete!(dict::ThreadSafeDict, k)
length(dict::ThreadSafeDict)
iterate(dict::ThreadSafeDict)
iterate(dict::ThreadSafeDict, i)
print(io::IO, dict::ThreadSafeDict)

All of the above methods work as in those of the base Dict type. However, they all lock a spinlock prior to passing the arguments to a base Dict within the struct, then unlock the base Dict prior to returning the function call results. Thus, with a single thread the functions are equivalent to those of a base Dict, but with multiple threads thread access to the underlying Dict is serialized per ThreadSafeDict.

Installation

You may install the package from Github in the usual way, or to install the current master copy:

using Pkg
Pkg.add("http://github.com/wherrera10/ThreadSafeDicts.jl")

threadsafedicts.jl's People

Contributors

jishnub avatar juliatagbot avatar storopoli avatar tortar avatar wherrera10 avatar zengmao avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

threadsafedicts.jl's Issues

Sporadic test failures

I was testing this using 2 threads (julia -t 2)

(ThreadSafeDicts) pkg> test
     Testing ThreadSafeDicts
      Status `/tmp/jl_7WaXf3/Project.toml`
  [e30172f5] Documenter v0.27.2
  [4239201d] ThreadSafeDicts v0.0.4 `~/Dropbox/JuliaPackages/ThreadSafeDicts.jl`
  [8ba89e20] Distributed `@stdlib/Distributed`
  [44cfe95a] Pkg `@stdlib/Pkg`
  [8dfed614] Test `@stdlib/Test`
      Status `/tmp/jl_7WaXf3/Manifest.toml`
  [ffbed154] DocStringExtensions v0.8.5
  [e30172f5] Documenter v0.27.2
  [b5f81e59] IOCapture v0.2.2
  [682c06a0] JSON v0.21.1
  [69de0a69] Parsers v1.1.0
  [4239201d] ThreadSafeDicts v0.0.4 `~/Dropbox/JuliaPackages/ThreadSafeDicts.jl`
  [0dad84c5] ArgTools `@stdlib/ArgTools`
  [56f22d72] Artifacts `@stdlib/Artifacts`
  [2a0f44e3] Base64 `@stdlib/Base64`
  [ade2ca70] Dates `@stdlib/Dates`
  [8ba89e20] Distributed `@stdlib/Distributed`
  [f43a241f] Downloads `@stdlib/Downloads`
  [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
  [b27032c2] LibCURL `@stdlib/LibCURL`
  [76f85450] LibGit2 `@stdlib/LibGit2`
  [8f399da3] Libdl `@stdlib/Libdl`
  [56ddb016] Logging `@stdlib/Logging`
  [d6f4376e] Markdown `@stdlib/Markdown`
  [a63ad114] Mmap `@stdlib/Mmap`
  [ca575930] NetworkOptions `@stdlib/NetworkOptions`
  [44cfe95a] Pkg `@stdlib/Pkg`
  [de0858da] Printf `@stdlib/Printf`
  [3fa0cd96] REPL `@stdlib/REPL`
  [9a3f8284] Random `@stdlib/Random`
  [ea8e919c] SHA `@stdlib/SHA`
  [9e88b42a] Serialization `@stdlib/Serialization`
  [6462fe0b] Sockets `@stdlib/Sockets`
  [fa267f1f] TOML `@stdlib/TOML`
  [a4e569a6] Tar `@stdlib/Tar`
  [8dfed614] Test `@stdlib/Test`
  [cf7118a7] UUIDs `@stdlib/UUIDs`
  [4ec0a83e] Unicode `@stdlib/Unicode`
  [deac9b47] LibCURL_jll `@stdlib/LibCURL_jll`
  [29816b5a] LibSSH2_jll `@stdlib/LibSSH2_jll`
  [c8ffd9c3] MbedTLS_jll `@stdlib/MbedTLS_jll`
  [14a3606d] MozillaCACerts_jll `@stdlib/MozillaCACerts_jll`
  [83775a58] Zlib_jll `@stdlib/Zlib_jll`
  [8e850ede] nghttp2_jll `@stdlib/nghttp2_jll`
  [3f19e933] p7zip_jll `@stdlib/p7zip_jll`
     Testing Running tests...
Test Failed at /home/jishnu/Dropbox/JuliaPackages/ThreadSafeDicts.jl/test/runtests.jl:52
  Expression: dict["number"] > 900
   Evaluated: 500 > 900
ERROR: LoadError: There was an error during testing
in expression starting at /home/jishnu/Dropbox/JuliaPackages/ThreadSafeDicts.jl/test/runtests.jl:56
ERROR: Package ThreadSafeDicts errored during testing

Is there some race condition that still exists?

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Type stability of the underlying dict

The struct definition

struct ThreadSafeDict{K, V} <: AbstractDict{K, V}
    dlock::Threads.SpinLock
    d::Dict
    ThreadSafeDict{K, V}() where V where K = new(Threads.SpinLock(), Dict{K, V}())
    ThreadSafeDict{K, V}(itr) where V where K = new(Threads.SpinLock(), Dict{K, V}(itr))
end

contains d::Dict which has an abstract type. This can cause runtime dispatch and affect performance. It should be easy to change it to d::Dict{K, V}, which also makes better use of the type information already supplied, unless I missed something.

Use a closure to also lock on value

Would it be possible to do something like:

get(dict, key) do
   # within lock
end

So we can reuse the lock and simplify logic while updating the value struct?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.