queryverse / datavoyager.jl Goto Github PK

View Code? Open in Web Editor NEW

128.0 128.0 11.0 3.02 MB

Julia wrapper for the Voyager data exploration tool

License: Other

Julia 0.04% JavaScript 97.10% HTML 0.01% Less 1.41% SCSS 1.43%

julia queryverse voyage

datavoyager.jl's People

Contributors

Stargazers

Watchers

Forkers

danimad tkf stjordanis arbrefleur ryanbrownnetworking777 standardgalactic playfloor ndnihart bryanchance pinkdiamond1

datavoyager.jl's Issues

Electron is not installing properly

Hi,

I "rm" and "add" Electron package 3 times :) however, the following error is not going away. I am using Julia 1.2 in Atom in Windows 10. thanks for looking

using Electron
using DataVoyager
v = Voyager()

julia> v = Voyager()
ERROR: IOError: could not spawn 'C:\Users\drcxcruz\.juliapro\JuliaPro_v1.2.0-1\packages\Electron\aRIgh\src\..\deps\electron\electron.exe' 'C:\Users\drcxcruz\.juliapro\JuliaPro_v1.2.0-1\packages\Electron\aRIgh\src\main.js' '\\.\pipe\juliaelectron-9696-1' '\\.\pipe\juliaelectron-sysnotify-9696-1' /h1S6/csgMKhJv0CmVALgGE4QkkaQ+qVfh+t4F5574t6Sid07pRcAgzrf/Mb7zICisduYR9s6cUIpuAyVoVCab+cMY/KZkOcNZg1qXQrhnBhwQnBPzF2e9Lad1ARgK1YT4ieVWV21JSapAnTTXkaN1az6Vw8xrpecH6D5fRTAms=: no such file or directory (ENOENT)
Stacktrace:
[1] _spawn_primitive(::String, ::Cmd, ::Array{Any,1}) at .\process.jl:401
[2] setup_stdios(::getfield(Base, Symbol("##526#527")){Cmd}, ::Array{Any,1}) at .\process.jl:414
[3] _spawn at .\process.jl:413 [inlined]
[4] #open#535(::Bool, ::Bool, ::typeof(open), ::Cmd, ::Base.TTY) at .\process.jl:662
[5] #open at .\none:0 [inlined]
[6] open(::Cmd, ::String, ::Base.TTY) at .\process.jl:631
[7] Electron._Application{Window}() at C:\Users\drcxcruz.juliapro\JuliaPro_v1.2.0-1\packages\Electron\aRIgh\src\Electron.jl:142
[8] Voyager() at C:\Users\drcxcruz.juliapro\JuliaPro_v1.2.0-1\packages\DataVoyager\7x2Tx\src\DataVoyager.jl:20
[9] top-level scope at none:0

Figure out how to pass type information of columns to voyager

JavaScript errors and EOFError with DataVoyager.

I am running Julia v1.1.1 on Windows 10, with all packages updated to latest versions as of today (28 May 2019). The example code:

julia> using FileIO, CSVFiles, Query, DataVoyager

julia> v = load("Data.csv") |> Voyager()

results in two JavaScript errors popping up

Uncaught Exection:
Error: connect ENOENT \\.\pipe\juliaelectron-13864-1
   at PipeConnectWrap.afterConnect [as oncomplete] (net.js:1113:14)

and

Uncaught Exection:
Error: connect ENOENT \\.\pipe\juliaelectron-sysnotify-13864-1
   at PipeConnectWrap.afterConnect [as oncomplete] (net.js:1113:14)

and in the REPL the following error

ERROR: EOFError: read end of file
Stacktrace:
 [1] unsafe_read(::Base.PipeEndpoint, ::Ptr{UInt8}, ::UInt64) at .\iobuffer.jl:165
 [2] unsafe_read at .\io.jl:585 [inlined]
 [3] macro expansion at .\gcutils.jl:87 [inlined]
 [4] read! at .\io.jl:597 [inlined]
 [5] Electron._Application{Electron.Window}() at C:\Users\vdykb\.julia\packages\Electron\wL84D\src\Electron.jl:136
 [6] Voyager() at C:\Users\vdykb\.julia\packages\DataVoyager\7x2Tx\src\DataVoyager.jl:20
 [7] top-level scope at none:0

I have verified that the file is read successfully. The error only comes in passing the data to Voyager(). I also tried converting to a DataFrame and passing that, but the results are similar.

I am more than happy to run any tests. I can also test this on other machines, if that would help.

Data Voyager not pre-compiling

I am getting this error LoadError: Failed to precompile DataVoyager, every time I try to run
using DataVoyager.
IN particular:
LoadError: Cannot locate artifact 'electronjs_app' for aarch64-apple-darwin-libgfortran5-cxx11-julia_version+1.7.1 in '/Users/moshialam/.julia/packages/Electron/ILvWR/Artifacts.toml'

Can anyone help with what's the possible solution?

"GtkDialog mapped without a transient parent. This is discouraged"

I'm trying to create a simple CLI program to pass a csv to Voyager, like this:

julia  myVoyager.jl  /path/to/local.csv

myVoyager.jl is something simple like this:

df = DataFrame(CSV.read(string(ARGS[1])))
Voyager(df)

When I pull up a dataframe in a REPL or in VS Code, it works fine, but when I try to run this as a CLI program, I get the error: "GtkDialog mapped without a transient parent. This is discouraged", and the window doesn't open.

Ideas?

ERROR: IOError: could not spawn Electron

When I try to execute the example on Windows 10 I get this error:
(I first had to install several packages)

 v = Voyager(data)
ERROR: IOError: could not spawn `'C:\Users\joe\.julia\packages\Electron\IV1np\src\..\deps\electron\electron.exe' 'C:\Users\joe\.julia\packages\Electron\IV1np\src\main.js' '\\.\pipe\juliaelectron-11496-1' '\\.\pipe\juliaelectron-sysnotify-11496-1' lU1QLq2cRozF4CJa3A/oQNgNhgfTEZqyTyI6FU/FJFXEh+cqxxooKzJBf0ywZc7apNTTt4PbQMK/PTo7gJUvN4CyFQ9UpljzL0nnqiqMNcDEfgibYxY7kPgyRg+Z/jo6d6shJJSKMhbar7hjQ1RqDT6CdzFgFWC949yuWg1xLDM=`: no such file or directory (ENOENT)
Stacktrace:
 [1] _jl_spawn(::String, ::Array{String,1}, ::Cmd, ::Tuple{Base.PipeEndpoint,Base.TTY,Base.TTY}) at .\process.jl:367
 [2] (::getfield(Base, Symbol("##494#495")){Cmd})(::Tuple{Base.PipeEndpoint,Base.TTY,Base.TTY}) at .\process.jl:509
 [3] setup_stdio(::getfield(Base, Symbol("##494#495")){Cmd}, ::Tuple{Pipe,Base.TTY,Base.TTY}) at .\process.jl:490
 [4] #_spawn#493(::Nothing, ::Function, ::Cmd, ::Tuple{Pipe,Base.TTY,Base.TTY}) at .\process.jl:508
 [5] _spawn(::Cmd, ::Tuple{Pipe,Base.TTY,Base.TTY}) at .\process.jl:504
 [6] #open#503(::Bool, ::Bool, ::Function, ::Cmd, ::Base.TTY) at .\process.jl:599
 [7] #open at .\none:0 [inlined]
 [8] open(::Cmd, ::String, ::Base.TTY) at .\process.jl:567
 [9] Electron._Application{Electron.Window}() at C:\Users\joe\.julia\packages\Electron\IV1np\src\Electron.jl:133
 [10] Voyager() at C:\Users\joe\.julia\packages\DataVoyager\7x2Tx\src\DataVoyager.jl:20
 [11] Voyager(::DataFrame) at C:\Users\joe\.julia\packages\DataVoyager\7x2Tx\src\DataVoyager.jl:66
 [12] top-level scope at none:0

Figure out how missing values should be passed on to voyager

Cannot `convert` an object of type Dates.Date to an object of type Int64

I am on Julia v1.0 with Queryverse installed. I load a csv from the Bike-Sharing-Dataset that has a date field that looks like 2011-01-01. When I try to pipe the data into Voyager I get the MethodError: Cannot convert an object of type Dates.Date to an object of type Int64. I noticed the cars dataset has a date field that looks like "1970-01-01". Are these quotes needed around the data in Voyager? If so are there any easy methods for casting the data into the correct format?

NOTE: I'm having to use Julia on an offline machine right this minute but when I get home I'll post the full stacktrace.

Tag a release?

I added the package and it downgraded Electron. I see on master it allows: 2, 3.0

Is there a point to 3.0? I'm not sure it's equivalent. 3.1 shouldn't be breaking, but that would disallow it?

And before up compatibility on DataStructures?

Might be bettor on a more recent master (and/or updated dependencies), but not too bad:

julia> @time using DataVoyager
  2.219824 seconds (2.75 M allocations: 172.632 MiB, 1.11% gc time, 52.71% compilation time)

julia> @time using VegaLite
  2.167565 seconds (2.71 M allocations: 170.363 MiB, 2.00% gc time, 52.53% compilation time)

And pretty ok:
$ ./julia -O0 --compile=min --startup-file=no -q
julia> @time using VegaLite
  1.098142 seconds (1.29 M allocations: 88.372 MiB, 2.95% gc time, 2.96% compilation time)

Unable to Convert Categorical Fields

Not to a be a thorn in your side but looks like I uncovered another issue. I was playing around some more with DataFrames and converted a few fields to the Categorical Type and then piped it into Voyager and got a similar error as this one. Its pretty much the same problem as issue #9 . I don't think its a super high priority issue but something to be aware of.

data: eeg-brain-wave-for-confusion

using Queryverse 

macro __str(x)
       :(Symbol($x))
end

# load some data into a dataframe
df = load("/home/mcamp/JuliaProjects/eeg-brain-wave-for-confusion/EEG_data.csv") |> DataFrame

cols = names(df)
cols[15] = _"UserDefinedLabel" 
cols[14] = _"PreDefinedLabel"
names!(df, cols) 

categorical!(df,:SubjectID)
categorical!(df,:UserDefinedLabel)
categorical!(df,:PreDefinedLabel)
categorical!(df,:VideoID)

df |> Voyager()

Retruns:

MethodError: Cannot `convert` an object of type Float64 to an object of type CategoricalValue{Float64,UInt32}
Closest candidates are:
  convert(::Type{T<:(Union{CategoricalString{R}, CategoricalValue{T,R} where T} where R)}, !Matched::T<:(Union{CategoricalString{R}, CategoricalValue{T,R} where T} where R)) where T<:(Union{CategoricalString{R}, CategoricalValue{T,R} where T} where R) at /home/mcamp/.julia/packages/CategoricalArrays/04bks/src/value.jl:65
  convert(::Type{S}, !Matched::T<:(Union{CategoricalString{R}, CategoricalValue{T,R} where T} where R)) where {S, T<:(Union{CategoricalString{R}, CategoricalValue{T,R} where T} where R)} at /home/mcamp/.julia/packages/CategoricalArrays/04bks/src/value.jl:66
  convert(::Type{T}, !Matched::T) where T at essentials.jl:154
  ...

Stacktrace:
 [1] setindex!(::Dict{Symbol,CategoricalValue{Float64,UInt32}}, ::Float64, ::Symbol) at ./dict.jl:381
 [2] Dict{Symbol,CategoricalValue{Float64,UInt32}}(::Base.Generator{Base.Iterators.Zip2{NTuple{15,Symbol},Tuple{CategoricalValue{Float64,UInt32},CategoricalValue{Float64,UInt32},Float64,Float64,Float64,Float64,Float64,Float64,Float64,Float64,Float64,Float64,Float64,CategoricalValue{Float64,UInt32},CategoricalValue{Float64,UInt32}}},getfield(DataVoyager, Symbol("##2#4"))}) at ./dict.jl:104
 [3] dict_with_eltype at ./abstractdict.jl:541 [inlined]
 [4] macro expansion at ./array.jl:591 [inlined]
 [5] dict_with_eltype at ./abstractdict.jl:546 [inlined]
 [6] Dict(::Base.Generator{Base.Iterators.Zip2{NTuple{15,Symbol},Tuple{CategoricalValue{Float64,UInt32},CategoricalValue{Float64,UInt32},Float64,Float64,Float64,Float64,Float64,Float64,Float64,Float64,Float64,Float64,Float64,CategoricalValue{Float64,UInt32},CategoricalValue{Float64,UInt32}}},getfield(DataVoyager, Symbol("##2#4"))}) at ./dict.jl:129
 [7] #1 at ./none:0 [inlined]
 [8] iterate at ./generator.jl:47 [inlined]
 [9] collect(::Base.Generator{TableTraitsUtils.TableIterator{NamedTuple{(:SubjectID, :VideoID, :Attention, :Mediation, :Raw, :Delta, :Theta, :Alpha1, :Alpha2, :Beta1, :Beta2, :Gamma1, :Gamma2, :PreDefinedLabel, :UserDefinedLabel),Tuple{CategoricalValue{Float64,UInt32},CategoricalValue{Float64,UInt32},Float64,Float64,Float64,Float64,Float64,Float64,Float64,Float64,Float64,Float64,Float64,CategoricalValue{Float64,UInt32},CategoricalValue{Float64,UInt32}}},Tuple{CategoricalArray{Float64,1,UInt32,Float64,CategoricalValue{Float64,UInt32},Union{}},CategoricalArray{Float64,1,UInt32,Float64,CategoricalValue{Float64,UInt32},Union{}},Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},CategoricalArray{Float64,1,UInt32,Float64,CategoricalValue{Float64,UInt32},Union{}},CategoricalArray{Float64,1,UInt32,Float64,CategoricalValue{Float64,UInt32},Union{}}}},getfield(DataVoyager, Symbol("##1#3"))}) at ./array.jl:619
 [10] (::Voyager)(::DataFrame) at /home/mcamp/.julia/packages/DataVoyager/h5fKO/src/DataVoyager.jl:36
 [11] |>(::DataFrame, ::Voyager) at ./operators.jl:813
 [12] top-level scope at In[18]:20

Is there a way to update the data in an existing Voyager window?

New release for VegaLite 1.0 compatibility?

Thanks to 4fbc7bc DataVoyager works with VegaLite 1.0 if I dev it. Can this be released as-is?

PS: Thanks for VegaLite 1.0 release! 🎉

How do I exit voyager on Windows Julia command line?

TaskFailedException

I am completely new to julia. I was following an intro youtube video. At some point I did something, and now it keeps popping up the TaskFailedException (error message: stream is closed or unusable) whenever I am trying to run the line. Is there anyway that I can restart it ?

IDE: Atom
OS: MacOS monterey
Machine: MacBook Pro

Thank you.

Find a way to pass large datasets fast

Naming the Voyager window

Would it be possible to have an optional arg to name our Voyager window? I happen to generate several windows from some datasets and it is quickly getting chaotic ><

Uncaught Exception: RangeError: Invalid string length

I'm getting error "Uncaught Exception: RangeError: Invalid string length".

This error comes about when loading the following data with julia> data |> Voyager().

Is there an upper limit on what Voyager can handle? Any assistance would be much appreciated.

Top panel missing?

Running on Julia 1.6.3 I do not see the top panel (A) that provides access to bookmarks and is described at https://data-voyager.gitbook.io/voyager/ Is their something particular I need to do to see that panel in the window? Thanks

`Vector{Any}` containing only `NamedTuples` is not a table

MWE

julia> using DataVoyager

julia> data = Any[(x=1, y=2), (x=2, y=3)]
2-element Array{Any,1}:
 (x = 1, y = 2)
 (x = 2, y = 3)

julia> Voyager(data)
ERROR: 'source' is not a table.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] (::Voyager)(::Array{Any,1}) at /Users/oxinabox/.julia/packages/DataVoyager/7x2Tx/src/DataVoyager.jl:30
 [3] Voyager(::Array{Any,1}) at /Users/oxinabox/.julia/packages/DataVoyager/7x2Tx/src/DataVoyager.jl:68
 [4] top-level scope at none:0

Seems like this might actually be a TableTrait's issue.

DataVoyager.jl/src/DataVoyager.jl

Line 30 in 910a20d

 TableTraits.isiterabletable(source)===false && error("'source' is not a table.") 

Incomplete copy

using VegaDatasets, DataVoyager, VegaLite
v = dataset("cars") |> Voyager()

I clicked copy button to get a vegalite specification of the plot, then the content is incomplete as below. It's missing almost all information needed to reproduce the same plot.

{
  "data": {
    "name": "source"
  },
  "mark": "point",
  "encoding": {
    "x": {
      "field": "Miles_per_Gallon",
      "type": "quantitative",
      "scale": {}
    },
    "y": {
      "field": "Horsepower",
      "type": "quantitative",
      "scale": {}
    }
  },
  "config": {
    "overlay": {
      "line": true
    },
    "scale": {
      "useUnaggregatedDomain": true
    }
  }
}

By the way, if I directly save the file as below, it then has all necessary data intact so that can be loaded up in i.e. Vega Editor.

v[] |> save("figure1.vegalite")

I use DataVoyager v1.0.0 and VegaLite v2.0.1 on macOS.

julia> versioninfo()
Julia Version 1.4.0
Commit b8e9a9ecc6 (2020-03-21 16:36 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.6.0)
  CPU: Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, haswell)

Bump min version on IterableTables.jl to 0.6.1

Error Precompiling DataVoyager

DataVoyager refuses to precompile. It throws this error below- the file exists, though. Not sure why it's telling me to run "Pkg.build(package)", when that doesn't do anything (package not defined, expectedly).

julia> using DataVoyager
[ Info: Precompiling DataVoyager [5721bf48-af8e-5845-8445-c9e18126e773]
ERROR: LoadError: LoadError: LoadError: Unable to load 

_jl_libpangocairo (/usr/lib/x86_64-linux-gnu/libpangocairo-1.0.so.0)

Please re-run Pkg.build(package), and restart Julia.

Accessing millisecond objects

I have a field in a dataframe that I am trying to visualize using Voyager. The data type is milliseconds (delta of two date times). On voyager, this gets displayed as [object Object]. It would be nice for Voyager to interpret the time delta. The current workaround is to convert the milliseconds into integers.

Known security vulnerabilities in two JavaScript dependencies

There are two known security vulnerabilities in JavaScript dependencies specified in this repository:

src/htmlui/package-lock.json specifies cryptiles version 3.1.2. There is a known security vulnerability in this version: CVE-2018-1000620. The recommendation is to upgrade to cryptiles version 4.1.2.
src/htmlui/package-lock.json specifies macaddress version 0.2.8. There is a known security vulnerability in this version: CVE-2018-13797. The recommendation is to upgrade to macaddress version 0.2.9.

Can we update package-lock.json to specify appropriately recent versions of these dependencies?

cc: @davidanthoff @queryverse

Cannot start Voyager because of name of user directory

julia> using DataVoyager

julia> Voyager()

ERROR: Path contained unexpected character
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] parse_url(::String) at C:\Users\Frederik Banning\.julia\packages\URIParser\IuWoB\src\parser.jl:278
 [3] URI at C:\Users\Frederik Banning\.julia\packages\URIParser\IuWoB\src\parser.jl:312 [inlined]
 [4] Voyager() at C:\Users\Frederik Banning\.julia\packages\DataVoyager\71yWt\src\DataVoyager.jl:23
 [5] top-level scope at REPL[5]:1

I suppose the problem might have to do with the space in my user directory path. Is this assumption correct? What can we do to avoid such an error?

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

cannot read property of 'var' undefined

libgobject-2.0.so.0: cannot open shared object file

I just installed DataVoyager 1.0.0 but it's giving me an error:

julia> using DataVoyager
[ Info: Precompiling DataVoyager [5721bf48-af8e-5845-8445-c9e18126e773]

julia> v = Voyager()
/home/tkwong/.julia/artifacts/41afcbdbe95343621162214b9fc07813c12b5d36/electron: error while loading shared libraries: libgobject-2.0.so.0: cannot open shared object file: No such file or directory

Nothing comes up and the call isn't returning...

My version:

julia> versioninfo()
Julia Version 1.4.1
Commit 381693d3df* (2020-04-14 17:20 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) Gold 6144 CPU @ 3.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)