Splitting JLD into its own package about jld.jl HOT 15 CLOSED

🎆

from jld.jl.

We're now even passing Travis tests: https://travis-ci.org/JuliaLang/JLD.jl, https://travis-ci.org/timholy/HDF5.jl. Will want to turn on OSX & AppVeyor for this repo.

Part of what also helped me decide to do this was discovering https://github-issue-mover.appspot.com/. I haven't tried it yet, though. I should ask: does anyone have any security concerns? It needs GitHub API access to the repos.

from jld.jl.

@timholy At some point I should also tell you that, after some discussion with @JeffBezanson at JuliaCon, I'm working on reimplementing JLD in pure Julia (no HDF5 dependency), which is another reason to split the repositories. The format will be a subset of HDF5, so other software will be able to read our files, but probably won't be able to modify/write them at least at first since I'm only implementing some of HDF5. I did some proof of concept benchmarking and it seems like we should be able to get pretty close to serialize in terms of performance for the current JLD torture case of a Vector{Any} of Ints.

from jld.jl.

does anyone have any security concerns?

Source is at https://github.com/google/github-issue-mover, and the instance there is official insofar as it's mentioned in the README, so this is down to "is there any motive for Googlers to do anything screwy here." While it is not an official product, the company's name is associated so there is a reputational factor. I would think that once you're done you can pull its API access if there were long-term concerns.

from jld.jl.

This is awesome, on both counts @timholy and @simonster!

I agree; I don't think that using that tool opens us up to undue risk. Definitely revoke privileges after you're done, though.

from jld.jl.

Sounds very cool, @simonster.

In the absence of bug reports about this, I will proceed with packaging. Then we'll see how it fares in the wild 😄.

from jld.jl.

Sometime I'd be interested in hearing the thinking on moving away from libhdf5. Seems very ambitious. Not that I didn't think about doing just that more than once myself.

from jld.jl.

What's the dependency direction here? This version of JLD will depend on the HDF5 package?

from jld.jl.

Yep. HDF5 lies at the center, specific "dialects" (MAT, JLD, and hopefully someday H5py) specialize it.

from jld.jl.

Cool, makes sense, thanks. Looking at the amount of code deletion from the split commits, this does look like a beneficial direction.

Will want to turn on OSX

Unfortunately they haven't been accepting osx requests for quite some time http://docs.travis-ci.com/user/multi-os/

If we're desperate, one approach would be to fake Travis into only running on OSX via language: objective-c then doing some extra manual steps to install Julia, and use some non-Travis CI service for Linux (which would also need manual steps since no others support language: julia afaik).

from jld.jl.

After installing HDF5 without issue, trying to add JLD package, getting the following error:

        _

_ _ ()_ | A fresh approach to technical computing
() | () () | Documentation: http://docs.julialang.org
_ _ | | __ _ | Type "help()" for help.
| | | | | | |/ ` | |
| | || | | | (| | | Version 0.3.10 (2015-06-24 13:54 UTC)
/ |_'|||__'| | Official http://julialang.org release
|__/ | x86_64-linux-gnu

julia> Pkg.add("JLD")
ERROR: unknown package JLD
in wait at task.jl:51
in sync_end at ./task.jl:311
in add at pkg/entry.jl:319
in add at pkg/entry.jl:71
in anonymous at pkg/dir.jl:28
in cd at ./file.jl:20
in cd#229 at ./pkg/dir.jl:28
in add at pkg.jl:20

Also added push!(Sys.DL_LOAD_PATH, "/opt/local/lib") to .juliarc.jl

Please help.

from jld.jl.

You probably just need to say Pkg.update(). The issue is that your local installation of the package manager hasn't been informed that JLD exists. Update the package manager's METADATA, and you should be good to go. Please let us know whether this works, and if so you can close the issue.

from jld.jl.

yup...did the trick. Thanks, you can close the issue.

from jld.jl.

+1

from jld.jl.

Travis is accepting OSX requests again, contact [email protected] while they've still got capacity.

from jld.jl.