Giter Club home page Giter Club logo

mbrainz-sample's Introduction

Datomic MusicBrainz sample database

Datomic is a database of flexible, time-based facts, supporting queries and joins with elastic scalability, and ACID transactions.

MusicBrainz is an open music encyclopedia that collects music metadata and makes it available to the public. This sample project uses the MusicBrainz dataset, but is in no way affiliated with or sponsored by MusicBrainz.

The MusicBrainz dataset makes a great example database for learning, evaluating, or testing Datomic. To create this sample database, we have exported the MusicBrainz distribution database as EDN data files, imported that data into Datomic according to the Schema described below, and backed up that database.

Included in this project are:

  • Instructions for downloading and restoring the Datomic backup to your local transactor
  • Datomic Datalog rules to be composed together to create interesting queries
  • Some sample queries as a starting point

Getting Started

Getting Datomic

Follow the directs in https://docs.datomic.com/pro/releases.html to download Datomic Pro and unzip it somewhere convenient.

Start the transactor:

cd datomic-pro-$VERSION
bin/transactor config/samples/dev-transactor-template.properties

Getting the Data

Next download the subset of the mbrainz database covering the period 1968-1973 (which the Datomic team has scientifically determined as being the most important period in the history of recorded music):

wget https://s3.amazonaws.com/mbrainz/datomic-mbrainz-1968-1973-backup-2017-07-20.tar -O mbrainz.tar
tar -xvf mbrainz.tar

From the datomic-pro-$VERSION directory, restore the backup:

# prints progress -- ~1,000 segments in restore
bin/datomic restore-db file://path/to/backup/mbrainz-1968-1973 datomic:dev://localhost:4334/mbrainz-1968-1973

Getting the Code

Clone this git repo somewhere convenient:

git clone [email protected]:Datomic/mbrainz-sample.git
cd mbrainz-sample

Running the examples

From Java

Fire up your favorite IDE, and configure it to use both the included pom.xml and the following Java options when running:

-Xmx2g -server

Then visit the queries page.

From Clojure

Start up a Clojure REPL:

# from the root of this mbrainz-sample repo
lein repl

Then connect to the database and run the queries.

Schema

Here is a diagram of the relationships:

Mbrainz Relationships

For information about the schema in general, or about individual entities and their attributes, please see the schema page in the wiki, or the EDN schema itself.

Example Queries and Rules

Please see the queries page in the wiki.

Thanks

We would like to thank the MusicBrainz project for defining and compiling a great dataset, and for making it freely available.

Contributing

This tutorial is developed internally by Cognitect. Issues can be filed using Github Issues. We do not accept pull request or patches.

License

Copyright © Metadata Partners, LLC. All rights reserved.

Distributed under the Eclipse Public License, the same as Clojure.

mbrainz-sample's People

Contributors

benkamphaus avatar bobby avatar keithharper avatar redinger avatar robertarandolph avatar stuarthalloway avatar wellfilverd avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

skhurram sleyzerzon trevor thickey klgraham martinklepsch avodonosov cswaroop cascadian mveytsman limapay yilab davidykay zolok forks-badal jasoncourcoux mariusz-jachimowicz-83 abhi18av ksseono ilovejs fmnoise mpenet shikya smahood ticean nikaxi priximmo clojure-land qpdb cloojure err ianrogers vlifeng srehorn jeffp42ker keithharper diego3 cpackard pemtajo

mbrainz-sample's Issues

AbstractRelease seems to be missing entirely

I've tried both the latest backup and the one from 2014-11-17.
There is not a single entity referenced from :release/abstractRelease

[:find ?artist :in $ :where [_ :abstractRelease/artists ?artist] ] returns empty set

[:find ?name :in $ :where [_ :abstractRelease/name ?name] ] too

Should it be deleted from the schema then?

Import fails on OS X with Datomic free 0.9.5173

Importing the mbrainz-file fails with following stacktrace:

Caused by: java.io.FileNotFoundException: Could not locate datomic/process_monitor__init.class or datomic/process_monitor.clj on classpath:
    at clojure.lang.RT.load(RT.java:443)
    at clojure.lang.RT.load(RT.java:411)
    at clojure.core$load$fn__5066.invoke(core.clj:5641)
    at clojure.core$load.doInvoke(core.clj:5640)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5446)
    at clojure.core$load_lib$fn__5015.invoke(core.clj:5486)
    at clojure.core$load_lib.doInvoke(core.clj:5485)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:626)
    at clojure.core$load_libs.doInvoke(core.clj:5524)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:626)
    at clojure.core$require.doInvoke(core.clj:5607)
    at clojure.lang.RestFn.invoke(RestFn.java:551)
    at datomic.backup_cli$loading__4958__auto__.invoke(backup_cli.clj:1)
    at datomic.backup_cli__init.load(Unknown Source)
    at datomic.backup_cli__init.<clinit>(Unknown Source)
    ... 34 more

It works with datomic free 0.9.5153.

"Queries" wiki page issues

  1. In section Running the Queries, there's a typo (probably from previous version of page): '(e.g. the string "Lupe Fiasco" in the example above).'
    It should be: '(e.g. the string "Janis Joplin" in the example above).'

  2. In section Fulltext the text description 'Let's find all of the tracks with the word "nevermore" in the title.' says "nevermore" but "always" is used later in text and in the query ('Query args' subsection).

  3. In section Graph walk the text description 'Let's find Paul McCartney's collaboration network to a depth of 2.' says "Paul McCartney" but later in text and in the query "George Harrison" is used.

Unable to restore-db

When trying to restore-db on OS X 10.11.6 with datomic-free-0.9.5390

bin/datomic restore-db file:///Users/piotrkosinski/Projects/mbrainz-1968-1973 datomic:free://localhost:4334/mbrainz-1968-1973

got this error

java.lang.NumberFormatException: For input string: ".DS_Store"
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    at java.lang.Long.parseLong(Long.java:578)
    at java.lang.Long.parseLong(Long.java:631)
    at datomic.backup$describe_backups$fn__9012.invoke(backup.clj:531)
    at clojure.core$mapv$fn__6953.invoke(core.clj:6627)
    at clojure.lang.PersistentVector.reduce(PersistentVector.java:341)
    at clojure.core$reduce.invokeStatic(core.clj:6544)
    at clojure.core$mapv.invokeStatic(core.clj:6618)
    at clojure.core$mapv.invoke(core.clj:6618)
    at datomic.backup$describe_backups.invokeStatic(backup.clj:531)
    at datomic.backup$describe_backups.invoke(backup.clj:522)
    at datomic.backup$latest_t.invokeStatic(backup.clj:627)
    at datomic.backup$latest_t.invoke(backup.clj:625)
    at datomic.backup$require_latest_t.invokeStatic(backup.clj:632)
    at datomic.backup$require_latest_t.invoke(backup.clj:630)
    at datomic.backup_cli$restore.invokeStatic(backup_cli.clj:51)
    at datomic.backup_cli$restore.invoke(backup_cli.clj:43)
    at clojure.lang.Var.invoke(Var.java:379)
    at clojure.lang.AFn.applyToHelper(AFn.java:154)
    at clojure.lang.Var.applyTo(Var.java:700)
    at clojure.core$apply.invokeStatic(core.clj:646)
    at clojure.core$apply.invoke(core.clj:641)
    at datomic.require$require_and_run.invokeStatic(require.clj:22)
    at datomic.require$require_and_run.doInvoke(require.clj:17)
    at clojure.lang.RestFn.invoke(RestFn.java:423)
    at datomic$_main$fn__28.invoke(datomic.clj:150)
    at datomic$_main.invokeStatic(datomic.clj:149)
    at datomic$_main.doInvoke(datomic.clj:142)
    at clojure.lang.RestFn.invoke(RestFn.java:436)
    at clojure.lang.Var.invoke(Var.java:388)
    at clojure.lang.AFn.applyToHelper(AFn.java:160)
    at clojure.lang.Var.applyTo(Var.java:700)
    at clojure.core$apply.invokeStatic(core.clj:646)
    at clojure.main$main_opt.invokeStatic(main.clj:314)
    at clojure.main$main_opt.invoke(main.clj:310)
    at clojure.main$main.invokeStatic(main.clj:421)
    at clojure.main$main.doInvoke(main.clj:384)
    at clojure.lang.RestFn.invoke(RestFn.java:551)
    at clojure.lang.Var.invoke(Var.java:419)
    at clojure.lang.AFn.applyToHelper(AFn.java:186)
    at clojure.lang.Var.applyTo(Var.java:700)
    at clojure.main.main(main.java:37)

Removing .DS_Store file from Projects/mbrainz-1968-1973 folder doesn't fix the problem.

Can't restore db

For some reasons i (as a datomic noob) can't restore tutorial db

~/dist/datomic-pro-0.9.5130/bin/datomic restore-db file:///home/leuser/work/clojure/tutorials/dod/mbrainz-1968-1973/ datomic:dev://localhost:4334/mbrainz-1968-1973
Copied 0 segments, skipped 0 segments.
Copied 631 segments, skipped 0 segments.
Copied 935 segments, skipped 0 segments.
java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: clojure.lang.ExceptionInfo: :restore/read-failed Unable to read 546abffa-c4cf-461e-aea5-2f0ec83c7de3_00000000000000A {:db/error :restore/read-failed, :key "546abffa-c4cf-461e-aea5-2f0ec83c7de3_00000000000000A"}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.