In DataScript 0.9.0, as far as I can tell, the pull function may lock the program in an infinite loop if it is used on an entity whose component attribute refers to itself, in a circular reference.

A demonstration script is at
https://gist.github.com/cigitia/c1f1fc48ebc041c979c5.
Line 24 needs to be uncommented to freeze the program.

This especially may happen when creating components in the same transaction as their parents, due to current behavior in the generation of component entities’ IDs; see #59.

DataScript is amazing; thank you for creating it.

If I'm reading the code correctly, only type and entity id are being checked when testing entity equality.

The problem with this approach is you can have the same entity from different db's which has different attributes, yet those entities test as equal.

Per the Datomic documentation:

Two entities are equal if they have the same id and their databases have the same id

I think that's reasonable as long as the db check is fairly cheap. It keeps you from having to realize all attributes to do the check, yet still lets you know that the entities aren't guaranteed to have equal attributes.

Related to #59. In DataScript 0.10.0, creating component entities by nesting maps in a transaction can still cause circular references, under certain circumstances when the new parent entity has an explicit ID but its first component entity has an implicit ID.

A demonstration script with two ways to produce the circular-reference behavior is at https://gist.github.com/cigitia/106e95c3824783442c5b.

Thanks a ton again for DataScript.

Can't figure out how to "clean" database with listeners attached.
I don't want to go through retract/add on entities, just clean it, and make all listeners jingle.
That's seems not very idiomatic, but how would one practically reset database?
PS great lib, thanks, fills a huge gap for me.

Hi,

The squiid function can return squuids without the leading zeros in the text:

e.g.
54eb0139-14dd-4f4f-b3af-369f8a653
vs
54eb0139-14dd-4f4f-b3af-000369f8a653

This means that they are not = to each other after a round trip to the server and back.

I'm very interested in using DataScript but wondering what your thoughts are when you are populating it with data from a backend database such as Mongo where the attributes are named without a prefix (i.e name vs room/name), is there a performant or idiomatic way of handling this without having to a) changing the naming convention in your backend db or b) convert names as you sync data which can be a performance hit.

Would love to hear thoughts on this matter if anyone has tackled this.

In Clojure, single-segment namespace names such as datascript are discouraged versus multi-segment names such as datascript.core, both as a matter of style and as a matter of being on the JVM (Stack Overflow thread; Google Group thread).

In ClojureScript, single-segment namespaces are similarly discouraged as a matter of style, but they also have problems with Google Closure—the latest version of ClojureScript (0.0-2814) has a new warning against single-segment namespaces for that reason.

[David Nolen] Compiler support for single segment namespaces has always been spotty and there's no intention to improve it. There are fundamental semantic issues - for example if you have a DOM element with a CSS id and you haven't supplied a namespace but you goog.require one with that name it will return the DOM element. Good luck debugging that.

Single segment namespaces are nearly alway representative of a problem waiting to manifest. If you want to use them then you can suppress the warning in your build options.

(See also: conversation between David Nolen and Luke VanderHart.)

It would be a breaking change, but it might be best for the future before version 1.0 to change the datascript namespace to something like datascript.core, and switch the current datascript.core to something like datascript.impl or whatever, in order to avoid the new ClojureScript compiler warning, and especially in case a Clojure-for-JVM version ever is created (#54).

Thank you so much for DataScript. It is absolutely amazing.

  (d/q '[:find ?a
         :in $
         :where [?a :foo 1]]
       [[1 :foo 1]])
  ;; => #{[1]}
  (d/q '[:find ?a
         :in $x
         :where [?a :foo 1]]
       [[1 :foo 1]])
  ;; => #{}

datascript.parser/parse-src-var works as expected. I'm not sure where the problem is.

It may sound a little bit bold to ask, but: What if transact acted as a pure function?

I don't see much utility in TxReport and transact could simply return the :db-after and let the user care about swapping the content of an atom if he/she really cares. That would eliminate the incidental complexity of listening to atom changes and allow the creation of temporary db states that we may not want to save in the application state (thus triggering something like om render etc..)

This will turn datascript into a purely functional data structure that supports complex queries, and that I think should be the main focus of the library. State management is really easy to add on top of that if somebody wants.

I was playing with an experiment recently (angling for a more functional front-end framework that uses React for the view) and I seriously considered building a model that looked datomic-like. I was using Mori for the data (which had good and bad aspects to it). It's nice to see this built in ClojureScript. As a JS programmer, I'd love to see a Mori-like build designed to be used from plain old JS.

I open this issue since i can not find a documentation/test on how to accomplish this.

How can i filter an entity based on one attribute value?

For example, i would like to set a filter that will return entities where :name=="Ivan"

[{:db/id 1
 :name  "Petr"
 :email "[email protected]"
 :aka   ["I" "Great"]
 :password "<SECRET>"}
 {:db/id 2
  :name  "Ivan"
  :aka   ["Terrible" "IV"]
  :password "<PROTECTED>"}]

(defn only-evans [udb datom] .....)

(d/filter db only-evans) 

will only return
[[{:db/id 1
:name "Petr"
:email "[email protected]"
:aka ["I" "Great"]
:password ""}]

This is the missing built-in. On the surface could this just be mapped to identity? But it seems it should instead somehow be utilized for query optimization?

Would be helpful to sneak this into the tutorials as an persistence (even if it's just for the life-time of the browser) example:

(cljs.reader/register-tag-parser! "datascript/DB" d/db-from-reader)
(->> conn d/db str (aset js/sessionStorage "last-state"))
(->> "last-state" (aget js/sessionStorage) cljs.reader/read-string atom)

Something like this:

(defn- retract-refs-to-entity [db v]
  (let [schema (:schema db)]
    (sequence
      (comp
        (filter #(= :db.type/ref (get-in schema [(.-a %) :db/valueType])))
        (map (fn [d] [:db/retract (.-e d) (.-a d) v])))
      (d/-search db [nil nil v]))))

Could anyone recommend how I might approach evicting data, perhaps in an LRU style? I display a lot of time series data that I assume will eventually blow up if I store it all. Thanks for datascript, tonksy!

It looks like there may be a possible race condition in d.listen - or that I am misunderstanding how to use it (quite possible).

I have one function A which transacts data to a db using data from a websocket connection:

function listener(args, kwargs) {
console.log('** Listener called', kwargs[':tx-data']);
d.transact(conn, kwargs[':tx-data']);
db1 = d.db(conn);
}

I have a separate reporter function B which generates an event for react js to trigger a re-render.

d.listen(conn, function(report) {
console.log('**Reporter heard and declaring showtime here is my report', report);
document.dispatchEvent(new CustomEvent('SHOWTIME'));
});

Re-rendering triggers a query from React and isn't always working - it turns out the problem is the data which the reporter thinks should be in the DB (from the the report tx_data) isn't always - so as if the listener trigger isn't when the transaction actually finishes.

Confusing is that moving the 'showtime' event dispatch to the end of A works.
This would make sense if d.transact was blocking, but if that was the true then I wouldn't expect d.listen to run until the transaction was finished.
If d.transact is non-blocking then I would expect to see the same issue with the event dispatch in either place - and I don't.

Any thoughts / suggestions appreciated.

Hello! Thanks for a very good project. Got rid of lots of pain at client-side.

There is blogpost about new enchancements in queries, for example disunction (or) without using rules and negation, via the new not and not-join clauses.

I found it very useful and seems it's not too hard to achieve this in datascript.

What do you think about adding this functionality to datascript?

I apologise if it's just me, but by reading the docs it's not very clear how to reference other entities inside datascript.

[
 {:db/id -10
  :name "C1"
  :value "foo"} 
 {:db/id -11
  :name "C2"
  :value "bar"}      
 {:db/id -1
  :name  "E"
  :components [-10, -11]}
]

How do I reference the generated ids 10 and 11 inside the components vector?

Hi,

I'm struggling to use d.init_db and create a schema at the same time like so:

d.init_db([[1, "name", "Ivan"],[1, "age", 17],[2, "name", "Igor"],[2, "age", 35][1, "friend", 2]], [{"friend": {":db/valueType": ":db.type/ref"}}]);

Gives
Error: nth not supported on this type function Object() { [native code] }

What am I doing wrong?

Mike

I know this might sound a bit silly at first, because of datomic, but I think it would be quite useful if datascript were clojure compatible as well.

Datomic is simply too heavy for some applications, and it would be nice to have something as lightweight as datascript.

Here are a few things this would be nice for:

• one could build a more lightweight version of https://github.com/kovasb/session
• complex configuration files could be managed more easily.
• some complex data structures that are on the boundary between representable by maps and requiring a whole datomic database could be better accommodated
• an awesome document serialisation format, just spit and slurp the database instead of some huge xml/json/edn
• an even smoother intro to datomic

Any thoughts on this?

Example:

(let [db                (d/create-conn {})
      {:keys [tempids]} (d/transact! db [{:db/id -1
                                          :name "Mazinger"}])
      new-id            (get tempids -1)]
  (println :entity  (-> @db (d/entity new-id) d/touch))
  (println :by-id   (d/q `[:find ?e
                           :where [?e :db/id ~new-id]]   @db))
  (println :by-name (d/q `[:find ?e
                           :where [?e :name "Mazinger"]] @db)))

;; Output:
:entity {:name Mazinger, :db/id 1}
:by-id #{}  ;; <-- why?
:by-name #{[1]}

Entity is not returned when querying using :db/id. Is this a limitation by design or a bug?

datascript/entity cannot pull in related entities because DataScript does not support :db/valueType of :db.type/ref in the schema.

It looks like upserts are only resolved for tx-data in the map format. Here's an example that demonstrates the problem:

(let [db (d/db-with (d/empty-db {:name  { :db/unique :db.unique/identity }})
                    [{:db/id 1 :name "Ivan"}])]
  (d/with db [[:db/add -1 :age 35]
              [:db/add -1 :name "Ivan"]]))

=> throws {:error :transact/unique, :attribute :name, :datom #datascript/Datom [2 :name Ivan 536870914 true]}

Added a test on my fork, which also fails in CI.

Hi,

I'm very excited about Datascript, so much so that I'm postponing the developpement of an open source web app for which it would be particularly suited: I'm waiting for it to mature from "Alpha quality" (as stated in the Readme).

@tonsky: could you tell us if you will be able to dedicate time to Datascript in the next weeks or if other people have expressed an interest in helping you develop it?

I think Datascript could get a lot of traction as people get more and more familiar with the concepts and their usefulness. As far as I know, there's no equivalent in the browser and it looks nothing short of revolutionary as compared to most existing state management solutions for SPAs (in js and cljs frameworks).

Many thanks for the code and the tutorials!

Vianney
(English is not my first language)

Hello,

I have an error while querying a database with retracted data.

Here is the code I am executing with datascript 0.1.4:

(let [db (-> (ds/empty-db {:aka {:db/cardinality :db.cardinality/many}})
               (ds/with [[:db/add 1 :name "Ivan"]])
               (ds/with [[:db/add 1 :name "Petr"]])
               (ds/with [[:db/add 1 :aka "Devil"]])
               (ds/with [[:db/add 1 :aka "Tupen"]]))]
    (ds/entity db 1))

=> {:db/id 1, :aka ("Devil" "Tupen"), :name "Petr"}

(let [db (-> (ds/empty-db {:aka {:db/cardinality :db.cardinality/many}})
               (ds/with [[:db/add 1 :name "Ivan"]])
               (ds/with [[:db/add 1 :name "Petr"]])
               (ds/with [[:db/add 1 :aka "Devil"]])
               (ds/with [[:db/add 1 :aka "Tupen"]]))]
    (let [db (-> db
                 (ds/with [[:db/retract 1 :name "Petr"]])
                 (ds/with [[:db/retract 1 :aka "Devil"]]))]
      (ds/entity db 1)))

=> TypeError: Cannot read property 'v' of null

Hi,

Error in the Readme. Should be:

CatChat walkthrough is at : http://tonsky.me/blog/datascript-chat/

CatChat demo is at : http://tonsky.me/datascript-chat/

Many thanks for the code and the tutorials!

Vianney
(English is not my first language)

(def db (atom (d/empty-db)))

(def data
 [ {:city/name           "New York City"
    :city/location       "New York"
    :city/image          "/img/home/NewYorkCity.jpg" }
   {:city/name           "Rio de Janerio"
    :city/location       "Brazil"
    :city/image          "/img/home/RioDeJaneiro.jpg" }
   {:city/name           "Auckland"
    :city/location       "New Zealand"
    :city/image          "/img/home/Auckland.jpg" }
   {:city/name           "Berlin"
    :city/location       "Germany"
    :city/image          "/img/home/Berlin.jpg" }
   {:city/name           "Amsterdam"
    :city/location       "Netherlands"
    :city/image          "/img/home/Amsterdam.jpg" }
   {:city/name           "Paris"
    :city/location       "France"
    :city/image          "/img/home/Paris.jpg" }])

(swap! db d/db-with data)

 (d/q '[:find ?e :where [?e :city/name]] @db)
;; #{[2]}

however, when the namespace is removed, it returns the correct number.

A simple rolling cache for entities would be a significant performance enhancement without adding too much complexity.

This works:

(let [regex (js/RegExp. search-term "i")
      search-fn (comp boolean (partial re-find regex))]
  (d/q '[:find ?category :in $ ?search-fn :where
         [?link :link/category ?category]
         [?link :link/title ?title]
         [(?search-fn ?title)]]
       db search-fn))

But this doesn't:

(let [regex (js/RegExp. search-term "i")
      search-fn (comp boolean (partial re-find regex))]
  (d/q '[:find ?category :in $ % ?search-fn :where
         [?link :link/category ?category]
         (search ?search-fn ?link)]
       db
       '[[(search ?search-fn ?link)
          [?link :link/title ?title]
          [(?search-fn ?title)]]]
       search-fn))

The predicate isn't making it through the rule bindings, because I get this error when trying to apply the predicate function.

"Cannot read property 'cljs$lang$maxFixedArity' of null"

Playing around with datascript, I'm surprised by the entity equality rules. When I run this minimal test case:

(deftest test-equality
  (let [db (d/create-conn {})]
    (d/transact! db [[:db/add 1 :name "Petr"]
                     [:db/add 1 :age 44]])
    (let [before (d/touch (d/entity @db 1))]
      (d/transact! db [[:db/add 1 :height 172]])
      (let [after (d/touch (d/entity @db 1))]
        (is (not= before after))))))

...I would expect it to pass. An attribute (height) has been added to Petr, but before and after are both equal. Is that right? Or have I mis-understood entities?

(I'm hoping it's a bug, and can be fixed, because I'm trying to render an entity with Om, and even though the data has changed, a re-render isn't triggered because the entity appears to be equal to its previous state...)

I want to pull an entity out of a Datascript db and send it to the server for processing. Currently, (entity db id) returns a datatype that cannot be serialized over the wire because it contains an atom of datascript/DB. Is there an easy way of converting the result of (entity db id) into a tree of maps - reference attributes are converted into maps?

Component entities can be implicitly created by nesting maps in a transaction. However, in DataScript 0.9.0, as far as I can tell:

• If explicit :db/id vals are omitted from within the nested maps,
• And if other non-component entities are also created elsewhere in the same transaction,
• Then :db/id vals may be generated that unexpectedly refer to both the new component entities and the non-component entities.

This can create circular references: if a new entity is created simultaneously with its component entity, then they may end up sharing the same ID. (This also currently can thus result in an infinite loop when using the Pull API; see #58.)

Including an explicit temporary ID (e.g., -1) in the components' parent entity is enough to correct the problem and to cause the other components to receive disjunct IDs, but I'm assuming that implicit temporary IDs are intended to be supported, as per DataScript Chat’s code.

A demonstration script of this behavior is at https://gist.github.com/cigitia/feafd70768fcd551cd16.

DataScript is amazing; thank you for creating it.

printing the db object reveals a missing attribute on an entity, but all the attributes on the map appear when the order of the keys is changed.

i don't have time to provide test cases for these issues now, but wanted to make a note of them.

I have a special use case where I (probably) don't want to use the in-memory Datomic version [1], but I do want all the Datomic features available in an in-memory fashion, just like Datascript. So a few questions:

• Would you like to have a Clojure version too? Does it make sense to you?
• Do you think it would require a lot of changes to the current codebase?

[1] It has unnecessary dependencies (when running in-memory). And I'm not sure about the properties of the in-memory version for long running processes and I can not easily find out due to the unknowns about the internals of Datomic.

Hi,
Datomic let us annotate transactions with custom attributes. Of course, datascript does not keep the database history so the exact same feature does not make sense. However, I think a way to provide information about a transaction would be useful. The data associated with a transaction could be added to the transaction report without being stored in the database.
Any opinion on this?

At a glance, it looks like it may be possible to separate out the query.cljs portion of DataScript for use with other storage mechanisms, like IndexedDB, if one makes sure to have the proper EAVT, AVET, and AEVT indexes available. Does that seem reasonable? Or does your implementation depend on more than just having those indexes available?

I'm a noob to all of ClojureScript and Datalog, not to mention DataScript, so what I'm asking may not even make sense—feel free to tell me if so. Thanks!

The newly announced Datomic Pull API (see blog post and API docs), could streamline data retrieval in many cases where we'd currently use helper functions (such as the ones in datascript-chat). Support for the Pull API could be a great solution for DataScript.

I'm so excited that you added lookup refs and upsert support! Trying them out right now.

I noticed in the changelog you also added entid support - so does that mean I can now specify :db/ident for entities?

 (-> (d/entity db nil) :some/key)

this will return the value of :same/key if :some/key exists on any entity in the database.

I built Radiant, a Datascript sandbox, recently. In it, I have found an issue with Datalog under advanced compilation.

This page hosts the :optimizations :none version of Radiant:

http://www.stuttaford.me/radiant/

As you can see, the datalog query runs and produces a result table.

This page hosts the :optimizations :advanced version of Radiant:

http://www.stuttaford.me/radiant-advanced/

You can see that I have printed out the parsed query, the full database, and the raw result from the datalog query above the result table.

The results do not match that of the first link. Looking at the results, for some reason, all the e and v values are missing.

You can visit the "Datoms" section to see that the datoms function is working as expected.

How do I go about debugging this?

I have already ruled out the possibility of externs not being used. They are.

Thank you for your time :-)

When I pass in a rule with a function to match a regex, I can call the function multiple times with different inputs so long as the attribute exists in all datoms. I can call the function once even when the attribute does not exist in all datoms. I get a long obscure error if I call the function multiple times when the attribute does not exist in all datoms. My test attribute here is :photo/does_not_exist.

(def test-schema {})
(def test-conn (d/create-conn schema))
(defn matcher-func [input to-match]
(re-find (re-pattern (.toLowerCase (clj->js input))) (.toLowerCase (clj->js to-match))))

(let [tx (d/transact! test-conn [{:db/id 1 :photo/thumb "foo.jpg" :photo/name "AAA" :photo/caption "BBB"}
                               {:db/id 2 :photo/thumb "bar.jpg" :photo/name "DDD" :photo/caption "AAA"}])]

;calling the function once with missing attributes
(d/q '{:find [?p]
       :in [$ % ?match-func]
       :where
       [[?p :photo/thumb _]
        (findall ?match-func ?p "a")]}
     @test-conn
     '[[(findall ?match-func ?p ?input) [?p :photo/name ?pn][(?match-func ?input ?pn)]]
       [(findall ?match-func ?p ?input) [?p :photo/caption ?pn][(?match-func ?input ?pn)]]
       [(findall ?match-func ?p ?input) [?p :photo/does_not_exist ?pn][(?match-func ?input ?pn)]]]
     matcher-func)

{[2] [1]}

  ;calling the function twice without missing attributes
  (d/q '{:find [?p]
       :in [$ % ?match-func]
       :where
       [[?p :photo/thumb _]
        (findall ?match-func ?p "a")
        (findall ?match-func ?p "b")]}
     @test-conn
     '[[(findall ?match-func ?p ?input) [?p :photo/name ?pn][(?match-func ?input ?pn)]]
       [(findall ?match-func ?p ?input) [?p :photo/caption ?pn][(?match-func ?input ?pn)]]]
     matcher-func)

#{[1]}

;calling the function twice with missing attributes
(d/q '{:find [?p]
       :in [$ % ?match-func]
       :where
       [[?p :photo/thumb _]
        (findall ?match-func ?p "a")
        (findall ?match-func ?p "b")]}
     @test-conn
     '[[(findall ?match-func ?p ?input) [?p :photo/name ?pn][(?match-func ?input ?pn)]]
       [(findall ?match-func ?p ?input) [?p :photo/caption ?pn][(?match-func ?input ?pn)]]
       [(findall ?match-func ?p ?input) [?p :photo/does_not_exist ?pn][(?match-func ?input ?pn)]]]
     matcher-func))

"Error evaluating:" (d/q (quote {:find [?p], :in [$ % ?match-func], :where [[?p :photo/thumb ](findall ?match-func ?p) (findall ?match-func ?p "b")]}) (clojure.core/deref test-conn) (quote [[(findall ?match-func ?p ?input) [?p :photo/name ?pn] [(?match-func ?input ?pn)]] [(findall ?match-func ?p ?input) [?p :photo/caption ?pn] [(?match-func ?input ?pn)]] [(findall ?match-func ?p ?input) [?p :photo/does_not_exist ?pn] [(?match-func ?input ?pn)]]]) matcher-func) :as "datascript.q.call(null,new cljs.core.PersistentArrayMap(null, 3, [new cljs.core.Keyword(null,"find","find",496279456),new cljs.core.PersistentVector(null, 1, 5, cljs.core.PersistentVector.EMPTY_NODE, [new cljs.core.Symbol(null,"?p","?p",-10896580,null)], null),new cljs.core.Keyword(null,"in","in",-1531184865),new cljs.core.PersistentVector(null, 3, 5, cljs.core.PersistentVector.EMPTY_NODE, [new cljs.core.Symbol(null,"$","$",-1580747756,null),new cljs.core.Symbol(null,"%","%",-950237169,null),new cljs.core.Symbol(null,"?ma
tch-func","?match-func",2022652070,null)], null),new cljs.core.Keyword(null,"where","where",-2044795965),new cljs.core.PersistentVector(null, 3, 5, cljs.core.PersistentVector.EMPTY_NODE, [new cljs.core.PersistentVector(null, 3, 5, cljs.core.PersistentVector.EMPTY_NODE, [new cljs.core.Symbol(null,"?p","?p",-10896580,null),new cljs.core.Keyword("photo","thumb","photo/thumb",1624132066),new cljs.core.Symbol(null,"","_",-1201019570,null)], null),cljs.core.list(new cljs.core.Symbol(null,"findall","findall",1888933708,null),new cljs.core.Symbol(null,"?match-func","?match-func",2022652070,null),new cljs.core.Symbol(null,"?p","?p",-10896580,null),"a"),cljs.core.list(new cljs.core.Symbol(null,"findall","findall",1888933708,null),new cljs.core.Symbol(null,"?match-func","?match-func",2022652070,null),new cljs.core.Symbol(null,"?p","?p",-10896580,null),"b")], null)], null),cljs.core.deref.call(null,blakbox.api.test_conn),new cljs.core.PersistentVector(null, 3, 5, cljs.cor
e.PersistentVector.EMPTY_NODE, [new cljs.core.PersistentVector(null, 3, 5, cljs.core.PersistentVector.EMPTY_NODE, [cljs.core.list(new cljs.core.Symbol(null,"findall","findall",1888933708,null),new cljs.core.Symbol(null,"?match-func","?match-func",2022652070,null),new cljs.core.Symbol(null,"?p","?p",-10896580,null),new cljs.core.Symbol(null,"?input","?input",-405387186,null)),new cljs.core.PersistentVector(null, 3, 5, cljs.core.PersistentVector.EMPTY_NODE, [new cljs.core.Symbol(null,"?p","?p",-10896580,null),new cljs.core.Keyword("photo","name","photo/name",1599012447),new cljs.core.Symbol(null,"?pn","?pn",704318294,null)], null),new cljs.core.PersistentVector(null, 1, 5, cljs.core.PersistentVector.EMPTY_NODE, [cljs.core.list(new cljs.core.Symbol(null,"?match-func","?match-func",2022652070,null),new cljs.core.Symbol(null,"?input","?input",-405387186,null),new cljs.core.Symbol(null,"?pn","?pn",704318294,null))], null)], null),new cljs.core.PersistentVector(null, 3, 5, c
ljs.core.PersistentVector.EMPTY_NODE, [cljs.core.list(new cljs.core.Symbol(null,"findall","findall",1888933708,null),new cljs.core.Symbol(null,"?match-func","?match-func",2022652070,null),new cljs.core.Symbol(null,"?p","?p",-10896580,null),new cljs.core.Symbol(null,"?input","?input",-405387186,null)),new cljs.core.PersistentVector(null, 3, 5, cljs.core.PersistentVector.EMPTY_NODE, [new cljs.core.Symbol(null,"?p","?p",-10896580,null),new cljs.core.Keyword("photo","caption","photo/caption",-978411948),new cljs.core.Symbol(null,"?pn","?pn",704318294,null)], null),new cljs.core.PersistentVector(null, 1, 5, cljs.core.PersistentVector.EMPTY_NODE, [cljs.core.list(new cljs.core.Symbol(null,"?match-func","?match-func",2022652070,null),new cljs.core.Symbol(null,"?input","?input",-405387186,null),new cljs.core.Symbol(null,"?pn","?pn",704318294,null))], null)], null),new cljs.core.PersistentVector(null, 3, 5, cljs.core.PersistentVector.EMPTY_NODE, [cljs.core.list(new cljs.core.Sy
mbol(null,"findall","findall",1888933708,null),new cljs.core.Symbol(null,"?match-func","?match-func",2022652070,null),new cljs.core.Symbol(null,"?p","?p",-10896580,null),new cljs.core.Symbol(null,"?input","?input",-405387186,null)),new cljs.core.PersistentVector(null, 3, 5, cljs.core.PersistentVector.EMPTY_NODE, [new cljs.core.Symbol(null,"?p","?p",-10896580,null),new cljs.core.Keyword("photo","does_not_exist","photo/does_not_exist",-133167301),new cljs.core.Symbol(null,"?pn","?pn",704318294,null)], null),new cljs.core.PersistentVector(null, 1, 5, cljs.core.PersistentVector.EMPTY_NODE, [cljs.core.list(new cljs.core.Symbol(null,"?match-func","?match-func",2022652070,null),new cljs.core.Symbol(null,"?input","?input",-405387186,null),new cljs.core.Symbol(null,"?pn","?pn",704318294,null))], null)], null)], null),blakbox.api.matcher_func);\n"

<TypeError: Cannot read property '1' of null>

TypeError: Cannot read property '1' of null
at context_resolve_val (http://localhost:8080/js/out/datascript/query.js:1097:127)
at http://localhost:8080/js/out/datascript/query.js:1142:45
at filter_by_pred (http://localhost:8080/js/out/datascript/query.js:1144:3)
at _resolve_clause (http://localhost:8080/js/out/datascript/query.js:1514:40)
at cljs.core.seq_reduce.seq_reduce__3 (http://localhost:8080/js/out/cljs/core.js:6463:14)
at cljs.core.seq_reduce.seq_reduce (http://localhost:8080/js/out/cljs/core.js:6484:22)
at cljs.core.LazySeq.cljs$core$IReduce$_reduce$arity$3 (http://localhost:8080/js/out/cljs/core.js:9200:29)
at cljs.core._reduce._reduce__3 (http://localhost:8080/js/out/cljs/core.js:1805:13)
at cljs.core._reduce._reduce (http://localhost:8080/js/out/cljs/core.js:1827:19)
at cljs.core.reduce.reduce__3 (http://localhost:8080/js/out/cljs/core.js:6559:26)

If we have wildcard and something else in a pull spec, something else should be merged on top of wildcard pull:

(deftest test-pull-wildcard
  (is (= {:db/id 2 :name "David" :_child [{:db/id 1}]}
         (d/pull test-db '[* :_child] 2))))

FAIL in (datascript.test.pull-api/test-pull-wildcard) (:)
expected: (= {:db/id 2, :name "David", :_child [{:db/id 1}]} (d/pull test-db (quote [* :_child]) 2))
  actual: (not (= {:db/id 2, :name "David", :_child [{:db/id 1}]} {:db/id 2, :name "David"}))

I was experimenting with external ids by using the second approach suggested by @tonsky at #31 (comment) and found an undocumented max entity-id limit exists at 0x1FFFFFFF but surpassing it gets unnoticed. Check the following code and his output:

(doseq [base-id [800 0x1FFFFFFF (inc 0x1FFFFFFF) 0xFFFFFFFFFFFFFFFF]]
  (let [db                (data/create-conn {})
        {:keys [tempids]} (data/transact! db [{:db/id base-id
                                               :name "Foo"}
                                              {:db/id -1  
                                              :name "Bar"}
                                              {:db/id -2
                                               :name "Baz"}
                                              ])]
    (println "Using base-id = " base-id)
    (println "  =>  tempids = " tempids)
    (println "  =>  entity  = " (-> @db (data/entity base-id) data/touch))
    (println "  => :max-eid = " (:max-eid @db))
    (println)
    ))

Output:

 Using base-id =  800
   =>  tempids =  {-1 801, -2 802, :db/current-tx 536870913}
   =>  entity  =  {:name Foo, :db/id 800}
   => :max-eid =  802

 Using base-id =  536870911
   =>  tempids =  {-1 536870912, -2 536870912, :db/current-tx 536870913}
   =>  entity  =  {:name Foo, :db/id 536870911}
   => :max-eid =  536870911

 Using base-id =  536870912
   =>  tempids =  {-1 1, -2 2, :db/current-tx 536870913}
   =>  entity  =  {:name Foo, :db/id 536870912}
   => :max-eid =  2

 Using base-id =  18446744073709552000
   =>  tempids =  {-1 1, -2 2, :db/current-tx 536870913}
   =>  entity  =  {:name Foo, :db/id 18446744073709552000}
   => :max-eid =  2

Notes and questions:

1- When using base-id = 536870911 (0x1FFFFFFF) datascript can't assign new temporal ids for the next entities, they all get 536870912 (0x1FFFFFFF + 1). I think an exception must be raised.

2- When using base-id >= (0x1FFFFFFF + 1) the database :max-eid is not updated, but you can sucesfully retrieve the created entity with the big id. Is ok and safe to use big ids greater than 0x1FFFFFFF + 1?

I'm just getting into clojurescript (long time React enthusiast), and I'd love to see a simple example of using datascript with react - maybe a TodoMVC something? Or a comparison to the Flux architecture?
If I figure things out on my own, I'll be sure to post here with examples.

Another bit of the Datomic schema that would be useful to have.

This hangs:

(deftest test-recursion
  (let [empty (d/empty-db {:part { :db/valueType :db.type/ref }
                           :spec { :db/valueType :db.type/ref }})]
    (let [db (d/db-with empty [[:db/add 1 :part 2]
                               [:db/add 2 :part 3]
                               [:db/add 3 :part 1]
                               [:db/add 1 :spec 2]
                               [:db/add 2 :spec 1]])]
      (is (= (d/pull db '[:db/id {:part ...} {:spec ...}] 1)
             {:db/id 1,
              :spec {:db/id 2
                     :spec {:db/id 1, :spec {:db/id 2}, :part {:db/id 2}}
                     :part {:db/id 3, :part {:db/id 1, :spec {:db/id 2}, :part {:db/id 2}}}}
              :part {:db/id 2
                     :spec {:db/id 1, :spec {:db/id 2}, :part {:db/id 2}}
                     :part {:db/id 3, :part {:db/id 1, :spec {:db/id 2}, :part {:db/id 2}}}}}))))

I believe it’s related to the way how different recursion branches are handled. When inside single branch, they should share “seen” set, so sequence of unwraps like :part → :spec → :part → :spec will eventually end on a seen entity no matter what actual path to it was.

Also when writing this I noticed that recursion is handled a little bit different than in Datomic:

(let [db (d/db-with empty [[:db/add 1 :part 1]])]
  (is (= (d/pull db '[:db/id {:part ...}] 1)
         {:db/id 1, :part {:db/id 1, :part {:db/id 1}}})))

From what I can tell, in Datomic, initial entity from where recursion starts is not considered as seen, that’s because when we see :part {:db/id 1} first time we try to expand that. When we see it second time, it’s now seen, and second expansion does not happen.

@dthume hope you don’t mind me assigning you to this?

This query works:

(d/q '[:find ?e :in $ ?et :where [?e :entity/type ?et]] db [:db/ident :entity.type/survey])

This one returns an empty result:

(d/q '[:find ?e :where [?e :entity/type [:db/ident :entity.type/survey]]] db)

Can you publish datascript as a CommonJS module?
I would like to use it with browserify/webpack in plain JS.

I'm getting this error occasionally when calling transact!. The only pattern I could find possibly has something to do with the underscores in the following example...

This always throws an error:

(d/transact! conn [{:db/id -1 :test/foo "foo" :test/_bar {:baz "qux"}}])

But these always work fine:

(d/transact! conn [{:db/id -1 :test/foo "foo" :test/bar {:baz "qux"}}])
(d/transact! conn [{:db/id -1 :test/_bar {:baz "qux"}}])
(d/transact! conn [{:db/id -1 :test/foo "foo" :test/_bar 123}])

Here's the JavaScript stack trace:

compare (core.cljs:1521)
cmp (core.cljs:44)
cmp_datoms_eavt (core.cljs:48)
binary_search_l (btset.cljs:33)
_seek (btset.cljs:382)
_slice (btset.cljs:404)
datascript.btset.slice.slice__3 (btset.cljs:414)
datascript.btset.slice.slice (btset.cljs:412)
datascript.btset.slice.slice__2 (btset.cljs:411)
datascript.btset.slice.slice (btset.cljs:412)
datascript.core.DB.datascript$core$ISearch$_search$arity$2 (core.cljs:80)
_search (core.cljs:31)
transact_add (core.cljs:192)
transact_tx_data (core.cljs:249)
with$ (datascript.cljs:37)
(anonymous function) (datascript.cljs:45)
cljs.core.swap_BANG_.swap_BANG___2 (core.cljs:3348)
cljs.core.swap_BANG_.swap_BANG_ (core.cljs:3358)
_transact_BANG_ (datascript.cljs:44)
transact_BANG_ (datascript.cljs:51)

Any ideas?