datodev / dato Goto Github PK

Latest version of dato.lib.server expects :user/email to exist, it should be agnostic.

Right now it's in a (let [local-storage-name (str "dato_save_state_" slot-idx)] ...), which is bothersome if you have multiple Dato apps

@dwwoelfel Is there a way to tell that a new entity is being created (both server-side and client-side), and either 1. require a :dato/guid k/v pair, or 2. auto-insert one? Would help with the "pit of success" story.

In the dato server constructor, I think it would be nicer to just pass in the connection uri. Would make the purpose more obvious and make it easier to inspect (answers the question "which database is this dato server connecting to?").

Unless there's a use-case where the conn-fn would be something other than (d/connect "some-uri")?

Somehow I get entities missing :exhibit/title on the server, which causes strange behavior in my client. The server should have a layer to validate a tx and all the entities involved, something like this for each entity involved:

(user-validation-fn entity new-entity? full-tx-report)

Similar to Sul or Om/next.
Just a rough sketch of what it might look like:

(defn my-com [data owner opts]
 (reify
   dato/IParams
   (params [_]
     {:tasks {:qes-by :task/title}
      :me {:qe-by :user/me? true}
      ;; Bit tricky w/ dep graph, something like prismatic fnk might enable this
      :local-session {:qe-by :session/user-id my-id}
      :filtered-task-ids {:q '[:find ?eid :in $ ?title-filter :where [?eid :task/title (re-match ?title-filter)]
      :current-company {:entity company-eid}
      :cases {:entities [case-eids]})
   dato/IRender
   (render [_]
     (let [data (dato/com-data owner)]
       ....))

1. The inputs required to instantiate my-com would be {:company-eid ... :title-filter #"..." :case-eids [...] :my-id ...} (and the current db at render time),
2. the value of data in render would have the same structure as specified in IParams

Other completely dynamic queries could be done in the render method if necessary, but whereas with the above format we can take care of (however inefficiently) automatically updating the component, it would the be up to the developer to make sure that listeners were added/removed on IWillMount and IWillUnmount for said dynamic queries.

There are lots of ways to implement this more efficiently, but for the first pass, re-running any currently "live" query on a tx-report (against a user-security-filtered db), diffing the new result with the previous one, and sending the new datoms to the client should suffice.

Once we see what that looks like as a practical API and what requirements it has at runtime, we can start to prioritize a simpler set of "live" primitives.

For now we're able to define the behavior of transition in meta, attached to transition return value. I think that it should be matter of datastore definition (or other declaration), we need an ability to define it on top, not just per transaction, because in application each kind of data always follows particular rule (don't dee any exceptions).

I think we need to summarize types of data in application in a sense of their life cycle and ownership.
My short list:

• request (session, location)
• CRUD data (persist/sync)
• computed data (read-only)

After a persisted transaction, but before confirmation of the transaction is received, Dato should have an api:

(dato/pending? db eid attr-1 attr-2 attr-3...) -> Are any of these attrs for this eid pending?
(dato/pending? db eid) -> Are any datoms about this entity pending?

It'd be easy to then show a spinner in the UI if we want to indicate data in-flight that's been optimistically inserted.

There isn't a separation between a connection and a DB in DataScript as there is in Datomic (will there be, with the webpeer?), but it might be nice to present the same interface on both platforms. Right now the DS DB inside (:db dato) is the atom, and can be deref'd for reading, or transacted into. I like the idea of having the atom in (:conn dato) and a separate (dato/db (:conn dato)) function for parity.

We should possibly encourage components to be annotated with a list of possible events they can cast whenever possible (dynamic events may not be possible to list, but are pretty rare, I haven't come across the need for one yet).

This + an event registry gives us two nice tooling features out of the box:

1. Tooling can look up the signature of an event (in the registry) and use that for e.g. test.check generating an event from a component
2. Upon changing a component/event signature, it should be easy to query (at run time) for which dependencies are now dirty and need to be fixed. Particularly handy for upgrading, as we iterate through design decisions.

Not sure if this is required for a public release, but if we were to build an email client with this architecture (that would be sublime...), there's too much data to fit into memory client-side. Need some intelligent datom streaming/caching/eviction strategies.

Hi @sgrove,

I'm excited about playing around with dato — seems like the future. I've got it running, but had a few problems along the way. After I cloned, I tried to run figwheel and got:

dato/ on master 
› lein figwheel
Possibly confusing dependencies found:
[lein-cljsbuild "1.0.6" :exclusions [org.clojure/clojurescript]] -> [lein-cljsbuild/cljs-compat "1.0.0-SNAPSHOT"] -> [org.clojure/clojure "1.5.1"]
 overrides
[lein-ancient "0.6.7"] -> [version-clj "0.1.2"] -> [org.clojure/clojure "1.6.0"]
 and
[lein-ancient "0.6.7"] -> [ancient-clj "0.3.9" :exclusions [com.amazonaws/aws-java-sdk-s3]] -> [org.clojure/clojure "1.6.0" :exclusions [joda-time org.clojure/clojure]]
 and
[lein-ancient "0.6.7"] -> [rewrite-clj "0.4.12"] -> [org.clojure/clojure "1.6.0" :exclusions [org.clojure/clojure]]

Consider using these exclusions:
[lein-ancient "0.6.7" :exclusions [org.clojure/clojure]]
[lein-ancient "0.6.7" :exclusions [org.clojure/clojure]]
[lein-ancient "0.6.7" :exclusions [org.clojure/clojure]]

Aborting due to :pedantic? :abort

I made the suggested changes, and the dependencies resolve fine. But now:

› rlwrap lein figwheel     
Figwheel: Starting server at http://localhost:3449
Focusing on build ids: dev
Compiling "resources/public/js/bin-debug/main.js" from ["src/cljs" "src/shared/" "yaks/datascript/src" "yaks/datascript/bench/src"]...
Compiling "resources/public/js/bin-debug/main.js" failed.
clojure.lang.ExceptionInfo: failed compiling file:src/cljs/dato/lib/db.cljs
 at clojure.core$ex_info.invoke (core.clj:4593)
Caused by: clojure.lang.ExceptionInfo: No such namespace: datascript, could not locate datascript.cljs, datascript.cljc, or Closure namespace "datascript" at line 1 src/cljs/dato/li
b/db.cljs
 at clojure.core$ex_info.invoke (core.clj:4593)

WARNING: update already refers to: #'clojure.core/update in namespace: plumbing.core, being replaced by: #'plumbing.core/update
WARNING: update already refers to: cljs.core/update being replaced by: plumbing.core/update at line 53 file:/Users/aroche/.m2/repository/prismatic/plumbing/0.4.0/plumbing-0.4.0.jar!
/plumbing/core.cljs
clojure.lang.ExceptionInfo: No such namespace: datascript, could not locate datascript.cljs, datascript.cljc, or Closure namespace "datascript" at line 1 /Users/aroche/Code/clj/dato
/src/cljs/dato/lib/db.cljs {:file "/Users/aroche/Code/clj/dato/src/cljs/dato/lib/db.cljs", :line 1, :column 1, :tag :cljs/analysis-error}
        at clojure.core$ex_info.invoke(core.clj:4593)
        at cljs.analyzer$error.invoke(analyzer.cljc:384)
        at cljs.analyzer$error.invoke(analyzer.cljc:382)
        at cljs.analyzer$analyze_deps.invoke(analyzer.cljc:1293)
        at cljs.analyzer$eval1679$fn__1681.invoke(analyzer.cljc:1545)
        at clojure.lang.MultiFn.invoke(MultiFn.java:251)
        at cljs.analyzer$analyze_seq.invoke(analyzer.cljc:1900)
        at cljs.analyzer$analyze$fn__1929.invoke(analyzer.cljc:1992)
        at cljs.analyzer$analyze.invoke(analyzer.cljc:1985)
        at cljs.analyzer$analyze_file$fn__1980.invoke(analyzer.cljc:2232)
        at cljs.analyzer$analyze_file.invoke(analyzer.cljc:2227)
        at cljs.analyzer$analyze_deps.invoke(analyzer.cljc:1291)
        at cljs.analyzer$eval1679$fn__1681.invoke(analyzer.cljc:1545)
        at clojure.lang.MultiFn.invoke(MultiFn.java:251)
        at cljs.analyzer$analyze_seq.invoke(analyzer.cljc:1900)
        at cljs.analyzer$analyze$fn__1929.invoke(analyzer.cljc:1992)
        at cljs.analyzer$analyze.invoke(analyzer.cljc:1985)
        at cljs.analyzer$analyze_file$fn__1980.invoke(analyzer.cljc:2232)
        at cljs.analyzer$analyze_file.invoke(analyzer.cljc:2227)
        at cljs.analyzer$analyze_deps.invoke(analyzer.cljc:1291)
        at cljs.analyzer$eval1679$fn__1681.invoke(analyzer.cljc:1545)
        at clojure.lang.MultiFn.invoke(MultiFn.java:251)
        at cljs.analyzer$analyze_seq.invoke(analyzer.cljc:1900)
        at cljs.analyzer$analyze$fn__1929.invoke(analyzer.cljc:1992)
        at cljs.analyzer$analyze.invoke(analyzer.cljc:1985)
        at cljs.analyzer$analyze_file$fn__1980.invoke(analyzer.cljc:2232)
        at cljs.analyzer$analyze_file.invoke(analyzer.cljc:2227)
        at figwheel_sidecar.repl$analyze_build.invoke(repl.clj:252)
        at figwheel_sidecar.repl$analyze_builds.invoke(repl.clj:257)
        at figwheel_sidecar.repl$run_autobuilder_helper.invoke(repl.clj:305)
        at figwheel_sidecar.repl$start_autobuild.invoke(repl.clj:376)
        at figwheel_sidecar.repl$run_autobuilder.invoke(repl.clj:535)
        at user$eval22223.invoke(form-init5551763719784528989.clj:1)
        at clojure.lang.Compiler.eval(Compiler.java:6782)
        at clojure.lang.Compiler.eval(Compiler.java:6772)
        at clojure.lang.Compiler.load(Compiler.java:7227)
        at clojure.lang.Compiler.loadFile(Compiler.java:7165)
        at clojure.main$load_script.invoke(main.clj:275)
        at clojure.main$init_opt.invoke(main.clj:280)
        at clojure.main$initialize.invoke(main.clj:308)
        at clojure.main$null_opt.invoke(main.clj:343)
        at clojure.main$main.doInvoke(main.clj:421)
        at clojure.lang.RestFn.invoke(RestFn.java:421)
        at clojure.lang.Var.invoke(Var.java:383)
        at clojure.lang.AFn.applyToHelper(AFn.java:156)
        at clojure.lang.Var.applyTo(Var.java:700)
        at clojure.main.main(main.java:37)
Subprocess failed

Running git submodule init && git submodule update solved this.

Also, I only needed the following env vars in my profiles.clj to get lein run working:

           :env          {:is-dev true
                          :datomic-local-uri "datomic:dev://localhost:4334/dato"}

Found out through experimentation that the database name doesn't matter, and doesn't need to exist before you do lein run. And that PORT is the web server port, not DATO_PORT =)

I'm still having a bit of trouble with the nrepl server, but I'll mess around with that when I actually need it.

Might be worth adding a few notes to the README for the next guy who comes along?

Cheers.

For e.g. a twitter client, in the client we want to indicate that we want the first 20 tweets. It's a concern that Datomic peers don't have (they just lazily stream data in and process as needed, but that's likely not a great idea to do purely client-side).

Dato is fine for now, but need a permanent name for the public release.

With React's Pit of Success in mind, we should list all the possible areas that need security considerations, and then think about convenient, secure-by-default ways of configuring access.