datomic / codeq Goto Github PK

Imported Clojure source and got:

analyzing file: 17592186082191  - sha:  5d237149f637806c43162a8cadc5de533636be59
clojure.lang.LispReader$ReaderException: java.lang.NumberFormatException: Invalid number: 2010-11-12T13:14:15.666

Imported Clojure source and got lots (408) of these:

analyzing file: 17592186071738  - sha:  cc1072d7293283ec6ebcb50db4c29af658229535
java.lang.RuntimeException: Unmatched delimiter: )
analyzing file: 17592186071758  - sha:  f7add067d181f17298537a4594efb38d87abb4e2
java.lang.RuntimeException: Unmatched delimiter: )
analyzing file: 17592186071960  - sha:  2fb14dc14b0dfb8aa1db9ba9adfb810ca13e9fd9
java.lang.RuntimeException: Unmatched delimiter: )

Imported Clojure source and got 77 of these:

analyzing file: 17592186070732  - sha:  b96cb60f0614446a4b94de2d718c4d0a9fa080d9
java.lang.IllegalArgumentException: Metadata must be Symbol,Keyword,String or Map

Imported Clojure source code and got several (about 10) of these:

analyzing file: 17592186080638  - sha:  917b8c5e4f4e8061057ca0dd3ce3e0d60b6ebfa1
java.lang.ClassNotFoundException: clojure.test_clojure.protocols.RecordToTestStatics1

Commits can still have linefeeds, if the message is also in a double quoted string. E.g. the rails repository has this commit (note the trailing ^M)

e04818d1b8de44d47003031791f142182320ed89 Revert "Make sure :environment task is executed before db:schema:load or db:structure:load^M"

The commit above throws off the import and causes

Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: 40

We need to either parse smarter than with line-seq, or pass a different --pretty flag when reading in commits.

P.S. Not all "Revert" commits have the trailing ^M -- there is no pattern that I have discovered.

Imported Clojure source and got two of these:

analyzing file: 17592186060332  - sha:  59049f7d7dc25422692f1fb9c9713a7611093cfd
java.lang.RuntimeException: Map literal must contain an even number of forms
analyzing file: 17592186060340  - sha:  85a2424170b9d24a10129a214c94963fbfadaea9
java.lang.RuntimeException: Map literal must contain an even number of forms

when running datomic-free-0.8.3551.zip I get an error about missing a jdbc driver class when trying to import code with codeq.

After adding (Class/forName "org.h2.Driver") to the beginning of codeq.core/main it runs fine.

This may be an issue with the datomic client and datomic free missing the loading of the h2 driver.

Just a heads up, I wanted to check out the project but following the current instructions doesn't seem to work.

I get a connection error when I try to import a repo. Datomic seems to be running correctly so I'm not sure the problem though I haven't dug in yet. Exception is below:

java -server -Xmx1g -jar ../codeq/target/codeq-0.1.0-SNAPSHOT-standalone.jar datomic:free://localhost:4334/free/git
Exception in thread "main" clojure.lang.ExceptionInfo: Error communicating with HOST localhost on PORT 4334 {:host "localhost", :alt-host nil, :port 4334, :username "x6r830jIopXHuvLeAmwSYRTPjFCtLaGOdLszxzX5t8U=", :password "csuC3mlEEs3ZjHW9+fK5MWJEfx1EeyJToeSd7Yk1nYM=", :timestamp 1364250608163, :version "0.8.3704", :encrypt-channel true}
at clojure.core$ex_info.invoke(core.clj:4327)
at datomic.connector$endpoint_error.invoke(connector.clj:43)
at datomic.connector$create_hornet_factory.invoke(connector.clj:133)
at datomic.connector$create_connector.invoke(connector.clj:281)
at datomic.peer$send_admin_request$fn__7657.invoke(peer.clj:448)
at datomic.peer$send_admin_request.invoke(peer.clj:442)
at datomic.peer$create_database.invoke(peer.clj:461)
at clojure.lang.Var.invoke(Var.java:415)
at datomic.Peer.createDatabase(Peer.java:76)
at datomic.api$create_database.invoke(api.clj:19)
at datomic.codeq.core$ensure_db.invoke(core.clj:446)
at datomic.codeq.core$main.doInvoke(core.clj:530)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:617)
at datomic.codeq.core$_main.doInvoke(core.clj:539)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at datomic.codeq.core.main(Unknown Source)
Caused by: HornetQException[errorCode=3 message=Timed out waiting to receive cluster topology. Group:null]
at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:804)
at datomic.hornet$create_session_factory.doInvoke(hornet.clj:218)
at clojure.lang.RestFn.applyTo(RestFn.java:139)
at clojure.core$apply.invoke(core.clj:619)
at datomic.connector$try_hornet_connect.invoke(connector.clj:87)
at datomic.connector$create_hornet_factory.invoke(connector.clj:126)
... 14 more

In core.clj in get-repo-uri (line 306) it is asserted that the origin remote url ends with ".git", but it's possible to have a remote url that doesn't end with .git:

/tmp ➤ git clone git://github.com/Datomic/codeq                                
Cloning into 'codeq'...
remote: Counting objects: 141, done.
remote: Compressing objects: 100% (40/40), done.
remote: Total 141 (delta 52), reused 140 (delta 51)
Receiving objects: 100% (141/141), 30.34 KiB, done.
Resolving deltas: 100% (52/52), done.
/tmp ➤ cd codeq                                                                
codeq ➤ git remote show -n origin                                    git:master
* remote origin
  Fetch URL: git://github.com/Datomic/codeq
  Push  URL: git://github.com/Datomic/codeq
  HEAD branch: (not queried)
  Remote branch: (status not queried)
    master
  Local branch configured for 'git pull':
    master merges with remote master
  Local ref configured for 'git push' (status not queried):
    (matching) pushes to (matching)

When running on such repository, it of-course fails:

java -server -Xmx1g -jar ~/usr/src/codeq/target/codeq-0.1.0-SNAPSHOT-standalone.jar datomic:free://localhost:4334/git
Exception in thread "main" java.lang.AssertionError: Assert failed: Can't find remote origin
(and (pos? (count name)) (.endsWith name ".git"))
    at datomic.codeq.core$get_repo_uri.invoke(core.clj:306)
    at datomic.codeq.core$main.doInvoke(core.clj:531)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:614)
    at datomic.codeq.core$_main.doInvoke(core.clj:539)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at datomic.codeq.core.main(Unknown Source)

(it actually hangs until killed with ctrl-c).

I tried importing a git repository with a single remote reference named github instead of origin, and got this error:

It worked fine after renaming the remote to origin.

Hi there!
I'm only trying the instructions README

These are the steps I've followed:

• download datomic-free-09.4699
• unzip and cd
• bin/transactor config/samples/free-transactor-template.properties

Launching with Java options -server -Xms1g -Xmx1g -XX:NewRatio=4 -XX:SurvivorRatio=8 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=60 -XX:+UseCMSInitiatingOccupancyOnly -XX:+CMSScavengeBeforeRemark Starting datomic:free://localhost:4334/<DB-NAME>, storing data in: data ... System started datomic:free://localhost:4334/<DB-NAME>, storing data in: data

• git clone https://github.com/Datomic/codeq.git
• cd codeq
• lein uberjar

Implicit AOT of :main will be removed in Leiningen 3.0.0. 
If you only need AOT for your uberjar, consider adding :aot :all into your
:uberjar profile instead.
Compiling datomic.codeq.core
Created /Users/tangrammer/git/codeq/target/codeq-0.1.0-SNAPSHOT.jar
Created /Users/tangrammer/git/codeq/target/codeq-0.1.0-SNAPSHOT-standalone.jar

• lein new dat00
• cd dat00
• java -server -Xmx1g -jar ../codeq/target/codeq-0.1.0-SNAPSHOT-standalone.jar datomic:free://localhost:4334/git

Exception in thread "main" java.lang.NullPointerException
    at java.util.UUID.fromString(UUID.java:192)
    at datomic.cluster$val_key__GT_uuid.invoke(cluster.clj:21)
    at datomic.log$find_log.invoke(log.clj:526)
    at datomic.peer$load_db_from_basis.invoke(peer.clj:95)
    at datomic.peer.Connection$fn__7316$load_db__7322.invoke(peer.clj:202)
    at datomic.peer.Connection$fn__7316.invoke(peer.clj:203)
    at datomic.peer.Connection.create_connection_state(peer.clj:187)
    at datomic.peer$create_connection$reconnect_fn__7361.invoke(peer.clj:388)
    at datomic.peer$create_connection.invoke(peer.clj:389)
    at datomic.peer$get_connection$fn__7411$fn__7413.invoke(peer.clj:539)
    at datomic.peer$get_connection$fn__7411.invoke(peer.clj:536)
    at datomic.peer$get_connection.invoke(peer.clj:533)
    at datomic.peer$connect_uri.invoke(peer.clj:597)
    at clojure.lang.Var.invoke(Var.java:415)
    at datomic.Peer.connect(Peer.java:74)
    at datomic.api$connect.invoke(api.clj:15)
    at datomic.codeq.core$ensure_db.invoke(core.clj:447)
    at datomic.codeq.core$main.doInvoke(core.clj:530)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:617)
    at datomic.codeq.core$_main.doInvoke(core.clj:539)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at datomic.codeq.core.main(Unknown Source)

Then i saw that in Release Notes this note
** Version 0.9.4532 upgrades the database log to format version 2. All peers and transactors in a system must move together to this version or later. **
bin/datomic revert-to-log-version-1 {your-database-uri}

Then I tried:

• bin/datomic revert-to-log-version-1 datomic:free://localhost:4334/git
  But....

clojure.lang.ExceptionInfo: :db.error/log-conversion This version of Datomic cannot convert log versions {:db/error :db.error/log-conversion}
    at datomic.error$raise.invoke(error.clj:46)
    at datomic.error$raise.invoke(error.clj:44)
    at datomic.log$convert_log_version.invoke(log.clj:239)
    at datomic.peer$revert_to_log_version_1.invoke(peer.clj:645)
    at clojure.lang.Var.invoke(Var.java:415)
    at clojure.lang.AFn.applyToHelper(AFn.java:161)
    at clojure.lang.Var.applyTo(Var.java:532)
    at clojure.core$apply.invoke(core.clj:617)
    at datomic.require$require_and_run.doInvoke(require.clj:22)
    at clojure.lang.RestFn.invoke(RestFn.java:423)
    at datomic$_main$fn__28.invoke(datomic.clj:139)
    at datomic$_main.doInvoke(datomic.clj:138)
    at clojure.lang.RestFn.invoke(RestFn.java:421)
    at clojure.lang.Var.invoke(Var.java:419)
    at clojure.lang.AFn.applyToHelper(AFn.java:163)
    at clojure.lang.Var.applyTo(Var.java:532)
    at clojure.core$apply.invoke(core.clj:617)
    at clojure.main$main_opt.invoke(main.clj:335)
    at clojure.main$main.doInvoke(main.clj:440)
    at clojure.lang.RestFn.invoke(RestFn.java:512)
    at clojure.lang.Var.invoke(Var.java:435)
    at clojure.lang.AFn.applyToHelper(AFn.java:185)
    at clojure.lang.Var.applyTo(Var.java:532)
    at clojure.main.main(main.java:37)

I'm working with:

• OSX 10.9.2
• java version "1.7.0_51" Java(TM) SE Runtime Environment (build 1.7.0_51-b13) Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
• Leiningen 2.3.4 on Java 1.7.0_51 Java HotSpot(TM) 64-Bit Server VM

Thanks in advance!
Juan

e.g. from clojure/test.generative:

java.lang.RuntimeException: Invalid token: ::ctest/testing-contexts

The title says it all, but it would be really nice to have. I'm new to Clojure and Datomic, but I'm very interested in this project and its potential for analyzing semi-structured legal texts. Building an analyzer that could recognize statute sections as "units":

47 USC § 303b - Consideration of children’s television service in broadcast license renewal

• (a) After the standards required by section 303a of this title are in effect, the Commission shall, in its review of any application for renewal of a commercial or noncommercial television broadcast license, consider the extent to which the licensee
  • (1) has complied with such standards; and
  • (2) has served the educational and informational needs of children through the licensee’s overall programming, including programming specifically designed to serve such needs.
• (b) In addition to consideration of the licensee’s programming as required under subsection (a) of this section, the Commission may consider—
  • (1) any special nonbroadcast efforts by the licensee which enhance the educational and informational value of such programming to children; and
  • (2) any special efforts by the licensee to produce or support programming broadcast by another station in the licensee’s marketplace which is specifically designed to serve the educational and informational needs of children.

A proper analyzer would recognize 47 USC § 303b as something like a method signature and intelligently track changes inside the "method definition". Even finer-grained units would be especially useful--imagine being able to query for changes down to the subsection such as 47 USC § 303b(b)(2).