Giter Club home page Giter Club logo

potemkin's Introduction

Clojars Project cljdoc badge CircleCI

Potemkin is a collection of facades and workarounds for things that are more difficult than they should be. All functions are within the potemkin namespace.

Usage

Leiningen
[potemkin "0.4.6"]
deps.edn
potemkin/potemkin {:mvn/version "0.4.6"}

import-vars

Clojure namespaces conflate the layout of your code and your API. For larger libraries, this generally means that you either have large namespaces (e.g. clojure.core) or a large number of namespaces that have to be used in concert to accomplish non-trivial tasks (e.g. Ring).

The former approach places an onus on the creator of the library; the various orthogonal pieces of his library all coexist, which can make it difficult to keep everything straight. The latter approach places an onus on the consumers of the library, forcing them to remember exactly what functionality resides where before they can actually use it.

import-vars allows functions, macros, and values to be defined in one namespace, and exposed in another. This means that the structure of your code and the structure of your API can be decoupled.

(import-vars
  [clojure.walk
    prewalk
    postwalk]
  [clojure.data
    diff])

def-map-type

A Clojure map implements the following interfaces: clojure.lang.IPersistentCollection, clojure.lang.IPersistentMap, clojure.lang.Counted, clojure.lang.Seqable, clojure.lang.ILookup, clojure.lang.Associative, clojure.lang.IObj, java.lang.Object, java.util.Map, java.util.concurrent.Callable, java.lang.Runnable, and clojure.lang.IFn. Between them, there's a few dozen functions, many with overlapping functionality, all of which need to be correctly implemented.

Despite this, there are only six functions which really matter: get, assoc, dissoc, keys, meta, and with-meta. def-map-type is a variant of deftype which, if those six functions are implemented, will look and act like a Clojure map.

For instance, here's a map which will automatically realize any delays, allowing for lazy evaluation semantics:

(def-map-type LazyMap [m mta]
  (get [_ k default-value]
    (if (contains? m k)
      (let [v (get m k)]
        (if (instance? clojure.lang.Delay v)
          @v
          v))
      default-value))
  (assoc [_ k v]
    (LazyMap. (assoc m k v) mta))
  (dissoc [_ k]
     (LazyMap. (dissoc m k) mta))
  (keys [_]
    (keys m))
  (meta [_]
    mta)
  (with-meta [_ mta]
    (LazyMap. m mta)))

def-derived-map

Often a map is just a view onto another object, especially when dealing with Java APIs. While we can create a function which converts it into an entirely separate object, for both performance and memory reasons it can be useful to create a map which simply acts as a delegate to the underlying objects:

(def-derived-map StringProperties [^String s]
  :base s
  :lower-case (.toLowerCase s)
  :upper-case (.toUpperCase s))

Each time the key :lower-case is looked up, it will invoke `.toLowerCase. The resulting datatype behaves exactly like a normal Clojure map; new keys can be added and derived keys can be removed.

def-abstract-type and deftype+

The reason it's so laborious to define a map-like data structure is because the implementation cannot be shared between different types. For instance, clojure.lang.ISeq has both next and more methods. However, while more can be implemented in terms of next, as it is in clojure.lang.ASeq, within Clojure it must be reimplemented anew for each new type.

However, using def-abstract-type, we can avoid this:

(def-abstract-type ASeq
  (more [this]
    (let [n (next this)]
      (if (empty? n)
        '()
        n)))))

This abstract type may be used within the body of deftype+, which is just like a vanilla deftype except for the support for abstract types.

(deftype+ CustomSeq [s]
  ASeq
  clojure.lang.ISeq
  (seq [_] s)
  (cons [_ x] (CustomSeq. (cons x s)))
  (next [_] (CustomSeq. (next s))))

defprotocol+

A drop in replacement for defprotocol that is more REPL-friendly.

A protocol created with Clojure's defprotocol always creates new instance at load time. If a protocol is reloaded, a defrecord in another namespace that is referencing the procotol will not automatically be updated to the new protocol instance.

One telltale symptom of this disconnect can be a No implementation of method exception when calling record methods.

Potemkin's defprotocol+ improves the REPL experience by only creating a new instance of a protocol if the procotol body has changed.

definterface+

Every method on a type must be defined within a protocol or an interface. The standard practice is to use defprotocol, but this imposes a certain overhead in both time and memory. Furthermore, protocols don't support primitive arguments. If you need the extensibility of protocols, then there isn't another option, but often interfaces suffice.

While definterface uses an entirely different convention than defprotocol, definterface+ uses the same convention, and automatically defines inline-able functions which call into the interface. Thus, any protocol which doesn't require the extensibility can be trivially turned into an interface, with all the inherent savings.

unify-gensyms

Gensyms enforce hygiene within macros, but when quote syntax is nested, they can become a pain. This, for instance, doesn't work:

`(let [x# 1]
   ~@(map
       (fn [n] `(+ x# ~n))
       (range 3)))

Because x# is going to expand to a different gensym in the two different contexts. One way to work around this is to explicitly create a gensym ourselves:

(let [x-sym (gensym "x")]
  `(let [~x-sym 1]
     ~@(map
         (fn [n] `(+ ~x-sym ~n))
         (range 3))))

However, this is pretty tedious, since we may need to define quite a few of these explicit gensym names. Using unify-gensyms, however, we can rely on the convention that any var with two hashes at the end should be unified:

(unify-gensyms
  `(let [x## 1]
     ~@(map
         (fn [n] `(+ x## ~n))
         (range 3)))

License

Copyright © 2013 Zachary Tellman

Distributed under the MIT License. This means that pieces of this library may be copied into other libraries if they don't wish to have this as an explicit dependency, as long as it is credited within the code.

potemkin's People

Contributors

alexander-yakushev avatar andrewboltachev avatar bcobb avatar bronsa avatar dakrone avatar dm3 avatar jcrossley3 avatar joegallo avatar kachayev avatar kingmob avatar lread avatar marick avatar markaddleman avatar ninjudd avatar savagematt avatar slipset avatar w01fe avatar williamparker avatar xsc avatar ztellman avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

potemkin's Issues

Query for enhancement: CLJS support for import-vars

I wanted to use an equivalent of import-vars in ClojureScript (actually, in cljx code for both). I tried Potemkin on the off chance that the pre-compile macro expansion of ClojureScript would work out ok. It didn't.

I have written a couple of very basic macros which match the signature of import vars and work for vars and functions in ClojureScript .

Would you like to include something like this into Potemkin? If yes, I thought namespace potemkin.cljs.namespaces (.clj). I can post the code for your review (it's short) or create a pull request.

Exception with definterface+ and defrecord+ in 0.3.4

Reduced from exception in our codebase for real usage when upgrading to 0.3.4-- compiling this ns:

(ns test
  (:require
   potemkin))

(potemkin/definterface+ I
  (zzz [this]))

(potemkin/defrecord+ R
  []
  I
  (zzz [this] nil))

results in an exception for me:

Wrong number of args (0) passed to: test$eval2466$fn

Any ideas? Thanks!

"Use symbol instead of var" on clj-3 branches break stuff

Hi,
The switch from (import-fn #'some-fun) to (import-fn some-fn) in the clj-3 branches of lamina, gloss and aleph doesn't seem to work -- and I think this might be because the import-fn macro in potemkin is missing some updates. (?)

Metadata handling by `postwalk` in some cases

In one of the projects I want to do a thing as complicated as transforming a structure based on it's value (with children already being processed) and it's metadata. And I want to return a brand new structure based on that (structure with children already processed and metadata).

This means that I should use postwalk, 'cause when I would return a brand new strucutre, I don't want a program to enter it anymore (and prewalk would enter).

My interest is the function f, which I expect to receive the value (with children already processed) and metadata. So I constructed a function which acts like identity (as I don't care about actual transformation yet) and check, what it's been called with:

(defn print-with-meta-and-return [value]
  (binding [*print-meta* true]
    (prn value)
    (newline))
  value)

And when I use it like that:

(potemkin.walk/postwalk
  print-with-meta-and-return
  '[^:foo (x)])

I've got such output (from prn and newline):

x

(x)

[^{:line 1, :column 54, :foo true} (x)]

But I've expected the 2nd element to receive metadata, i.e. to be:

^{:line 1, :column 54, :foo true} (x)

(This additional line/column meta-information might be another question, although I ignore it for a moment. This is prehaps just how Clojure reader works on lists)

I later realized that on vectors it behaves well:

user=> (potemkin.walk/postwalk print-with-meta-and-return '[^:foo [x]])
x

^{:foo true} [x]

[^{:foo true} [x]]

[[x]]

(the last line here is the result)


Long story short, if we look at the code, we see that in case of postwalk the f would be outer function (in walk).

And in case of list it receives the result of (apply list ...):
https://github.com/ztellman/potemkin/blob/f1a13fd07e294413610a09d4eade92a46102bfa7/src/potemkin/walk.clj#L9

but e.g. in case of a vector it would receive (into (empty form) ...), which rather preserves metadata:
https://github.com/ztellman/potemkin/blob/f1a13fd07e294413610a09d4eade92a46102bfa7/src/potemkin/walk.clj#L14


Going on, it turns out that results are similar for both potemkin.walk and clojure.walk:

tracer.core=> (binding[*print-meta* true]
         #_=>   (println "potemkin on list:")
         #_=>   (prn (potemkin.walk/postwalk print-with-meta-and-return '[^:foo (x)]))
         #_=>   (println "potemkin on vector:")
         #_=>   (prn (potemkin.walk/postwalk print-with-meta-and-return '[^:foo [x]]))
         #_=>   (println "clojure on list:")
         #_=>   (prn (clojure.walk/postwalk  print-with-meta-and-return '[^:foo (x)]))
         #_=>   (println "clojure on vector:")
         #_=>   (prn (clojure.walk/postwalk  print-with-meta-and-return '[^:foo [x]])))
potemkin on list:
x

(x)

[^{:line 3, :column 61, :foo true} (x)]

[^{:line 3, :column 61, :foo true} (x)]
potemkin on vector:
x

^{:foo true} [x]

[^{:foo true} [x]]

[^{:foo true} [x]]
clojure on list:
x

(x)

[(x)]

[(x)]
clojure on vector:
x

^{:foo true} [x]

[^{:foo true} [x]]

[^{:foo true} [x]]

I.e. when handling list, the 2nd node is metadata-less ((x)), but in case of vector the 2nd node has metadata (^{:foo true} [x]). And the reason is that (apply list ...) as opposed to (into (empty form) ...).


So, speaking of metadata preservation, the main question is — how the actual metadata perservation should happen?

  1. Is potemkin.walk intended to pass structure's metadata into f (as it does now for e.g. vector and other structures but not for list)?
  2. Is it intended rather to restore metadata after f has returned (as it does now)?

On that, consider such “wrapper” use case:

user=> (binding [*print-meta* true]
              (prn (clojure.walk/postwalk (fn [x] (if (vector? x) {:value x} x))
^{:a 111} [1 ^{:a 222} [2 3] 4])))

{:value ^{:a 111} [1 {:value ^{:a 222} [2 3]} 4]}
nil


user=> (binding [*print-meta* true]
              (prn (potemkin.walk/postwalk (fn [x] (if (vector? x) {:value x} x))
^{:a 111} [1 ^{:a 222} [2 3] 4])))

^{:a 111} {:value ^{:a 111} [1 ^{:a 222} {:value ^{:a 222} [2 3]} 4]}

i.e. in f here I'm destroying the actual structure, and potemkin.walk adds metadata once again.

Update 0.3.7 breaks Vertigo's `def-typed-struct`

The newer versions of Potemkin break something with Vertigo. On a project with the following dependencies:

[potemkin "0.3.8"] ;; also 0.3.7
[vertigo "0.1.3"]

the following code:

(ns potemkin-vertigo.core
  (:require [vertigo.core :as v]
            [vertigo.structs :as s :refer [def-typed-struct]]))

(def-typed-struct Foobar :type s/int32)

results in:

CompilerException java.lang.ClassFormatError: Duplicate interface name in class file compile__stub/potemkin_vertigo/core$eval9546$reify$reify__9549, compiling:(/private/var/folders/j2/zvt92c1s39d_0kdmhrhbgtkm0000gn/T/form-init6935275163352736010.clj:1:1) 

Question: why so many ancestors difference betwence Clj map and a PotemkinMap?

Thank you very much for this amazing library. I wish I were as deeply knowledgeable of Clojure and I had fathomed it as deep as you!

See here for the full differences: https://github.com/piotr-yuxuan/closeable-map/blob/39eb776d8452386e9068562bcac3e41084f86ad5/README.md#technicalities.

Notably, there are:

 ;; Ancestors of Clojure map only but not a custom map.
 #{clojure.lang.AFn ; Concrete type, but see below for IFn.
   clojure.lang.APersistentMap
   clojure.lang.IEditableCollection
   clojure.lang.IKVReduce
   clojure.lang.IMapIterable
   java.io.Serializable}

 ;; Ancestors of some custom map only.
 #{clojure.lang.IType
   java.util.Iterator
   potemkin.collections.PotemkinMap
   potemkin.types.PotemkinType}

Allow to import all vars from a namespace when no vars are specified

Hi Zach,

I was wondering if you could extend import-vars to allow something like that:

(import-vars [some-ns] [some-other-ns vars] ...)

where some-ns would have all it's vars aliased

I am using the following macro to work around this now:

(defmacro import-all-vars
  [namespace]
  `(potemkin/import-vars
    [~namespace
     ~@(map key (ns-publics (the-ns namespace)))]))

preserve file and line metadata?

It would be nice for Emacs users for import-fn to preserve file and line metadata, so that M-. jumps straight to the actual definition and avoids cluttering the edit-definition stack. I'm not sure that the metadata really should point to the actual definition over the import-fn call site, but I'm using the following patch for this convenience:

diff --git a/src/potemkin.clj b/src/potemkin.clj
index eb4e307..4ec5a72 100644
--- a/src/potemkin.clj
+++ b/src/potemkin.clj
@@ -18,4 +18,10 @@
         n (:name m)
         arglists (:arglists m)
         doc (:doc m)]
-    (list `def (with-meta n {:doc doc :arglists (list 'quote arglists)}) (eval sym))))
\ No newline at end of file
+    `(do
+       (def ~(with-meta n {:doc doc :arglists (list 'quote arglists)})
+            ~(eval sym))
+       (alter-meta! ~(list 'var n) assoc
+                    :file ~(:file m)
+                    :line ~(:line m))
+       ~(list 'var n))))

docs: add caveat about import-vars?

Some time ago I merged rewrite-cljs v0 and rewrite-clj v0 to create rewrite-clj v1.

Rewrite-clj v0 used an inlined version of potemkin import-vars.
My initial decision was to create an inlined cljs version of import-vars to match.
This was a ton of learning and effort, but it technically worked.

Over time I discovered import-vars does has enough downsides that I ultimately decided to switch to code generation instead. Here are my rewrite-clj v1 notes on import-vars.

So that users can make a more informed choice, I think we should probably mention some of the cons of using import-vars in the readme. I am happy to take a stab at this.

Reliable JVM crash using definterface+

I'm not sure if this is properly a Potemkin, Clojure, or JVM bug, but I thought I'd start by reporting it here. Please let me know if you think it belongs elsewhere. Anyway, I can reliably crash JVM 11.0.1 on Windows 10 with the attached clojure file (this works with Clojure 1.10 betas 2 and 3 as well as Clojure 1.9). At the repl, I require the temp.clj and run show-bug -- everything is fine. But then I require it again (using :reload true) and run show-bug and the JVM crashes producing the attached error report.
jvmcrash.zip

:forms meta data loses quoting

Hey Zach,

Thanks for a great package here. It's very handy

I've had a small issue with some macros that use :forms meta data (like let does)
to provide a usage summary in documentation automatically. This is a quoted data structure, but it loses its quotes under import-vars (i.e., import-macro).

For instance,

(ns A)
(defmacro a 
  "doc"
  {:forms '[(a [bbb] ccc)]}
  [b]
  `(list :do :something :with ~b))
(ns B
  (:require potemkin A))
(potemkin/import-vars (A a))

will raise an error like

CompilerException java.lang.RuntimeException: Unable to resolve symbol: bbb in this context, compiling....

This isn't specific to :forms of course; I just happen to have a use for that case.
The same issue arises with quoted forms in metadata, with the quoted form losing its quoting protection when being expanded by import-vars and then apparently being evaluated/compiled subsequently.

Thanks,

Chris

def-map-type's predefined count doesn't run in constant time

I think the culprit is here. (counted? (keys {:a 1})) returns false, so this is actually a lie :)

I think you cannot provide a default implementation and should instead force the users to define count themselves.

Right now defining count myself within def-map-type solves the performance issue.

def-map-type doesn't support metadata

It looks like because types from def-map-type don't derive from IObj, with-meta doesn't work. I get ClassCastException cannot be cast to clojure.lang.IObj clojure.core/with-meta (core.clj:214)

def-derived-map has a broken hashCode if any key is nil

(require '[potemkin :as p])
(p/def-derived-map Up [v] :value (some-> v (.toUpperCase)))
(.hashCode (->Up nil))

Expected a hash code returned, got a NullPointerException.

E.g. in a repl:

#error{:cause nil,
       :via [{:type java.lang.NullPointerException,
              :message nil,
              :at [user.Up$fn__17016 invoke "form-init4972760988041368194.clj" 121]}],
       :trace [[user.Up$fn__17016 invoke "form-init4972760988041368194.clj" 121]
               [clojure.core.protocols$fn__6755 invokeStatic "protocols.clj" 167]
               [clojure.core.protocols$fn__6755 invoke "protocols.clj" 124]
               [clojure.core.protocols$fn__6710$G__6705__6719 invoke "protocols.clj" 19]
               [clojure.core.protocols$seq_reduce invokeStatic "protocols.clj" 31]
               [clojure.core.protocols$fn__6736 invokeStatic "protocols.clj" 75]
               [clojure.core.protocols$fn__6736 invoke "protocols.clj" 75]
               [clojure.core.protocols$fn__6684$G__6679__6697 invoke "protocols.clj" 13]
               [clojure.core$reduce invokeStatic "core.clj" 6545]
               [clojure.core$reduce invoke "core.clj" 6527]
               [user.Up hashCode "form-init4972760988041368194.clj" 119]
               [sun.reflect.NativeMethodAccessorImpl invoke0 "NativeMethodAccessorImpl.java" -2]
               [sun.reflect.NativeMethodAccessorImpl invoke "NativeMethodAccessorImpl.java" 62]
               [sun.reflect.DelegatingMethodAccessorImpl invoke "DelegatingMethodAccessorImpl.java" 43]
               [java.lang.reflect.Method invoke "Method.java" 498]
               [clojure.lang.Reflector invokeMatchingMethod "Reflector.java" 93]
               [clojure.lang.Reflector invokeNoArgInstanceMember "Reflector.java" 313]
               [user$eval17059 invokeStatic "form-init4972760988041368194.clj" 1]
               [user$eval17059 invoke "form-init4972760988041368194.clj" 1]
               [clojure.lang.Compiler eval "Compiler.java" 6927]
               [clojure.lang.Compiler eval "Compiler.java" 6890]
               [clojure.core$eval invokeStatic "core.clj" 3105]
               [clojure.core$eval invoke "core.clj" 3101]
               [clojure.main$repl$read_eval_print__7408$fn__7411 invoke "main.clj" 240]
               [clojure.main$repl$read_eval_print__7408 invoke "main.clj" 240]
               [clojure.main$repl$fn__7417 invoke "main.clj" 258]
               [clojure.main$repl invokeStatic "main.clj" 258]
               [clojure.main$repl doInvoke "main.clj" 174]
               [clojure.lang.RestFn invoke "RestFn.java" 1523]
               [clojure.tools.nrepl.middleware.interruptible_eval$evaluate$fn__952 invoke "interruptible_eval.clj" 87]
               [clojure.lang.AFn applyToHelper "AFn.java" 152]
               [clojure.lang.AFn applyTo "AFn.java" 144]
               [clojure.core$apply invokeStatic "core.clj" 646]
               [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1881]
               [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1881]
               [clojure.lang.RestFn invoke "RestFn.java" 425]
               [clojure.tools.nrepl.middleware.interruptible_eval$evaluate invokeStatic "interruptible_eval.clj" 85]
               [clojure.tools.nrepl.middleware.interruptible_eval$evaluate invoke "interruptible_eval.clj" 55]
               [clojure.tools.nrepl.middleware.interruptible_eval$interruptible_eval$fn__997$fn__1000
                invoke
                "interruptible_eval.clj"
                222]
               [clojure.tools.nrepl.middleware.interruptible_eval$run_next$fn__992 invoke "interruptible_eval.clj" 190]
               [clojure.lang.AFn run "AFn.java" 22]
               [java.util.concurrent.ThreadPoolExecutor runWorker "ThreadPoolExecutor.java" 1142]
               [java.util.concurrent.ThreadPoolExecutor$Worker run "ThreadPoolExecutor.java" 617]
               [java.lang.Thread run "Thread.java" 745]]}

Top-level namespace breaks jdeps analysis

Hi,

I'm experimenting creating a JRE from a JDK as outlined here, in this blog post: https://blog.adoptium.net/2021/10/jlink-to-produce-own-runtime/. Unfortunately, I'm receiving this error when attempting to analyse the jdeps of my jar:

Caused by: java.lang.module.InvalidModuleDescriptorException: clj_tuple__init.class found in top-level directory (unnamed package not allowed in module)
	at java.base/jdk.internal.module.ModulePath.toPackageName(ModulePath.java:720)

This is because I'm using clj-http 3.12.3 which uses potemkin/potemkin 0.4.5 which has a dependency upon clj-tuple 0.2.2.

I'm wondering, is there a way to replace clj-tuple in potemkin - considering clj-tuple is archived by the author and perhaps there is no need for it (since in potemkin, it's only used in one place, for the t/vector here https://github.com/clj-commons/potemkin/blob/3e404364ae2fd32f7a53b362a79d2012ab958ab2/src/potemkin/utils.clj#L97. Perhaps the built-in vector form in Clojure is fine to use instead?

Add examples for optional names to README

PR #9 seems to have implemented this feature but there's no other mention of it or any examples.

In case anyone else looks for this using the same search keywords as myself, I tried finding this by searching for "alias" and "rename".

I'll submit a PR if you'd like.

It might also be able to supply optional names with import-vars too, e.g. something like this:

(import-vars
  [clojure.walk
    [prewalk walk-pre]
    [postwalk walk-post]]
  [clojure.data
    [diff data-diff]])

I'll make another issue for this change if you're interested in it (either yourself or by others).

import-vars and tools.namespace do not seem to be friends

I have a namespace in my project that imports some names (including 'middlewares) from compojure.api.sweet. c.a.s uses import-vars to export a bunch of names defined elsewhere in compojure-api. If I screw something up in my namespace, such that it doesn't compile anymore, and then run clojure.tools.namespace.repl/refresh, the error I get isn't the reason my namespace doesn't compile, but something about how 'middlewares already refers to compojure.api.meta/middlewares. Worse, if I fix the compile problem in my namespace, repeated calls to refresh return the same "middlewares already refers" error, and as far as I can tell, the only way to make that go away is to manually call ns-unmap on everything that my namespace imported from someone using import-vars.

I realize this is a convoluted error-description -- I can upload a small demonstration project if that will help.

Codox doesn't see protocol functions imported by Potemkin

I don't really know whether this is a codox issue or a potemkin issue. I'm raising it as an issue here, because my best guess is that the underlying issue is that something about the metadata attached to potemkin-imported protocol functions isn't quite right, so codox doesn't recognize it.

NullPointer on call to keys of empty def-map-type

(def-map-type
    TestMap [m]
    (get [this k default-value]
         (get m k default-value))

    (assoc [this key value]
        (throw (UnsupportedOperationException.)))

    (dissoc [this key]
            (throw (UnsupportedOperationException.)))

    (keys [this]
          (keys m)))

(keys (TestMap. {}))

Macros define var even if not evaluated

Midje wants to conditionally import certain vars when running in the repl. I discovered that the vars are created even when not running in the repl. Potemkin inherits this (mis)behavior from def:

user=> (if false (do (def derp3 3)))
nil
user=> ( (ns-publics *ns*) 'derp3)
#'user/derp3
user=> (deref #'derp3)
#object[clojure.lang.Var$Unbound 0x745c8706 "Unbound: #'user/derp3"]

Because of this, Potemkin's behavior is arguably correct.

Here's a simplified example of the Midje case:

user=> ( (ns-publics *ns*) 'load-config-files)
nil
user=> (if false (potemkin.namespaces/import-vars [midje.config load-config-files]))
nil
user=> ( (ns-publics *ns*) 'load-config-files)
#'user/load-config-files

;;; This is not a copy of the var. For example, the metadata is wrong:

user=> (meta ( (ns-publics *ns*) 'load-config-files))
{:ns #object[clojure.lang.Namespace 0x4d1f34a6 "user"], :name load-config-files}
user=> (meta #'midje.config/load-config-files)
{:arglists ([]), :line 131, :column 1, :file "midje/config.clj", :name load-config-files, :ns #object[clojure.lang.Namespace 0x58378d16 "midje.config"]}

;;; ... and there's no value.
user=> (deref ( (ns-publics *ns*) 'load-config-files))
#object[clojure.lang.Var$Unbound 0x94a34ff "Unbound: #'user/load-config-files"]

def-map-type broken in 0.3.12-SNAPSHOT

Trying to evaluate this:

(def-map-type MyMap [x]
  (get [_ key default] (get x key default))
  (assoc [_ key val] (assoc x key val))
  (dissoc [_ key] (dissoc x key))
  (keys [_] (keys x)))

Results in this:

1. Caused by java.lang.IllegalArgumentException
   No single method: with_meta_STAR_ of interface:
   potemkin.collections.PotemkinMap found for function: with-meta* of protocol:
   PotemkinMap

                 Compiler.java: 3566  clojure.lang.Compiler$InvokeExpr/<init>
                 Compiler.java: 3773  clojure.lang.Compiler$InvokeExpr/parse
                 Compiler.java: 6711  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6500  clojure.lang.Compiler/analyze
                 Compiler.java: 6461  clojure.lang.Compiler/analyze
                 Compiler.java: 5837  clojure.lang.Compiler$BodyExpr$Parser/parse
                 Compiler.java: 8073  clojure.lang.Compiler$NewInstanceMethod/parse
                 Compiler.java: 7609  clojure.lang.Compiler$NewInstanceExpr/build
                 Compiler.java: 7490  clojure.lang.Compiler$NewInstanceExpr$DeftypeParser/parse
                 Compiler.java: 6709  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6500  clojure.lang.Compiler/analyze
                 Compiler.java: 6461  clojure.lang.Compiler/analyze
                 Compiler.java: 5837  clojure.lang.Compiler$BodyExpr$Parser/parse
                 Compiler.java: 6155  clojure.lang.Compiler$LetExpr$Parser/parse
                 Compiler.java: 6709  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6500  clojure.lang.Compiler/analyze
                 Compiler.java: 6461  clojure.lang.Compiler/analyze
                 Compiler.java: 5837  clojure.lang.Compiler$BodyExpr$Parser/parse
                 Compiler.java: 5272  clojure.lang.Compiler$FnMethod/parse
                 Compiler.java: 3901  clojure.lang.Compiler$FnExpr/parse
                 Compiler.java: 6707  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6500  clojure.lang.Compiler/analyze
                 Compiler.java: 6765  clojure.lang.Compiler/eval
                 Compiler.java: 6757  clojure.lang.Compiler/eval
                 Compiler.java: 7195  clojure.lang.Compiler/load
                          REPL:    1  user/eval14701
                 Compiler.java: 6768  clojure.lang.Compiler/eval
                 Compiler.java: 6731  clojure.lang.Compiler/eval
                      core.clj: 3076  clojure.core/eval
                      main.clj:  239  clojure.main/repl/read-eval-print/fn
                      main.clj:  239  clojure.main/repl/read-eval-print
                      main.clj:  257  clojure.main/repl/fn
                      main.clj:  257  clojure.main/repl
                   RestFn.java: 1523  clojure.lang.RestFn/invoke
        interruptible_eval.clj:   67  clojure.tools.nrepl.middleware.interruptible-eval/evaluate/fn
                      AFn.java:  152  clojure.lang.AFn/applyToHelper
                      AFn.java:  144  clojure.lang.AFn/applyTo
                      core.clj:  626  clojure.core/apply
                      core.clj: 1864  clojure.core/with-bindings*
                   RestFn.java:  425  clojure.lang.RestFn/invoke
        interruptible_eval.clj:   51  clojure.tools.nrepl.middleware.interruptible-eval/evaluate
        interruptible_eval.clj:  183  clojure.tools.nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
        interruptible_eval.clj:  152  clojure.tools.nrepl.middleware.interruptible-eval/run-next/fn
                      AFn.java:   22  clojure.lang.AFn/run
       ThreadPoolExecutor.java: 1145  java.util.concurrent.ThreadPoolExecutor/runWorker
       ThreadPoolExecutor.java:  615  java.util.concurrent.ThreadPoolExecutor$Worker/run
                   Thread.java:  744  java.lang.Thread/run

Potential breakage

Setup

CLJ-1929: "Literal collections are tagged as IPersistentVector, IPersistentMap, etc and as constant expressions won’t preserve type hints. If these collections are then used in a Java interop call, there is no way to indicate the desired type to choose a method overload between Collection and some other type."

This ticket has a patch that is on the shortlist for inclusion to 1.12.

Issue

As I commented in the issue, the strict version of the patch breaks Potemkin's import-* macros. Given Potemkin's prominence within the community, it seems reasonable to me to proactively fix the issue.

(keys (LazyMap. {:a (delay "B")})) will eagerly instantiate MapEntrys

Not a bug so much as an observation. LazyMap (which is what I was using def-map-type for) isn't quite as lazy as it looks.

clojure.core/keys will eagerly create all MapEntrys. It may be worth pointing that out in the docs?

Is there any way to fix the problem? There's some magic in clojure.core's implementation of keys that I don't understand yet.

(def-map-type LazyMap [m]
              (get [_ k default-value]
                   (if (contains? m k)
                     (let [v (get m k)]
                       (if (instance? clojure.lang.Delay v)
                         @v
                         v))
                     default-value))
              (assoc [_ k v]
                (LazyMap. (assoc m k v)))
              (dissoc [_ k]
                      (LazyMap. (dissoc m k)))
              (keys [_]
                    (keys m)))

(let [was-called (atom false)
      d (delay (reset! was-called true))
      m (LazyMap. {:d d})]

; contains is ok
  (contains? m :d)
  (println @was-called) ; => false

; potemkin.collections/keys* is also well-behaved
  (doall (potemkin.collections/keys* m))
  (println @was-called)  ; => false

; .keySet as well
  (doall (.keySet m))
  (println @was-called) ; => false

; clojure.core/keys* not so much
  (doall (keys m))
  (println @was-called))  ; => true

Add documentation/examples

but that’s pretty well covered by the doc-strings.

Downloading the library and diving into docstrings is too much work if I just want to know how it works/if it solves my problems.

A simple example or two would help a lot I think.

`def-map-type` breaks in namespaces that shadow core Clojure methods like `instance?`

If you have a namespace that does something like :refer-clojure :exclude [instance?] and use def-map-type in it,

(merge instance-of-my-map-type nil)

will break because

def-map-type uses the def-abstract-type AbstractMap which defines cons like this:

(cons [this o]
(cond
(map? o)
(reduce #(apply assoc %1 %2) this o)
(instance? java.util.Map o)
(reduce #(apply assoc %1 %2) this (into {} o))
:else
(if-let [[k v] (seq o)]
(assoc this k v)
this)))

and the ultimate macroexpansion copies that call to instance? directly, rather than qualifying it as clojure.core/instance?.

Empty defprotocol+ bodies cause definition to not occur

When defprotocol+ is called with just a name the protocol isn't actually created.

potemkin=> (defprotocol+ SomeProtocol)
nil
potemkin=> SomeProtocol

CompilerException java.lang.RuntimeException: Unable to resolve symbol: SomeProtocol in this context, compiling:(/private/var/folders/3m/tvc28b5d7p50v5_8q5ntj0pmflbdh9/T/form-init5752369463507367463.clj:1:5505) 
potemkin=> (defprotocol+ SomeProtocol2 "Doc string")
SomeProtocol2
potemkin=> SomeProtocol2
{:doc "Doc string"
 :method-builders {}
 :method-map {}
 :on potemkin.SomeProtocol2
 :on-interface potemkin.SomeProtocol2
 :sigs nil
 :var #'potemkin/SomeProtocol2}

The root cause of this is that prev-body will be nil when the protocol doesn't already exist, which will be equal to the body when just the name is present.

We can avoid this by just adding a docstring, as the REPL session above shows, but ideally defprotocol+ would behave in the same way as defprotocol from clojure.core. See http://dev.clojure.org/jira/browse/CLJ-966

potemkin=> (clojure.core/defprotocol NormalProtocol)
NormalProtocol
potemkin=> NormalProtocol
{:method-builders {}
 :method-map {}
 :on potemkin.NormalProtocol
 :on-interface potemkin.NormalProtocol
 :sigs nil
 :var #'potemkin/NormalProtocol}

Our use case for this was somewhat odd; we basically have a protocol that we want to remove but it still needs to be resolvable. The only usages were inside a macro that we control, so we can just get rid of it and substitute something else, but we still need to have a Var and a class of the same name that are resolvable.

AbstractMethodError when using def-map-type without defining meta

Potemkin: 0.3.7
Clojure: 1.6.0
Java: 1.7

Having trouble with even the simple example from the README. This is in a fresh repl:

user=> (require '[potemkin.collections :refer [def-map-type]])
nil
user=> 

user=> (def-map-type LazyMap [m]   
  #_=>   (get [_ k default-value]
  #_=>     (if (contains? m k)
  #_=>       (let [v (get m k)]
  #_=>         (if (instance? clojure.lang.Delay v)
  #_=>           @v
  #_=>           v))
  #_=>       default-value))
  #_=>   (assoc [_ k v]
  #_=>     (LazyMap. (assoc m k v)))
  #_=>   (dissoc [_ k]
  #_=>      (LazyMap. (dissoc m k)))
  #_=>   (keys [_]
  #_=>     (keys m)))
user.LazyMap
user=> (LazyMap. {})

AbstractMethodError user.LazyMap.meta_STAR_()Ljava/lang/Object;  user.LazyMap (form-init3281556375762400900.clj:78)
user=> (->LazyMap {})

AbstractMethodError user.LazyMap.meta_STAR_()Ljava/lang/Object;  user.LazyMap (form-init3281556375762400900.clj:78)

Defining a meta method fixes it, but the README implies that it shouldn't be required:

Despite this, there are only four functions which really matter: get, assoc, dissoc, and keys. def-map-type is a variant of deftype which, if those four functions are implemented, will look and act like a Clojure map.

(def-map-type LazyMap [m]   
  (get [_ k default-value]
    (if (contains? m k)
      (let [v (get m k)]
        (if (instance? clojure.lang.Delay v)
          @v
          v))
      default-value))
  (assoc [_ k v]
    (LazyMap. (assoc m k v)))
  (dissoc [_ k]
     (LazyMap. (dissoc m k)))
  (keys [_]
    (keys m)
  (meta [_] nil)) ; this makes the readme example work

Add Serializable support

Long story short, I am using clj-http HeaderMap (which uses def-map-type) via Storm. However, I am getting a java.io.NotSerializableException when it's being serialized. Would you be interested in a patch that adds java.io.Serializable to the types?

docs: describe defprocotol+

Potemkin includes defprotocol+ but the README does not make mention of it.

A recent conversation on slack reminded me I moved from an inlined version of defprotocol+ to defprotocol when merging rewrite-clj v0 and rewrite-cljs v0 to create rewrite-clj v1.

At the time I, wrongly, assumed defprotocol+ was somehow related to performance. Maybe I was swayed by the definterface+ description mentioning "time and memory". I dunno.

@dpsutton kindly helped me to understand that defprotocol+ is about REPL reload friendliness and deals with this kind of scenario.

I'd be happy to add a few words in the README about this with the sentiment that it is something that would have helped me understand, so it will probably help others.

NPE while using import-vars with records

I have a few record defined in civs.model.basic and I would like to make available to users importing civs.model.core, so I tried this:

(ns
  ^{:author ftomassetti}
  civs.model.core
  (:require
    [civs.model.basic :refer :all]
    [potemkin :refer :all])
  (:import
    [civs.model.basic Tribe Population Town]))

(import-vars
  [civs.model.basic
   Tribe Population Town])

I got this:

[federico@normandie civs]$ lein test
Exception in thread "main" java.lang.NullPointerException, compiling:(civs/model/core.clj:8:47)
    at clojure.lang.Compiler.load(Compiler.java:7142)
    at clojure.lang.RT.loadResourceScript(RT.java:370)
    at clojure.lang.RT.loadResourceScript(RT.java:361)
    at clojure.lang.RT.load(RT.java:440)
    at clojure.lang.RT.load(RT.java:411)
    at clojure.core$load$fn__5066.invoke(core.clj:5641)
    at clojure.core$load.doInvoke(core.clj:5640)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5446)
    at clojure.core$load_lib$fn__5015.invoke(core.clj:5486)
    at clojure.core$load_lib.doInvoke(core.clj:5485)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:626)
    at clojure.core$load_libs.doInvoke(core.clj:5524)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:626)
    at clojure.core$require.doInvoke(core.clj:5607)
    at clojure.lang.RestFn.invoke(RestFn.java:1289)
    at civs.io$eval418$loading__4958__auto____419.invoke(io.clj:1)
    at civs.io$eval418.invoke(io.clj:1)
    at clojure.lang.Compiler.eval(Compiler.java:6703)
    at clojure.lang.Compiler.eval(Compiler.java:6692)
    at clojure.lang.Compiler.load(Compiler.java:7130)
    at clojure.lang.RT.loadResourceScript(RT.java:370)
    at clojure.lang.RT.loadResourceScript(RT.java:361)
    at clojure.lang.RT.load(RT.java:440)
    at clojure.lang.RT.load(RT.java:411)
    at clojure.core$load$fn__5066.invoke(core.clj:5641)
    at clojure.core$load.doInvoke(core.clj:5640)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5446)
    at clojure.core$load_lib$fn__5015.invoke(core.clj:5486)
    at clojure.core$load_lib.doInvoke(core.clj:5485)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:626)
    at clojure.core$load_libs.doInvoke(core.clj:5524)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:626)
    at clojure.core$require.doInvoke(core.clj:5607)
    at clojure.lang.RestFn.invoke(RestFn.java:1096)
    at civs.core$eval188$loading__4958__auto____189.invoke(core.clj:1)
    at civs.core$eval188.invoke(core.clj:1)
    at clojure.lang.Compiler.eval(Compiler.java:6703)
    at clojure.lang.Compiler.eval(Compiler.java:6692)
    at clojure.lang.Compiler.load(Compiler.java:7130)
    at clojure.lang.RT.loadResourceScript(RT.java:370)
    at clojure.lang.RT.loadResourceScript(RT.java:361)
    at clojure.lang.RT.load(RT.java:440)
    at clojure.lang.RT.load(RT.java:411)
    at clojure.core$load$fn__5066.invoke(core.clj:5641)
    at clojure.core$load.doInvoke(core.clj:5640)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5446)
    at clojure.core$load_lib$fn__5015.invoke(core.clj:5486)
    at clojure.core$load_lib.doInvoke(core.clj:5485)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:626)
    at clojure.core$load_libs.doInvoke(core.clj:5524)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:626)
    at clojure.core$require.doInvoke(core.clj:5607)
    at clojure.lang.RestFn.invoke(RestFn.java:619)
    at civs.acceptance_test$eval182$loading__4958__auto____183.invoke(acceptance_test.clj:1)
    at civs.acceptance_test$eval182.invoke(acceptance_test.clj:1)
    at clojure.lang.Compiler.eval(Compiler.java:6703)
    at clojure.lang.Compiler.eval(Compiler.java:6692)
    at clojure.lang.Compiler.load(Compiler.java:7130)
    at clojure.lang.RT.loadResourceScript(RT.java:370)
    at clojure.lang.RT.loadResourceScript(RT.java:361)
    at clojure.lang.RT.load(RT.java:440)
    at clojure.lang.RT.load(RT.java:411)
    at clojure.core$load$fn__5066.invoke(core.clj:5641)
    at clojure.core$load.doInvoke(core.clj:5640)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5446)
    at clojure.core$load_lib$fn__5015.invoke(core.clj:5486)
    at clojure.core$load_lib.doInvoke(core.clj:5485)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:626)
    at clojure.core$load_libs.doInvoke(core.clj:5524)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:626)
    at clojure.core$require.doInvoke(core.clj:5607)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:626)
    at user$eval85.invoke(form-init8154316109887346215.clj:1)
    at clojure.lang.Compiler.eval(Compiler.java:6703)
    at clojure.lang.Compiler.eval(Compiler.java:6693)
    at clojure.lang.Compiler.load(Compiler.java:7130)
    at clojure.lang.Compiler.loadFile(Compiler.java:7086)
    at clojure.main$load_script.invoke(main.clj:274)
    at clojure.main$init_opt.invoke(main.clj:279)
    at clojure.main$initialize.invoke(main.clj:307)
    at clojure.main$null_opt.invoke(main.clj:342)
    at clojure.main$main.doInvoke(main.clj:420)
    at clojure.lang.RestFn.invoke(RestFn.java:421)
    at clojure.lang.Var.invoke(Var.java:383)
    at clojure.lang.AFn.applyToHelper(AFn.java:156)
    at clojure.lang.Var.applyTo(Var.java:700)
    at clojure.main.main(main.java:37)
Caused by: java.lang.NullPointerException
    at clojure.core$with_meta.invoke(core.clj:214)
    at potemkin.namespaces$import_def.invoke(namespaces.clj:67)
    at clojure.lang.Var.invoke(Var.java:394)
    at clojure.lang.AFn.applyToHelper(AFn.java:165)
    at clojure.lang.Var.applyTo(Var.java:700)
    at clojure.lang.Compiler.macroexpand1(Compiler.java:6552)
    at clojure.lang.Compiler.macroexpand(Compiler.java:6613)
    at clojure.lang.Compiler.macroexpand(Compiler.java:6615)
    at clojure.lang.Compiler.eval(Compiler.java:6687)
    at clojure.lang.Compiler.eval(Compiler.java:6692)
    at clojure.lang.Compiler.load(Compiler.java:7130)
    ... 99 more
Tests failed.

Source for 0.3.13

0.3.13 was released this month, but this repo hasn't been updated since march and contains 0.3.12

Has the repo moved, or is there some other reason the code has not been pushed? 😕

Document how `import-vars` interacts with direct-linking and namespace reloading

It seems like there would be issue with direct-linking, AOT, namespace reloading etc, and import-vars can we get some documentation on these interactions? I've run in to quite a few Potemkin related bugs in the past in these areas and I'd love to see some documentation on import-vars if we're going to continue to see this library crop up in CLJ libraries.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.