jackson issues + swap to `clojure.data.json` about mulog HOT 11 CLOSED

Small feedback on charred, it looks like write-json-fn is the preferred api for performance. Something like:

(def write-json  (json/write-json-fn  {:escape-slash false :indent-str nil}))
(.toString (doto (java.io.StringWriter.) (write-json x)))

from mulog.

Hi @darindouglass

I totally agree with your frustration I shouldn't have accepted the PR: #47
We gained some performance improvements of a couple of microseconds at the cost of a dependency hell.

However swapping again, isn't the solution, there are other constraints.
Currently, if you have to serialise objects other than clojure maps you have to register and encoder using Jasonista APIs

https://cljdoc.org/d/com.brunobonacci/mulog/0.7.1/doc/howtos/how-to-json-encode-custom-java-classes

swapping implementation would certainly disrupt all the people who use this.

Generally adding the jackson dependency directly in your project solves the problem:
it is important to put the jackson dependencies on TOP of the dependency list, just after Clojure

  :dependencies [[org.clojure/clojure "1.10.1"]
                 ;; put these on TOP of the dependency list, just after clojure   
                 [com.fasterxml.jackson.core/jackson-core "2.12.0"]
                 [com.fasterxml.jackson.dataformat/jackson-dataformat-smile "2.12.0"]
                 [com.fasterxml.jackson.dataformat/jackson-dataformat-cbor "2.12.0"]
                 ;; then the rest
                 [com.brunobonacci/mulog "0.6.4"]
                 ]

I'm open to suggestions here, maybe making a PR for cheshire to use the same Jackson version would simplify things for the community (fewer conflicts in general).

from mulog.

clojure.data.json should have the same capabilities that jsonista provides. I've implemented the java.awt.Color example:

(require '[clojure.data.json :as json]
         '[clojure.string :as str])

(defn write-color
  [color out options]
  (let [hex-code (str/upper-case
                  (str "#"
                       (Integer/toHexString (.getRed color))
                       (Integer/toHexString (.getGreen color))
                       (Integer/toHexString (.getBlue color))))]
    (json/-write hex-code out options)))

(extend java.awt.Color json/JSONWriter {:-write write-color})
(json/write-str {:color (java.awt.Color. 202 255 238)}) ;; -> "{\"color\":\"#CAFFEE\"}"

Regardless, you have a point that it would result in a breaking public api change. In my opinion, getting rid of jackson would be worth that breaking change but I'm aware that's not the general sentiment in the clojure ecosystem.

It's probably worth doing a quick grep to see how many people are using the encoders functionality.

EDIT:

the only public repos that I can find that use mulog are yours: https://grep.app/search?q=mulog&filter[lang][0]=Clojure
we currently use mulog but the repo is private so it won't show up.

from mulog.

I'm sure all JSON libraries have a way to extend, but the issue is that it will affect the people who currently use Jasonista/Jackson extension APIs (like you pointed out).

Most of μ/log users are on the corporate side hence no public repos.

from mulog.

Looks like borkdude made some selmer changes recently around pluggable JSON.

The paradigm (or something similar) would likely work here too.

https://github.com/yogthos/Selmer/pull/263/files

from mulog.

Hi,

Interesting approach.
I've looked a bit more into clojure.data.json and despite the performance improvements it is still significantly slower than Jasonista (and Cheshire).

I've made PR for Chesire bumping the Jackson dependency to the same version as Jasonista, hopefully, it will be accepted and merged.

There is one more issue on removing the Jasonista dependency: the default behaviour of Jasonista for serialising Java classes for which no custom encoder has been defined is to use reflection and generate a JSON representation.
On the other hand, clojure.data.json throws an exception. Now it is disputable which approach is better, however, maybe people might have involuntarily leveraged this behaviour without even knowing.
Switching JSON library would certainly cause more and more errors to appear in places that currently do work fine.

I'll promise to look deeper into this issue, and find a better long term solution.

from mulog.

Thanks for the effort, I appreciate it!

Likely, adopting some well-defined key in the publisher config where users can do arbitrary manipulations seems to me to be the best solution (akin to c.d.j's :key-fn and :value-fn).

from mulog.

Just wanted to put this on the radar in case it's a good candidate...

Posted by u/chrisnuernberger 14 days ago
Fast JSON and CSV encode/decode
Introducing Charred - fast json/csv encode and decode. This library finalizes my research into csv and json parsing and is a complete drop-in replacement for clojure.data.csv and clojure.data.json. Same API, much better (5-10x) performance. This library gets as good performance for those tasks as anything on the JVM and avoids the jackson hairball entirely...
... Finally this library has the same conformance suite as the libraries it replaces so you can feel at least somewhat confident it will handle your data with respect.

from mulog.

Thanks @jonathn that's very interesting...

I will have a look at Charred.

from mulog.

I've pushed on the main branch an implementation of mulog-json which replaces Jasonista with Charred.

I've also published this version to Clojars as [com.brunobonacci/mulog "0.9.0-SNAPSHOT"] and [com.brunobonacci/mulog-json "0.9.0-SNAPSHOT"], please do let me know if anyone is experiencing conflicts or change in behaviours.

From a μ/log perspective, there is no behaviour change, but if you have custom serialisers you will need to update your code as described here

from mulog.

This is now released as [com.brunobonacci/mulog-json "0.9.0"]

from mulog.