Comments (11)
Small feedback on charred, it looks like write-json-fn
is the preferred api for performance. Something like:
(def write-json (json/write-json-fn {:escape-slash false :indent-str nil}))
(.toString (doto (java.io.StringWriter.) (write-json x)))
from mulog.
I totally agree with your frustration I shouldn't have accepted the PR: #47
We gained some performance improvements of a couple of microseconds at the cost of a dependency hell.
However swapping again, isn't the solution, there are other constraints.
Currently, if you have to serialise objects other than clojure maps you have to register and encoder using Jasonista APIs
https://cljdoc.org/d/com.brunobonacci/mulog/0.7.1/doc/howtos/how-to-json-encode-custom-java-classes
swapping implementation would certainly disrupt all the people who use this.
Generally adding the jackson dependency directly in your project solves the problem:
it is important to put the jackson dependencies on TOP of the dependency list, just after Clojure
:dependencies [[org.clojure/clojure "1.10.1"]
;; put these on TOP of the dependency list, just after clojure
[com.fasterxml.jackson.core/jackson-core "2.12.0"]
[com.fasterxml.jackson.dataformat/jackson-dataformat-smile "2.12.0"]
[com.fasterxml.jackson.dataformat/jackson-dataformat-cbor "2.12.0"]
;; then the rest
[com.brunobonacci/mulog "0.6.4"]
]
I'm open to suggestions here, maybe making a PR for cheshire to use the same Jackson version would simplify things for the community (fewer conflicts in general).
from mulog.
clojure.data.json
should have the same capabilities that jsonista
provides. I've implemented the java.awt.Color
example:
(require '[clojure.data.json :as json]
'[clojure.string :as str])
(defn write-color
[color out options]
(let [hex-code (str/upper-case
(str "#"
(Integer/toHexString (.getRed color))
(Integer/toHexString (.getGreen color))
(Integer/toHexString (.getBlue color))))]
(json/-write hex-code out options)))
(extend java.awt.Color json/JSONWriter {:-write write-color})
(json/write-str {:color (java.awt.Color. 202 255 238)}) ;; -> "{\"color\":\"#CAFFEE\"}"
Regardless, you have a point that it would result in a breaking public api change. In my opinion, getting rid of jackson would be worth that breaking change but I'm aware that's not the general sentiment in the clojure ecosystem.
It's probably worth doing a quick grep to see how many people are using the encoders functionality.
EDIT:
the only public repos that I can find that use mulog are yours: https://grep.app/search?q=mulog&filter[lang][0]=Clojure
we currently use mulog but the repo is private so it won't show up.
from mulog.
I'm sure all JSON libraries have a way to extend, but the issue is that it will affect the people who currently use Jasonista/Jackson extension APIs (like you pointed out).
Most of μ/log users are on the corporate side hence no public repos.
from mulog.
Looks like borkdude made some selmer changes recently around pluggable JSON.
The paradigm (or something similar) would likely work here too.
https://github.com/yogthos/Selmer/pull/263/files
from mulog.
Hi,
Interesting approach.
I've looked a bit more into clojure.data.json
and despite the performance improvements it is still significantly slower than Jasonista (and Cheshire).
I've made PR for Chesire bumping the Jackson dependency to the same version as Jasonista, hopefully, it will be accepted and merged.
There is one more issue on removing the Jasonista dependency: the default behaviour of Jasonista for serialising Java classes for which no custom encoder has been defined is to use reflection and generate a JSON representation.
On the other hand, clojure.data.json
throws an exception. Now it is disputable which approach is better, however, maybe people might have involuntarily leveraged this behaviour without even knowing.
Switching JSON library would certainly cause more and more errors to appear in places that currently do work fine.
I'll promise to look deeper into this issue, and find a better long term solution.
from mulog.
Thanks for the effort, I appreciate it!
Likely, adopting some well-defined key in the publisher config where users can do arbitrary manipulations seems to me to be the best solution (akin to c.d.j's :key-fn
and :value-fn
).
from mulog.
Just wanted to put this on the radar in case it's a good candidate...
Posted by
u/chrisnuernberger
14 days ago
Fast JSON and CSV encode/decode
Introducing Charred - fast json/csv encode and decode. This library finalizes my research into csv and json parsing and is a complete drop-in replacement for clojure.data.csv and clojure.data.json. Same API, much better (5-10x) performance. This library gets as good performance for those tasks as anything on the JVM and avoids the jackson hairball entirely...
... Finally this library has the same conformance suite as the libraries it replaces so you can feel at least somewhat confident it will handle your data with respect.
from mulog.
Thanks @jonathn that's very interesting...
I will have a look at Charred
.
from mulog.
I've pushed on the main branch an implementation of mulog-json
which replaces Jasonista
with Charred
.
I've also published this version to Clojars as [com.brunobonacci/mulog "0.9.0-SNAPSHOT"]
and [com.brunobonacci/mulog-json "0.9.0-SNAPSHOT"]
, please do let me know if anyone is experiencing conflicts or change in behaviours.
From a μ/log perspective, there is no behaviour change, but if you have custom serialisers you will need to update your code as described here
from mulog.
This is now released as [com.brunobonacci/mulog-json "0.9.0"]
from mulog.
Related Issues (20)
- Character code for `µ` HOT 2
- Does mulog publisher for zipkin support Annotations? HOT 1
- Extract file/line info from callsites?
- µ/log publisher for OpenTelemetry HOT 5
- Logging maps HOT 7
- Merge map keys into µ/log statement HOT 2
- Add payload length to the zipkin publisher HOT 1
- AWS Cloudwatch logs are not generally "discoverable" HOT 2
- Publish errors with Cloudwatch Logs HOT 8
- Slack Publisher Documentation implies support for Block Kit messages HOT 1
- Errors serializing to JSON are unreasonably difficult to debug HOT 2
- Publishers continually retrying HOT 3
- JSON-file publisher? HOT 4
- `:data-stream` shows up as index in Elasticsearch 8 HOT 3
- :mulog/publisher-error NullPointerException HOT 4
- Cascading errors in elasticsearch publisher when using multi publisher HOT 8
- mulog-elasticsearch does not work with data-streams HOT 3
- suggestion: lmax disruptor queues HOT 2
- ElasticSearch `type-mangle` fn causes type problems HOT 1
- Put event name in options map HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mulog.