andreyvit / json-diff Goto Github PK
View Code? Open in Web Editor NEWStructural diff for JSON files
License: MIT License
Structural diff for JSON files
License: MIT License
If I change a value from true to false it gives me this output:
...
{
- alive: true
}
...
It does not recognize that false was added, only that true is gone
It would be nice do a fine-grained comparison into nested lists, instead of marking the whole list as different.
Test case: diff [1, [2,3]]
and [1, [2,42]]
Current result:
[
1
- [
- 2
- 3
- ]
+ [
+ 2
+ 42
+ ]
]
Expected result:
[
1
[
2
- 3
+ 42
]
]
I have an array of objects, and json-diff
BEAUTIFULLY shows me that two deep elements are different.
However, I'd like to be able to always include a non-different element when printing a diff in an object.
Example: (printing id
)
{
abilities: {
Nova: [
{
id: "Snipe",
- description: "Deals 268 (+0.04 per level) damage to the first enemy hit."
+ description: "Deals 295 (+0.04 per level) damage to the first enemy hit."
}
]
}
talents: {
7: [
{
id: "Snipe Master",
- description: "All stacks are lost if Snipe fails to hit an enemy Hero."
+ description: "All stacks are lost if Snipe fails to hit an enemy."
}
]
}
}
{
abilities: {
Rehgar: [
{
id: "Another Ability",
- cooldown: 70
+ cooldown: 80
}
]
}
Without id
, I'd have no idea what the object is in the array of objects.
baseJson: [{
"a": 100,
"b": 100
}]
actualJson:
[{
"a": 101,
"b": 105
}]
threshold = 0.01
diff = abs((actualJsonValue - baseJsonValue)/baseJsonValue)
if diff > threshold; then regard as diff;
else, it regard as no diff
In this case
abs((101-100) / 100) > 0.01 is false, so key a is no diff
abs((105-100) / 100) > 0.01 is ture, so key b is diff
With recent update as of yesterday, running into Error: Cannot find module 'iced-runtime'
after running commands. Looks like this module requirement isn't specified in the package now?
example:
json1
{
'test': [
{
'a': 1,
'b': 2
},
{
'a': 1,
'b': 2
},
]
}
json2
{
'test': [
{
'a': 1,
'b': 2
},
{
'a': 1,
'b': 2
},
]
}
json-diff was working perfectly fine with my angular 11 but when i migrated to 12. it started giving errors for assert. So I installed it ( npm i assert) after doing this now when i run ng serve it is givng an error in browser console.
core.js:6498 ERROR Error: Uncaught (in promise): RangeError: Wrong length!
RangeError: Wrong length!
at validateArrayBufferArguments (core.js:5566:40)
at new ArrayBuffer (core.js:5656:30)
at 88048 (types.js:206:1)
at __webpack_require__ (bootstrap:19:1)
at 94655 (util.js:467:1)
at __webpack_require__ (bootstrap:19:1)
at 15025 (assertion_error.js:35:16)
at __webpack_require__ (bootstrap:19:1)
at 25652 (assert.js:36:22)
at __webpack_require__ (bootstrap:19:1)
at resolvePromise (zone.js:1211:1)
at resolvePromise (zone.js:1165:1)
at zone.js:1278:1
at _ZoneDelegate.invokeTask (zone.js:406:1)
at Object.onInvokeTask (core.js:28679:1)
at _ZoneDelegate.invokeTask (zone.js:405:1)
at Zone.runTask (zone.js:178:1)
at drainMicroTaskQueue (zone.js:585:1)
Please check if your version if this is not echma script.
As you can see by the screenshot, the diff tool seems to be up and straight missing. I don't know what the tool is missing here, but these two files are the same ( they're actually the exact same content, used for testing purposes. )
I'm not really able to research and build a PR, sadly. I'm going to have to look for another tool. If anyone can figure this one out that would be awesome. :)
Edit: Just thought I'd mention that this is a series of objects in an array that's nested in an object which is a part of a series nested in an array, etc.
Creating a single file consisting of a JSON array with three entries: a string, an object, then a duplicate of that object and then diffing that file with itself reports differences (despite diffing the file with itself, which should be identical).
$ echo '[ "a", { "foo": "bar" }, { "foo": "bar" } ]' >array.json
$ json-diff --full array.json array.json
[
"a"
- {
- foo: "bar"
- }
{
foo: "bar"
}
+ {
+ foo: "bar"
+ }
]
Removing the initial string entry correctly reports no diff, so that's somehow a trigger for the bug. I'm guessing something is going wrong with the diff trying to identify moved array elements (which it should probably only be doing after it's detected that array elements aren't structurally equal?).
Tested locally with version 0.7.1 using Node v12.22.10.
Runkit demonstrating issue directly with NodeJS API: https://runkit.com/embed/gpzkiqddw0e0
I had to log out what properties the exported package contained to know that it had diff
and diffString
README could benefit from letting people know this isn't limited to being a commandline tool
this is not a issue, but a feature request.
If we can ignore some fields from getting the diff, that would be great.
TL;DR: json-diff should depend on dreamopt 0.8.0 instead of 0.6.0.
Version 0.6.0 has a corrupted tarball. npm
seems to handle this fine, but other package managers like pnpm
fail to install it (you can also try extracting it yourself using 7zip, and observing the error).
Because of this, running pnpm install
on a package that depends on json-diff
will fail with the following message:
ERROR· Unexpected end of data
--
174 | at _final ../../../../../../../../../root/.rush/node-v12.13.0/pnpm-4.9.3/node_modules/pnpm/lib/node_modules/tar-stream/extract.js:253 if (this._partial) retur…
175 | at callFinal ../../../../../../../../../root/.rush/node-v12.13.0/pnpm-4.9.3/node_modules/pnpm/lib/node_modules/tar-stream/node_modules/readable-stream/lib/_stream_writable.js:590 stream._final(function (…
176 | at processTicksAndRejections internal/process/task_queues.js:80
In other words, pnpm
users are blocked from using json-diff
until it updates its dependency on cloudopt. The most recent version of cloudopt (0.8.0) has a fixed tarball. I did not test 0.7.0.
Just as this issue pointed out, you get this specific module error
#46
Cannot assign to read only property 'name' of function 'function SequenceMatcher2(isjunk, a, b, autojunk) {
this.isjunk = isjunk;
if (a == null) {
This is just from the import example given in the readme and on npm
import { diffString, diff } from 'json-diff';
Which means it's not usable on a client app in the browser.
Is there a way to import this that works on browser. I'm using this with svelte/vite/javascript/chrome
I created a benchmark suite to test multiple JSON diff frameworks head-to-head: https://github.com/justsml/json-diff-performance
It's coverage is hardly complete, but it's pretty easy to extend - main goals were testing ops/sec and output string byte length.
Hope this helps in performance tuning and comparing various library techniques.
🚨 You need to enable Continuous Integration on all branches of this repository. 🚨
To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because it uses your CI build statuses to figure out when to notify you about breaking changes.
Since we didn’t receive a CI status on the greenkeeper/initial
branch, it’s possible that you don’t have CI set up yet. We recommend using Travis CI, but Greenkeeper will work with every other CI service as well.
If you have already set up a CI for this repository, you might need to check how it’s configured. Make sure it is set to run on all new branches. If you don’t want it to run on absolutely every branch, you can whitelist branches starting with greenkeeper/
.
Once you have installed and configured CI on this repository correctly, you’ll need to re-trigger Greenkeeper’s initial pull request. To do this, please delete the greenkeeper/initial
branch in this repository, and then remove and re-add this repository to the Greenkeeper App’s white list on Github. You'll find this list on your repo or organization’s settings page, under Installed GitHub Apps.
$ npm install json-diff
npm http GET https://registry.npmjs.org/json-diff
npm http 304 https://registry.npmjs.org/json-diff
npm http GET https://registry.npmjs.org/cli-color
npm http GET https://registry.npmjs.org/dreamopt
npm http GET https://registry.npmjs.org/difflib
npm http 304 https://registry.npmjs.org/difflib
npm http 304 https://registry.npmjs.org/dreamopt
npm http 304 https://registry.npmjs.org/cli-color
npm http GET https://registry.npmjs.org/es5-ext
npm http GET https://registry.npmjs.org/wordwrap
npm http 304 https://registry.npmjs.org/wordwrap
npm http GET https://registry.npmjs.org/heap
npm http 304 https://registry.npmjs.org/heap
npm http 304 https://registry.npmjs.org/es5-ext
[email protected] node_modules/json-diff
├── [email protected] ([email protected])
├── [email protected] ([email protected])
└── [email protected] ([email protected])
$ ls node_modules/json-diff
LICENSE-MIT README.md bin node_modules package.json
node_modules/json-diff/lib directory does not exists. I can't run json-diff.
why?
Hi!
We have an old application that could use this library, but because it's old we don´t have any kind of module resolution. Any chance to offer a global namespace for this?
I'm trying to diff something that has a lot of lines that are moved, i'd like to be able to ignore these in order to see if something else changed, e.g. spelling or added something new.
incase i'm not making any sense, this is what i want to be able to ignore
myobject: {
alist: [
+ "something"
"foobarstuff"
- "something"
]
}
where "something" is the same but just moved down a row, but hasn't changed content.
The library wasn't noticing differences in dates. I've forked and here's a link to the commit with my test and fix.
Fork commit
json-diff overlooks differences of number and string in array.
For example, when json-diff compares [1]
with ["1"]
, returns nothing.
expected:
[
- 1
+ "1"
]
const jsonDiff = require('json-diff')
let a = {
"s": [[{ "b": '123' }]]
}
let b = {
"s": [[{ "b": 'abc' }], []],
}
let diff_out = jsonDiff.diffString(a, b);
console.log(diff_out);
b object has empty array will throw error at
/node_modules/json-diff/lib/index.js:100
if (!keyScores[bestMatch.key] || bestMatch.score > keyScores[bestMatch.key].score) {
^
TypeError: Cannot read property 'key' of null
In my pipeline I was using json-diff to monitor different languages files.
Now with the latest versions I always get success even if there are missing keys between files.
json-diff ....
{
container_form: {
+ required_fields: "The name of the {entity}, the warehouse and the unit are required fields"
}
}
echo $?
#=> 0
Is it possible to have the exit code not 0 when json-diff found some differences between files, like it was before?
Hi,
Thank you for your library that we use in our company, it is excellent for our needs.
We are currently using the diff function without options, which returns this type of response. However we find it awkward in our context not to have the unmodified values of an array.
"updatedScope": {
"value": {"__old": "test", "__new": "dev"},
"values": [ [" "], ["+", "item1"]]
}
We have tested the full
option, which is not adapted to our case because it returns the whole object, with the values out of context.
{
"nonModifiedScope": {...},
"updatedScope": {
"value": {"__old": "test", "__new": "dev"},
"values": [ [" ", "item0"], ["+", "item1"]]
}
}
I would like to suggest an improvement that would allow to keep the unmodified values in the diff of an object, without returning the whole object.
This would consist in adding an option, for example keepUnchangedValues: boolean
, which would be taken into account here in the same way as the full
option, while remaining in the context of modified values.
ideal answer will look like that:
"updatedScope": {
"value": {"__old": "test", "__new": "dev"},
"values": [ [" ", "item0"], ["+", "item1"]]
}
What do you think of this option which would bring a lot of clarity in the context of the modified values scope?
As you're no doubt familiar with, Github's diffs show not only which lines have changed, but also frequently which bits of the line changed, when you make changes to an already existing line.
I'm not sure exactly how this is implemented, but I imagine it would be something like: perform the usual diff algorithm using lines as "atoms". Then, if the resulting diff would have a -
followed by +
, perform a char-by-char diff of these two lines. If the lines are at least n% similar, highlight the char-by-char diff in a stronger formatting to emphasize which bits of the line changed.
I think something similar for string values in the JSON data would be very handy. In the particular JSON data I've worked with, I'm importing data from a source, and from one import to the next there might've been typo fixes or small changes to description strings in the data. It would be very convenient to be able to tell at a glance exactly which bits were added/removed. So, I think it would be nice to perform a standard char-by-char diff of string values, if there is a -
of a string value, followed by a +
of a string value.
Is there a way to see common keys/values?
I'd like to have an inbrowser json compare tool. My app is all client side. Can this be used in browser?
diffString
is erroring out when it comes to an empty array of objects:
const patch = diffString(
{
services: [],
},
{
services: [{ service_code: '123' }],
}
);
index.js:54 TypeError: Cannot read properties of null (reading 'key')
at JsonDiff.scalarize (index.js:103)
at JsonDiff.arrayDiff (index.js:144)
at JsonDiff.diff (index.js:289)
at JsonDiff.objectDiff (index.js:39)
at JsonDiff.diff (index.js:286)
at diff (index.js:315)
at diffString (index.js:319)
at DiffTest (index.stories.js:54)
at finalStoryFn (story_store.js:568)
at hooks.js:182
I need to compare many large objects that I shouldn't mutate. Some will have empty arrays of objects which will cause the app to crash.
Would it be possible to fix this?
** Update: this was not working on version 0.6.1
apparently there was a fix earlier today -- working on 0.6.3
I found that in a json array, if an object has 2 or more keys, the diff result is different. Here is an example with 2 keys then 3 keys :
$ cat a.json
[
{
"key1": "a",
"key2": "12"
}
]
$ cat b.json
[
{
"key1": "b",
"key2": "1"
},
{
"key1": "a",
"key2": "12"
},
{
"key1": "c",
"key2": "1"
}
]
$ json-diff a.json b.json
[
+ {
+ key1: "b"
+ key2: "1"
+ }
...
+ {
+ key1: "c"
+ key2: "1"
+ }
]
$ cat c.json
[
{
"key1": "a",
"key2": "12",
"key3": "cm"
}
]
$ cat d.json
[
{
"key1": "b",
"key2": "1",
"key3": "m"
},
{
"key1": "a",
"key2": "12",
"key3": "cm"
},
{
"key1": "c",
"key2": "1",
"key3": "dm"
}
]
$ json-diff c.json d.json
[
{
- key1: "a"
+ key1: "b"
- key2: "12"
+ key2: "1"
- key3: "cm"
+ key3: "m"
}
+ {
+ key1: "a"
+ key2: "12"
+ key3: "cm"
+ }
+ {
+ key1: "c"
+ key2: "1"
+ key3: "dm"
+ }
]
Is this behaviour expected ?
Currently, when diffing arrays, elements that are objects and equivalent on both sides get elided with a ...
. However, the fact that the elision marker is there is important to highlight the position of the differences in the array.
In some cases (especially when diffing API responses and similar), the arrays might be very long, and the output gets rather tedious. Instead of one elision marker for each elided object, I suggest to replace repetated elision with a single line.
E.g.,
% json-diff <(echo '{ "hello": [{},{},{},{},{},{},1] }') <(echo '{ "hello": [{},{},{},{},{},{},2] }')
{
hello: [
...
...
...
...
...
...
- 1
+ 2
]
}
would instead become
% json-diff <(echo '{ "hello": [{},{},{},{},{},{},1] }') <(echo '{ "hello": [{},{},{},{},{},{},2] }')
{
hello: [
... (6 entries)
- 1
+ 2
]
}
For smaller arrays, it would get noisy and reduce clarity to use the elisions everywhere. Therefore, I think it makes sense to by default only collapse e.g. 5 or more elisions into one line. Probably makes sense to provide a command-line flag to set this threshold, though!
I would like to compare two JSON files to see what is different, but also what is the same.
Currently, it seems that json-diff prints out some 'context' lines around diff chunks, but there is no option for printing out all of first.json, even if it is the same as second.json.
Could such an option be added?
For some inputs, the diff shows an existing item in the list as added, and the new item as only having one field modified:
$ cat one.json
[
{ "key": "value1", "foo": "exists", "bar": "exists" }
]
$ cat two.json
[
{ "key": "value2", "foo": "new", "bar": "new" },
{ "key": "value2", "foo": "exists", "bar": "exists" }
]
$ json-diff one.json two.json
[
{
- key: "value1"
+ key: "value2"
}
+ {
+ key: "value2"
+ foo: "exists"
+ bar: "exists"
+ }
]
The diff above shows the { "key": "value2", "foo": "exists", "bar": "exists" }
as being added, and fails to output the "new"
string values at all.
If the list items are reversed in two.json
, the diff output is even more wrong:
$ cat one.json
[
{ "key": "value1", "foo": "exists", "bar": "exists" }
]
$ cat two.json
[
{ "key": "value2", "foo": "exists", "bar": "exists" },
{ "key": "value2", "foo": "new", "bar": "new" }
]
$ json-diff one.json two.json
[
{
- key: "value1"
+ key: "value2"
- foo: "exists"
+ foo: "new"
- bar: "exists"
+ bar: "new"
}
+ {
+ key: "value2"
+ foo: "new"
+ bar: "new"
+ }
]
In this case, it shows the existing item as being removed, and the new item as being added twice.
I'm getting:
difflib.js:78 Uncaught TypeError: Cannot assign to read only property 'name' of function 'function SequenceMatcher(isjunk, a, b, autojunk)
for:
SequenceMatcher = (function() {
SequenceMatcher.name = 'SequenceMatcher'; // <<== here
Trying to compare two files of size 701MB results in
FATAL ERROR: CALL_AND_RETRY_0 Allocation failed - process out of memory
Hi. Running npm test
just gives me:
0 tests complete (1 ms)
Am I missing something? Are the tests distributed separately?
Please add an option to filter out certain keys.
Like diff maybe use the capital i as parameter.
For example:
json-diff -I id -I created_at -I updated_at -I random_object_id a.json b.json
Or maybe read a json file with an array of keys to be ignored
json-diff --ignore_file ignore_keys.json
So those keys will be ignored, in the comparison.
It would be a great help in comparing json files across a production and test environment.
Document in readme on how to use options from code.
Do we have the provision to call "json-diff" from javascript files and use the diff obtained for some further purpose ?
like var jsonDiff = require('json-diff');
$ cat t.json
{"foo":[1,2,3]}
$ json-diff t.json t.json
undefined
Ideally there would be an option (or default behavior) whereby no output would be generated if the objects are the same.
Hi there,
nice tool here :)
I ran into (what I assume is) a small bug, shown by this example:
var jsondiff = require("json-diff");
var val = [{}];
console.log(jsondiff.diff(val, val)); // => [ [ '-', {} ], [ '+', {} ] ]
Which happens in the cli as well:
$ cat a.json
[{}]
$ cat b.json
[{}]
$ json-diff a.json b.json
[
- {
- }
+ {
+ }
]
I understand if this is hard to fix based on the nature of diffing JSON in the first place, but it would be a nice improvement to have :)
I should mention that this actually occurred in a larger object for me, and this is just the simplest version I could find. I can provide the larger example too if that would be helpful.
Usage: import { diff } from 'json-diff';
v0.7.1 Working perfectly.
v0.7.2 Throws an error:
Error: Webpack Compilation Error
./node_modules/json-diff/lib/colorize.js 17:34
Module parse failed: Unexpected token (17:34)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
|
| const outputElisions = (n) => {
> if (n < (options.maxElisions ?? Infinity)) {
| for (let i = 0; i < n; i++) {
| output(' ', subindent + '...')
@ ./node_modules/json-diff/lib/index.js 3:21-42
"cc": NaN,
^
SyntaxError: Unexpected token N
at Object.parse (native)
at module.exports (/mnt/c/Users/REDACTED/node_modules/json-diff/lib/cli.js:31:18)
at Object.<anonymous> (/mnt/c/Users/REDACTED/node_modules/json-diff/bin/json-diff.js:2:22)
at Module._compile (module.js:410:26)
at Object.Module._extensions..js (module.js:417:10)
at Module.load (module.js:344:32)
at Function.Module._load (module.js:301:12)
at Function.Module.runMain (module.js:442:10)
at startup (node.js:136:18)
at node.js:966:3
I'd like to have an option to ignore values, so if I would compare:
a.json:
{"env":"skeleton","key":"foo"}
b.json:
{"env":"dev"}
json-diff --ignore-values a.json b.json
It would give a warning on the missing "key", but not on the different value in "env".
The current version (version 0.1.7) being used of cli-color does not have a license.
or at least support for https://esm.sh
json-diff ....
echo $?
#=> 1
There is a case that marking remains when checking the difference of arrays with the output-new-only specification.
When output-new-only is specified, it expects a changed element, so the same element ...
should also be absent.
$ echo '{"test":[1,2]}' > 1.json
$ echo '{"test":[2,3]}' > 2.json
$ json-diff -n 1.json 2.json
{
test: [
- 1
...
+ 3
]
}
$ npm list --depth=0 -g /usr/local/lib `-- [email protected]
expected.
{
test: [
3
]
}
While comparing:
{
"eventId": "PaymentMethodMigrationCreateNotification_25",
"event": {
"paymentMethod": {
"id": "PaymentMethod_3649901670902106",
"details": {
"accountNumberType": "other",
"accountType": "other",
"accountNumber": "ES7621030722890030024676"
},
"lastModifiedOn": "2020-07-17T14:23:09.000Z"
}
}
}
with
{
"eventId": "PaymentMethodMigrationCreateNotification_25",
"event": {
"paymentMethod": {
"id": "PaymentMethod_3649901670902106",
"name": "test",
"details": {
"accountNumberType": "other",
"accountNumber": "ES7621030722890030024676"
},
"lastModifiedOn": "2020-07-17T14:23:09.000Z"
}
}
}
It just shows the the object details as if it have changed (but doesn't shows the change):
{
event: { paymentMethod: { name__added: 'test', details: [Object] } }
}
The code is:
var jsonDiff = require('json-diff');
var fs = require('fs');
var f1 = fs.readFileSync("v1.json",'utf8');
var f2 = fs.readFileSync("v2.json",'utf8');
var j1 =JSON.parse(f1);
var j2 =JSON.parse(f2);
console.log(jsonDiff.diff(j1, j2, {keysOnly: true}));
a.json
"settings": { "react": { "version": "detect" // Tells eslint-plugin-react to automatically detect the version of React to use }, "import/resolver": { "node": { "extensions": [ ".js", ".jsx", ".tsx" ] }, // 解决tsconfig下的path别名导致eslint插件无法解决的bug "typescript": { "directory": "." } } }
b.json
"settings": { "react": { "version": "detect" // Tells eslint-plugin-react to automatically detect the version of React to use }, "import/resolver": { "node": { "extensions": [ ".js", ".jsx" ] }, // 解决tsconfig下的path别名导致eslint插件无法解决的bug "typescript": { "directory": "." } } }
console.log(jsonDiff.diffString(a, b);
output:
why extenstions become an object ?
The colorization is completely broken, despite the dependent console colorization library, cli-color, claiming to support Windows. Here's what my output looks like:
Starting Feeds
----
Testing: RSS (2.0)
Testing: Atom (1.0)
Testing: RDF test
C:\Users\altano\Documents\Projects\htmlparser2\node_modules\assert-diff\lib\assert-diff.js:23
throw e
^
AssertionError: didn't get expected output←[m
[
...
...
{
data: [
←[31m- "html"←[39m
←[31m- {←[39m
←[31m- }←[39m
←[32m+ "body"←[39m
←[32m+ {←[39m
←[32m+ }←[39m
]
}
...
A brief look at colorize.js leads me to believe you aren't actually using cli-color except to get the color code. I tested the library and it works fine on Windows. Can you use the helper functions built into it to actually generate the colorized strings to be used?
https://unpkg.com/browse/[email protected]/lib/colorize.js contains the ??
operator (added in #73); v0.7.1 does not.
package.json explicitly says all node versions in engines.node
(https://unpkg.com/browse/[email protected]/package.json), and in npm's semver, v0.8.0 would be a breaking change, but v0.7.2 can only be minor or patch.
Can you publish a v0.7.3 that avoids this operator?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.