inist-cnrs / node-jbj Goto Github PK

View Code? Open in Web Editor NEW

13.0 9.0 3.0 822 KB

Like XSL/XML but for JSON.

Home Page: http://Inist-CNRS.github.io/jbj-playground/

JavaScript 100.00%

node-jbj's Introduction

JBJ : transform Json By Json

A new way to transform JSON object with a JSON stylesheet. It's like XSL/XML but only with JSON.

Try it online !

http://inist-cnrs.github.io/jbj-playground/

Contributors

Installation

With npm do:

$ npm install jbj

Tests

Use mocha to run the tests.

$ npm install
$ npm test

Documentation

API

render(stylesheet : Object, input : Mixed, callback : Function) : None

Render input with stylesheet.

	var JBJ = require('jbj'),
	JBJ.render({ "truncate" : 3 }, "1234", function(err, out) {
			console.log(out);
	});

	// Output : 123

Variables

Variable can be set using $ plus a dot notation path. The set value can only be a JBJ expression (not a JSON literal).

Input:

{
  "a": {
    "b": {
      "c": 1
    },
    "d": "Second"
  }
}

Stylesheet:

{
    "$x" : {
        "get": "a.b.c"
    },
    "$y.y1.y2" : {
        "get": "a.d"
    }
}

Output:

{
  "a": {
    "b": {
      "c": 1
    },
    "d": "Second"
  },
  "x": 1,
  "y": {
    "y1": {
      "y2": "Second"
    }
  }
}

inject(stylesheet : Object, input : Mixed, callback : Function) : None

Alertative mode to use JBJ actions. You can inject in stylesheet the input. Injections are identified by a prefix and suffix in the key. The prefix and the suffix are the less-than sign <. Prefix inject the result of a JBJ action in the current scope. Suffix inject the result of a JBJ render in the current key.

	var JBJ = require('jbj'),
	JBJ.render({ "<truncate" : 3 }, "1234", function(err, out) {
			console.log(out);
	});

	// Output : 123

prefix

var input = {
    key : "hello"
}
var stylesheet = {
	"obj" : {
		"<get" : "key",
		upcase: true
	}
};
// output : { obj : "HELLO" }

suffix

var input = {
    key : "hello"
}
var stylesheet = {
	"obj" : {
		"keyword<" : {
			get : "key",
   		    upcase: true
		}
	}
};
// output : { obj : { keyword : "HELLO" } }

register(protocol : String, callback : Function) : None

Add a function to fetch data for a specific protocol

	JBJ.register('http:', function request(urlObj, callback) {
		var buf = ''
	      , req = require('http').get(urlObj, function(res) {
			if (res.statusCode !== 200) {
				return callback(new Error('HTTP Error ' + res.statusCode));
			}
			res.setEncoding('utf8');
			res.on('data', function (chunk) {
			  buf += chunk.toString();
			});
			res.on('error', callback);
			res.on('end', function() {
			  callback(null, buf);
			});
		});
		req.on('error', callback);
	});

getActions() : Array

List all available actions.

	var JBJ = require('jbj'),

	console.log(JBJ.getActions());

	// Output : ['get', 'set', 'expect' ...]

getFilter(filterName : String) : Function

Get the statement function for an action name.

	var JBJ = require('jbj'),

	var func = JBJ.getFilter('append');

	func('Hello', 'World' , function(err, output) {
	   console.log(output)
	})

	// Output : HelloWorld

use(module: Function) : None

Adding filters/actions for external module. see the avaible modules here : https://www.npmjs.com/browse/keyword/jbj

	var JBJ = require('jbj'),

	JBJ.use(require('jbj-numerical'));
	JBJ.use(require('jbj-ist'));
	JBJ.use(require('jbj-rdfa'));

Warning: the method has change since v4.0

Actions

A stylesheet is a JSON object where each key is an action. The actions are divided into modules (since v4.0):

basics: all the basic actions for JBJ (debug, default, extend, set, get, add, expect, foreach, select, cast, mask, trim, required, assert, breakif, inject, render, compute)
ejs: mainly the filters borrowed from EJS (first, last, capitalize, downcase, upcase, slug, sort, sort_by, size, max, min, plus, minus, times, dividedBy, join, truncate, shift, truncateWords, replace, prepend, append, reverse, flatten, deduplicate, remove, sum, slug)
parse: file format conversion, through parsing (csv, parseCSV, parseCSVFile, json, parseJSON, xml, parseXML)
template: template, templateURL
array: complex actions implying arrays (mapping, mappingVar, zip, array2object, arrays2objects, coalesce, substring, getindex, getindexvar)
rdfa: generate HTML+RDFa from a JSON-LD (getJsonLdField, style, class, tag, toHtml)
nlp: natural language processing (anglicize, countCharacters, countWords, tokenize, metaphone)
numerical: numerical text processing (floatprint, loselose, pearson, sdbm, fnv1a, murmur3, distincter, reseter)
jsonld: facilitate the generation of JSON-LD from JSON (context, jsonld)

The modules you can use by defaults are basics and ejs.

To use another module, first install it via npm:

$ npm install jbj-array

then declare its use via:

JBJ.use(require('jbj-array'));

Note: basics and ejs modules are distributed with the jbj package, and used by default: you can use their actions without any further declaration. However, parse, template, and array, which were parts of the pre-3.0 versions of JBJ are now separate packages (simply add jbj- before the modules names to get their matching packages).

set: value

module: basics

Set value and ignore input

	var stylesheet1 = {
		"set": "value"
	};
	var stylesheet2 = {
		"set": ["value", "value", "value"]
	};
	var stylesheet3 = {
		"set": {
			"a": 1,
			"b": 2
		}
	};

get: path | [path,path, ...]

module: basics
aliases : find , path

Get value in input with some paths (with dot notation style)

	var stylesheet1 = {
		"set": {
			"a" : {
				"b" : {
					"c" : "Yo"
				}
			}
		},
		"get": "a.b.c"
	};
	// output : Yo
	var stylesheet2 = {
		"set" : {
			"a" : {
				"b" : 1,
				"c" : 2,
				"d" : 3
			}
		},
		"get": ["a.b", "a.c", "a.d"]
	};
// output : [1, 2, 3]

default: value

module: basics

Fix value if input is not set

	var stylesheet = {
		var stylesheet = {
			"default": "value"
		};
	};

fetch

module: basics
aliases : fetchURL, $?

Stylesheet can contain a reference to data source. Source can be a file or an URL. By default, only the file: protocol is supported. Add your own protocol with register

	var stylesheet_1 = {
      "fetch" : "https://raw.githubusercontent.com/castorjs/node-jbj/master/package.json",
      "$name" : {
        "upcase": true
      },
      "$main": {
        "upcase": true
      }
    };
	var stylesheet_2 = {
      "$name" : {
        "fetch" : "file://" + path.resolve(__dirname, '../dataset/1.json'),
        "parseJSON" : true,
        "path": "name"
      },
      "$main": {
        "fetch" : "file://" + path.resolve(__dirname, '../dataset/1.json'),
        "parseJSON" : true,
        "path": "main",
      }
    };

add: [key, value]

module: basics

Add a key/value pair into the input object.

{
  "input": { },
  "stylesheet": {
    "add": ["tag", "span"]
  },
  "expected": {
    "tag": "span"
  }
}

{
  "input": {
    "content": "not empty"
  },
  "stylesheet": {
    "add": ["tag", "span"]
  },
  "expected": {
    "content": "not empty",
    "tag": "span"
  }
}

expect: object

module: basics

Set default key/values for the input object: when a key is not present in the input object, it is set the value given in the argument object.

{
  "input": {
    "a": 3
  },
  "stylesheet": {
    "expect": {
      "a": 1,
      "b": 2
    }
  },
  "expected": {
    "a": 3,
    "b": 2
  }
}

{
  "stylesheet": {
    "expect": {
      "a": 1,
      "b": 2
    }
  },
  "expected": {
    "a": 1,
    "b": 2
  }
}

inject:

module: basics

Return a new Object with JBJ.inject.

render:

module: basics

Return a new Object with JBJ.render.

compute:

module: basics

Compute an expression with all variables of the input. Use the filtrex syntax. Note : this variable contains input

var stylesheet = {
    "set" : {
        "a" : 20,
        "b" : 3,
        "c" : 5,
        "d" : 8
    },
    "$x" : {
        "compute#1": "a / b",
        "compute#2": "round(this)",
        "cast": "number"
    },
    "$y" : {
        "path": "b",
        "cast": "number"
    },
    "$z" : {
        "compute": "x + y",
    }
};
// output : 10

debug: none

module: basics

Print input with console.log

	var stylesheet = {
		"set": "value",
		"debug": true
	};
	// output: value

foreach: stylesheet

module: basics

Apply stylesheet on all elements of input

	var stylesheet1 = {
		"set": ["value", "value", "value"]
		"foreach" : {
			"upcase" : true
		}
	};
	// output : ["VALUE", "VALUE", "VALUE"]
	var stylesheet2 = {
		"set": [
			{ "b" : "x" },
			{ "b" : "y" }
		],
		"foreach" : {
			"get": "b",
			"upcase" : true
		}
	};
	// output : ["X", "Y"]
	var stylesheet3 = {
		"set": [
			{ "b" : "x" },
			{ "b" : "y" }
		],
		"foreach" : {
			"$b" : {
				"get": "b",
				"upcase" : true
			}
		}
	};
	// output : [ { "b" : "X" }, { "b" : "Y" } ]

extend: object

module: basics
aliases : extendWith

Extend input with another object

	var stylesheet = {
		"set": {
			"a" : 1
		},
		"extend" : {
			"b" : 2
		}
	};
	// output : { a: 1, b: 2}

select: path | [path, path, ...]

module: basics

Peck element(s) in input with "CSS selector"

	var stylesheet = {
		"set" : {
			"a" : {
				"b" : [
					{ "c" : "1" },
					{ "c" : "2" }
				]
			}
		},
		"select" : ".a > .b .#c"
	};
	// output : [1, 2]

for syntax see JSONSelect and JSONSelect Expression Tester

Note: select always returns an array (an empty array when nothing was selected).

cast: (number|string|boolean) | [(string|date), pattern]

module: basics

Convert input to specific type

	var stylesheet1 = {
		"set" : "1"
		"cast": "number"
	};
	// output : 1
	var stylesheet2 = {
		"set" : 1
		"cast": "string"
	};
	// output: "1"

for syntax see transtype

mask: pattern

module: basics

Selecting specific parts of input, hiding the rest, return object

	var stylesheet = {
		"set" : {
			"a" : 1,
			"b" : 2,
			"c" : 3
		},
		"mask": "a,c"
	};
	// output : { a: 1, c: 3}

For syntax see json-mask

omit: pattern

module: basics

Unselecting specific parts of input, show the rest, return object

    var stylesheet = {
        "set" : {
            "a" : 1,
            "b" : 2,
            "c" : 3
        },
        "omit": "a,c"
    };
    // output : { b: 2 }

For syntax see object.omit

required: none

module: basics

If input is not set, return Error

trim: none

module: basics

Trim input, return string

	var stylesheet = {
		"set" : "    xxx    ",
		"trim": true
	};
	// output : "xxx"

assert: expression

module: basics

If expression is true, then statements will be continued, otherwise it is stopped and it returns null

Note : this variable contains input

	var stylesheet1 = {
		"set" : {
			"a" : 1
		},
		"$val#1" : {
			"assert": "a == 1",
			"set" : "if val"
		}
	};
	// output : "if val"
	var stylesheet2 = {
		"set" : {
			"a" : 0
		},
		"$val#1" : {
			"assert": "a == 1",
			"set" : "if val"
		},
	    "$val#2" : {
			"get" : "val",
	        "default": "else val",
		  }
	};
	// output : "else val"

capitalize:

module: ejs

Capitalize the first letter of input

	var stylesheet = {
		"set" : "xyz",
		"capitalize": true
	};
	// output : "Xyz"

downcase:

module: ejs

Downcase input

	var stylesheet = {
		"set" : "XYZ",
		"downcase": true
	};
	// output : "xyz"

upcase:

module: ejs

Uppercase input

	var stylesheet = {
		"set" : "xyz",
		"upcase": true
	};
	// output : "XYZ"

slug:

module: ejs

Convert the input string to something valid in an URI. See https://tools.ietf.org/html/rfc3986

{
  "slug spaces": {
    "input": "with space",
    "stylesheet": {
      "slug": true
    },
    "expected": "with-space"
  },

  "slug lowercase": {
    "input": "IN UPPERCASE",
    "stylesheet": {
      "slug": true
    },
    "expected": "in-uppercase"
  },

  "slug diacritics": {
    "input": "En français",
    "stylesheet": {
      "slug": true
    },
    "expected": "en-francais"
  },

  "slug diacritics #2": {
    "input": "Le Cinquième Élément",
    "stylesheet": {
      "slug": true
    },
    "expected": "le-cinquieme-element"
  },

  "slug unicode": {
    "input": "i ♥ unicode",
    "stylesheet": {
      "slug": true
    },
    "expected": "i-unicode"
  }
}

first:

module: ejs

Get the first element of input

	var stylesheet = {
		"set" : ["a", "b", "c"],
		"first": true
	};
	// output : "a"

last:

module: ejs

Get the last element of input

	var stylesheet = {
		"set" : ["a", "b", "c"],
		"last": true
	};
	// output : "c"

sort:

module: ejs

Sort input object or array.

	var stylesheet = {
		"set": ["b", "c", "a"],
		"sort": true
	};
	// output : ["a", "b", "c"]

sortBy: prop | [prop, prop, ...]

module: ejs
aliases : sort_by

Sort input object the given prop ascending.

	var stylesheet = {
		"set": [
			{ "name": "zert" },
			{ "name": "abcd" }
		],
		"sortBy": "name"
	};
	// output : [{ "name": "abcd" }, { "name": "zert" }]

size:

module: ejs
aliases : length

Get the size or the length of input

	var stylesheet1 = {
		"set" : "12345",
		"size": true
	};
	var stylesheet2 = {
		"set" : [1,2,3,4,5],
		"size": true
	};
	// output : 5

max:

module: ejs

Add input and value

	var stylesheet1 = {
		"set" : [2, 4, 1, 7, 9, 3],
		"max" : true
	};
	// output : 9
	var stylesheet2 = {
		"set" : {a: 9, b: 4, c: 3, d: 5},
		"max" : true
	};
	// output : 9

min:

module: ejs

Subtract value from input

	var stylesheet1 = {
		"set" : [2, 4, 1, 7, 9, 3],
		"min" : true
	};
	// output : 1
	var stylesheet2 = {
		"set" : {a: 9, b: 4, c: 3, d: 5},
		"min" : true
	};
	// output : 3

plus: value | [value, value, ...]

module: ejs

Add input and value

	var stylesheet = {
		"set" : 4,
		"plus": 3
	};
	// output : 7
	var stylesheet = {
		"set" : 4,
		"plus": [1,2,3]
	};
	// output : [5,6,7]

minus: value | [value, value, ...]

module: ejs

Subtract value from input

	var stylesheet = {
		"set" : 4,
		"minus": 3
	};
	// output : 1
	var stylesheet = {
		"set" : 4,
		"minus": [1,2,3]
	};
	// output : [3,2,1]

times: value | [value, value, ...]

module: ejs

Multiply input by value"

	var stylesheet = {
		"set" : 5,
		"times": 5
	};
	// output : 25
	var stylesheet = {
		"set" : 4,
		"times": [1,2,3]
	};
	// output : [4,8,12]

dividedBy: value | [value, value, ...]

module: ejs
aliases : divided_by

Divide input by value"

	var stylesheet = {
		"set" : 10,
		"dividedBy": 2
	};
	// output : 5
	var stylesheet = {
		"set" : 4,
		"times": [1,2]
	};
	// output : [4,2]

join: string = ', '

module: ejs
aliases : glue

Join input with the given string.

	var stylesheet = {
		"set" : ["a","b","c"],
		"join": " | "
	};
	// output : "a | b | c"

shift: n | [n, n, ...]

module: ejs

Shift input to the left by n

	var stylesheet = {
		"set" : "The world",
		"shift": 4
	};
	// output : "world"
	var stylesheet = {
		"set" : [1,2,3,4,5],
		"shift": 2
	};
	// output : [3,4,5]
	var stylesheet = {
		"set" : [1,2,3,4,5],
		"shift": [2,3]
	};
	// output : [[3,4,5],[4,5]]

truncate: length | [length, length, ...]

module: ejs

Truncate input to length.

	var stylesheet = {
		"set" : "hello world",
		"truncate": 5
	};
	// output : "hello"

truncateWords: n | [n, n, ...]

module: ejs
aliases : truncate_words

Truncate input to n words (separator: space).

	var stylesheet = {
		"set" : "This is JBJ!",
		"truncateWords": 2
	}
	// output "This is"
	var stylesheet = {
		"set" : "This is JBJ!",
		"truncateWords": [1,2]
	}
	// output ["This","This is"]

replace: [pattern, substitution] | pattern

module: ejs

Replace pattern (as a regular expression) with substitution in input.

	var stylesheet = {
		"set" : "XoXoXoX",
		"replace": ["o", "."]
	};
	// output :  X.X.X.X
	var stylesheet = {
		"set" : "XoXoXoX",
		"replace": "o"
	};
	// output :  XXXX

Tip: to escape any character, use \\ instead of just \. Example: use "replace": "\$trash\$" removes (trash) from input, whereas "replace": "(trash)" removes only trash.

prepend: something | [something, something, ...]

module: ejs

Prepend something to input

	var stylesheet = {
		"set" : "world"
		"prepend": "hello"
	};
	// output : "hello world"
	var stylesheet = {
		"set" : "h"
		"prepend": ["a","e","i","o","u"]
	};
	// output : ["ah","eh","ih","oh","uh"]

append: something | [something, something, ...]

module: ejs

Append something to input

	var stylesheet = {
		"set" : "cool",
		"append": "!"
	};
	// output : "cool!"
	var stylesheet = {
		"set" : "cool",
		"append": ["!","?","."]
	};
	// output : ["cool!","cool?","cool."]

reverse:

module: ejs

Reverse items order of input

	var stylesheet = {
		"set" : [1,2,3]
	};
	// output : [3,2,1]

flatten:

module: ejs

Flatten an array.

    var stylesheet = {
      "set"     : [ ['a', 'b'], ['c', 'd'], 'e'],
      "flatten" : true
    };
	// output : ["a","b","c","d","e"]

deduplicate:

module: ejs
aliases : dedupe , unique

Deduplicate values in an array.

    var stylesheet = {
      "set"         : [ 1, 2, 3, 1, 2],
      "deduplicate" : true
    };
	// output : [1,2,3]

remove:

module: ejs
alias : del

Remove one value in an array.

    var stylesheet = {
      "set"    : [ 1, 2, 3],
      "remove" : 2
    };
	// output : [1,3]
    var stylesheet = {
      "set"    : [ "a", "", "b"],
      "remove" : ""
    };
    // output : ["a","b"]
    var stylesheet = {
      "set"    : [ "a", "b", "c"],
      "remove" : "b"
    };
    // output : ["a","c"]

sum:

module: ejs
alias : total

Return the sum of all the value of an array.

    var stylesheet = {
      "set"    : [ 1, 2, 3],
      "sum" : true
    };
	// output : 6

FAQ

How to chain the same action

just add #

	var stylesheet = {
		"default":  "123456789",
		"truncate#1": 8,
		"truncate#2": 4,
		"truncate#3": 2
	};

How to use input as variable in an expression

just use this

	var stylesheet = {
      "$e" : {
        "compute#1": "a / b",
        "compute#2": "round(this)",
        "cast": "number"
      }
    }

How to find more examples

see unit tests : https://github.com/castorjs/node-jbj/tree/master/test

Also

License

MIT

node-jbj's People

Contributors

Stargazers

Watchers

Forkers

nojhamster claussi touv

node-jbj's Issues

Support JSON5

in the playground (is it compatible)
In a new method that can analyze String instead an Object

Debug traces when using JBJ command

In several JBJ stylesheets, there is parasite output (key and res beginning lines in the following sample):

key 0
key 0
key 1
key 0
key 0
key 0
res [ 'd' ]
res [ 'c', 'r' ]
res [ 'n' ]
res [ 'm' ]
res [ 'm' ]
5542C09F834F6D9586B5F92017918EB675357E88, DC,placename,1,3,ISTEX,TK,Null,d,3
5542C09F834F6D9586B5F92017918EB675357E88, Costa Rica,placename,2,11,ISTEX,KSTRK,Null,"c,r",3
5542C09F834F6D9586B5F92017918EB675357E88, Niger,placename,1,6,ISTEX,NJR,Null,n,3
5542C09F834F6D9586B5F92017918EB675357E88, Malta,placename,1,6,ISTEX,MLT,Null,m,3
5542C09F834F6D9586B5F92017918EB675357E88, Morocco,placename,1,8,ISTEX,MRKK,Null,m,3

No obvious console present in JBJ source...

Add remove action

Sometimes, we get empty values in arrays (after a get or select on an array of paths):

["Animals", "Organisms", ""]

We may want to remove this empty value.

"remove": ""

could be useful.

Another use case would be when we want to remove the most occurent value (for example a value present in all documents, which is not discriminant).

"remove": "USA"

"remove": "Biology"

bin/jbj

It could be great to have a jbj command line.

use case:

cat data.json | jbj '{ "get": "test" }' > mytransformeddata.json

Add a "zip" action to join two arrays

See http://stackoverflow.com/questions/22015684/how-do-i-zip-two-arrays-in-javascript

To compute expressions involving the values of two arrays, it would be great to be able to join the two same-length arrays.

Example:

{
  "publiPerYear": [
    {"_id":"2007","value":538},
    {"_id":"2008","value":577},
    {"_id":"2009","value":611},
    {"_id":"2010","value":548},
    {"_id":"2011","value":567},
    {"_id":"2012","value":608}],
  "citationsPerYear": [
    {"_id":"2007","value":7681},
    {"_id":"2008","value":5479},
    {"_id":"2009","value":5043},
    {"_id":"2010","value":3698},
    {"_id":"2011","value":2927},
    {"_id":"2012","value":2049}]
}

We would like to divide citationsPerYear[i] by publiPerYear[i]:

{
  "zip": [ "citationsPerYear", "publiPerYear" ],
  "foreach": {
    "compute": "this.citationsPerYear / this.publiPerYear"
  }
}

In this example, "zip": [ "citationsPerYear", "publiPerYear" ] should return:

[
    {"_id":"2007","citationsPerYear":7681, "publiPerYear": 538},
    {"_id":"2008","citationsPerYear":5479, "publiPerYear": 577},
    {"_id":"2009","citationsPerYear":5043, "publiPerYear": 611},
    {"_id":"2010","citationsPerYear":3698, "publiPerYear": 548},
    {"_id":"2011","citationsPerYear":2927, "publiPerYear": 567},
    {"_id":"2012","citationsPerYear":2049, "publiPerYear": 608}]

assert action does not work in async mode

In playground, having

input:

{
    "a": 2
}

stylesheet:

{
    "$val": {
        "assert": "a == 1",
               "set": "ko"
    }
}

gives:

{
 "a": 2,
 "val": "ko"
}

It seems that it works with JBJ.renderSync, but not with JBJ.render.

JSONSelect installation number limited by GitHub

The version of JSONSelect used is the one forked by @touv , and which URL is https://github.com/touv/JSONSelect/archive/master.tar.gz (see package.json).

That means that JBJ installation is constrained in rate by unauthenticated GitHub requests rate limit: one can install JBJ from one IP at most 60 times per hour (this limit can go drastically down if your behind a proxy, and/or if you do other requests to GitHub from the same IP during same hours).

Maybe we should publish on npm the forked JSONSelect?

Add a "getPropertyVar" action

Like the getProperty action, but taking the value of a variable, instead of a literal:

Ex:

{
  "set": {
    "i": 1,
    "t": ["a","b","c"]
  },
  "getPropertyVar": ["t", "i"]
}

should return

"b"

And

{
  "set": {
    "i" : "b",
    "o" : { "a": 0, "b": 1, "c":2 },
  },
  "getPropertyVar": ["o", "i"]
}

should return

Add a "mapArray" action

Need: transform an array of { number: occurrences } to an array of number * occurrences.
computeis not sufficient, since we need to apply the expression to each element of the array.

Ex:

    "$citations": {
      "default": 0,
      "$?": "local:///compute.json?operator=distinct&field=NbCitations&itemsPerPage=10000000",
      "parseJSON": true,
      "select": ".data",
      "mapArray": "_id * value"
    }

which would transform:

{
  "data": [
  { "_id": "0", "value": 882 },
  { "_id": "1","value": 372 }
  ]
}

into

[0, 372]

Try it link is 404

http://castorjs.github.io/node-jbj/

Add a "getProperty" action

At the moment, there is no way to extract one element of an array / one property of an object.

Ex:

{
  "set": [ "a", "b", "c" ],
  "getProperty": "0"
}

should return

"a"

And

{
  "set": { "a": 0, "b": 1, "c":2 },
  "getProperty": "b"
}

should return

Add flatten, and deduplicate action

When using multiple path as a parameter to get or select, you can get an array of arrays (if at least one of the paths leads to a multivalued field):

[['b','c'],['e','a'],['a','d']]

A flatten action could produce:

['b','c','e','a','a','d']

And then, a deduplicate (or unique, dedupe) action could produce:

['b','c','e','a','d']

Add a "array2object" action

When you have an array containing elements of _id and value (no random example, this is what castorjs operators return), you may need to transform them to one object where _ids are property names and values are their values.

Ex:

[
{
    "_id": "2007",
    "value": 538
},
{
    "_id": "2008",
    "value": 577
},
{
    "_id": "2009",
    "value": 611
}
]

should be transformed into:

{
  "2007": 538,
  "2008": 377,
  "2009": 611
}

That way, you can access the 2008 value using getPropertyVar.

So, maybe the array2object action should have [ "_id", "value" ] as a default parameter (when the parameter is not an array).

introduce "node-jbj-ist" in the playground

cqfd

Add a command to jbj-command to install all known modules

$ jbj --install-all-modules

Add a sum filter

sum would sum the elements of an array.

Assert seems to lose context

After a assert on a true value, already created preoperties are no more accessible.

Ex:
input:

{
"content":
  {
    "json": 
    {
      "doi": "10.1007/1-4020-2365-0_11"
    }
  }
}

stylesheet:

{
  "$doi": {
    "path": "content.json.doi",
    "default":"0"
  },
  "$doiurl": {
    "assert": "doi!=0",
    "get": "doi",
   "prepend": "http://dx.doi.org/"
  }
}

output:

{
  "content": {
    "json": {
      "doi": "10.1007/1-4020-2365-0_11"
    }
  },
  "doi": "10.1007/1-4020-2365-0_11",
  "doiurl": "http://dx.doi.org/undefined"
}

Where doiurl should not contain undefined but the value of doi.

When you remove the assert part of the stylesheet, prepend works as intented.

jbj-array: coalesce documentation not up to date

The coalesce documentation says that coalesce returns only the first value not undefined, but this is no more true: it returns an array with all non null or undefined values.

Tests does not really cover coalesce.

Link to github project is not active any more in JBJ playground!

Likely a CSS problem, preventing to click on the Fork me on GitHub link.

jbj-command: parsing error when the input is a CSV

input.csv:

type-article,1
entitee-nommee,2
data-set,3

stylesheet.json:

{
  "parseCSVFile": ",",
  "foreach": {
    "get": 0,
    "prepend": "http://data-set.lod.istex.fr/=/",
    "append": "?alt=jsonld",
    "fetch": true,
    "parseJSON": true,
    "get#2":"0._description"
  }
}

Using jbj-command:

hector@troie:~/Documents/API-test$ jbj -u parse stylesheet.json input2.csv
/home/hector/.nvm/versions/node/v4.2.3/lib/node_modules/jbj-command/node_modules/json5/lib/json5.js:54
            throw error;
            ^

SyntaxError: Expected 'r' instead of 'y'
    at JSON5.parse.error (/home/hector/.nvm/versions/node/v4.2.3/lib/node_modules/jbj-command/node_modules/json5/lib/json5.js:50:25)
    at JSON5.parse.next (/home/hector/.nvm/versions/node/v4.2.3/lib/node_modules/jbj-command/node_modules/json5/lib/json5.js:62:17)
    at JSON5.parse.word (/home/hector/.nvm/versions/node/v4.2.3/lib/node_modules/jbj-command/node_modules/json5/lib/json5.js:345:17)
    at JSON5.parse.value (/home/hector/.nvm/versions/node/v4.2.3/lib/node_modules/jbj-command/node_modules/json5/lib/json5.js:478:56)
    at Object.parse (/home/hector/.nvm/versions/node/v4.2.3/lib/node_modules/jbj-command/node_modules/json5/lib/json5.js:491:18)
    at ReadStream.<anonymous> (/home/hector/.nvm/versions/node/v4.2.3/lib/node_modules/jbj-command/lib/jbj-command.js:120:19)
    at emitNone (events.js:72:20)
    at ReadStream.emit (events.js:166:7)
    at endReadableNT (_stream_readable.js:905:12)
    at doNTCallback2 (node.js:441:9)
    at process._tickCallback (node.js:355:17)

The concerned line tries to JSON5.parse the input.
In the case of non-JSON files (or JSON5 files), this yield an error.

Maybe when the first character is not in [{", we could nest the input characters between two ".

How to ?

I have this data:

{ keyAsString: '7051', key: 160340860800000, docCount: 1 }

I tried this stylesheet:

      {
        "$year": {
          "get" : "keyAsString",
          "cast": "number"
        },
        "$nbDocument": {
          "get" : "docCount",
        },
      }

I got this result:

{ keyAsString: '9167',
  key: 227115360000000,
  docCount: 1,
  year: 9167,
  nbDocument: 1 }

I'd like to have this result instead:

{ year: 9167,
  nbDocument: 1 }

Thanks in advance for any help.

Add substring action

For the moment, if you want to extract the month from a string "20150310", you have to mess withtruncateandshift`.

{
  "truncate": 6,
  "shift": 4
}

Maybe a substring: [offset, length] action would be more user friendly:

{
  "substring": [4,2]
}

Try it section more visible

put the try it section at the top of the readme and in the github repository description
http://castorjs.github.io/node-jbj/

cause coming to jbj without this online helper is very rude

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.