invalid JSON configuration files about fairroot HOT 8 CLOSED

Regarding the multiple keys - the functionality to avoid this is already there, as you can see in other examples. This file was overlooked, I'll fix it in the next commit.

The integer values also work, you can use integer instead of string for sndBufSize, etc.

from fairroot.

fair enough.

what I am reaching at would be some kind of an official schema for the device(s) configuration, or really, anything under the "fairMQOptions" key.

i.e. something like that, in Go:

type Config struct {
    Options Options `json:"fairMQOptions"`
}

type Options struct {
    Devices []Device `json:"devices"`
}

type Device struct {
    ID       string    `json:"id"`
    Channels []Channel `json:"channels"`
}

type Channel struct {
    Name    string   `json:"name"`
    Sockets []Socket `json:"sockets"`
}

type Socket struct {
    Type        string `json:"type"`
    Method      string `json:"method"`
    Address     string `json:"address"`
    SendBufSize int    `json:"sndBufSize"`
    RecvBufSize int    `json:"rcvBufSize"`
    RateLogging int    `json:"rateLogging"`
}

this would help document what is accepted by FairMQ as a configuration file.

I can produce a little Go program that tries to load JSON files from the FairRoot repo and detect those which do not abide to the above schema.

(EDIT: updated the json:"sockets" struct-tag of type Channel)

from fairroot.

ah!

and what about the "key" and "id" keys in: "examples/MQ/2-sampler-processor-sink/ex2-sampler-processor-sink.json" ?
have they different meaning? different usage?

thanks.
-s

from fairroot.

The "id" corresponds to the device ID, which should be unique across the topology. The "key" allows multiple devices to share same configuration. In the second example multiple processors are started with the same config.

The Go snippet above it looks good, except socket can also be multiple. The program you mention would indeed be helpful.

The idea was to allow singular keys (device, channel, etc.) for cases where there is only one entry, to avoid having to create an array.

There is one more new feature that hasn't been used a lot yet - if no socket is specified, common settings defined under channel will be applied to the socket(s) of this channel. You can see it in the json of the third example.

from fairroot.

the command at:
https://github.com/sbinet-alice/fer/blob/master/cmd/fer-validate-json/main.go

gives something like this:

$> fer-validate-json -v ./example/cmd/testdata/ex2-sampler-processor-sink.json 
2016/11/09 10:44:36 ./example/cmd/testdata/ex2-sampler-processor-sink.json: use of "channel" keyword (device="sampler1")
2016/11/09 10:44:36 ./example/cmd/testdata/ex2-sampler-processor-sink.json: use of "socket" keyword (device="sampler1.data1")
2016/11/09 10:44:36 ./example/cmd/testdata/ex2-sampler-processor-sink.json: use of "socket" keyword (device="processor.data1")
2016/11/09 10:44:36 ./example/cmd/testdata/ex2-sampler-processor-sink.json: use of "socket" keyword (device="processor.data2")
2016/11/09 10:44:36 ./example/cmd/testdata/ex2-sampler-processor-sink.json: use of "channel" keyword (device="sink1")
2016/11/09 10:44:36 ./example/cmd/testdata/ex2-sampler-processor-sink.json: use of "socket" keyword (device="sink1.data2")
2016/11/09 10:44:36 [./example/cmd/testdata/ex2-sampler-processor-sink.json] validation FAILED

it doesn't detect int as string, yet.
(it wasn't clear whether this was something you wanted to enforce)

from fairroot.

The singular words are valid if they are not repeated (e.g. "socket" for single object or "sockets" for an array).

Both int and string work for the numerical values from the perspective of the FairMQParser. Maybe put it as a warning?

from fairroot.

I've updated all the non-standard JSON files. Now everything should be valid JSON. I also added a readme in fairmq/options to describe the details of the JSON configuration.

from fairroot.

ok.
the command fer-json-validate now produces:

$> fer-json-validate -v ./toto.json 
fer-json-validate: [./toto.json] validation FAILED:
ref={fairMQOptions {devices [{id sampler1 channel {name data1 socket {type push method bind address tcp://*:5555 sndBufSize 1000 rcvBufSize 1000 rateLogging 0}}channel {name data1 socket {type push method bind address tcp://*:5555 sndBufSize 1000 rcvBufSize 1000 rateLogging 0}}}{key processor channels [{name data1 socket {type pull method connect address tcp://localhost:5555 sndBufSize 1000 rcvBufSize 1000 rateLogging 0}}{name data2 socket {type push method connect address tcp://localhost:5556 sndBufSize 1000 rcvBufSize 1000 rateLogging 0}}]}{id sink1 channel {name data2 socket {type pull method bind address tcp://*:5556 sndBufSize 1000 rcvBufSize 1000 rateLogging 0}}}]}}
chk={fairMQOptions {devices [{id sampler1 channel {name data1 socket {type push method bind address tcp://*:5555 sndBufSize 1000 rcvBufSize 1000 rateLogging 0}}}{key processor channels [{name data1 socket {type pull method connect address tcp://localhost:5555 sndBufSize 1000 rcvBufSize 1000 rateLogging 0}}{name data2 socket {type push method connect address tcp://localhost:5556 sndBufSize 1000 rcvBufSize 1000 rateLogging 0}}]}{id sink1 channel {name data2 socket {type pull method bind address tcp://*:5556 sndBufSize 1000 rcvBufSize 1000 rateLogging 0}}}]}}

when toto.json has multiple "channel" keys.
less pretty, but still useful.

I have also produced a json reformatter a-la clang-format (or, rather, like go fmt from where clang-format draws its inspiration).
it's there:
https://github.com/sbinet-alice/fer/blob/master/cmd/fer-json-fmt/main.go

automatically applying this kind of reformatter would reduce the git-diff-noise...

wrt the int VS string issue, I think it would be better to be stricter to begin with (with maybe be more lenient if need be).
i.e. only accept int for the buffer sizes and rates.

from fairroot.