An alternative encapsulation for IRC messages based on JSON about ircv3-specifications HOT 85 CLOSED

{tags: {}, 'source': 'nick!ident@host', 'command': 'TEST', 'content': [ 'Some stuff' ]}

I like this, but would rename content to args

from ircv3-specifications.

seems reasonable enough. feel free to edit the initial bug.

from ircv3-specifications.

An empty leading {} shouldn't be necessary, it's implied. If the first item is an object, check it for tags etc, else assume it's empty.

from ircv3-specifications.

All the the given examples do is move the string splitting by the space character up to the server and wrap it in JSON, no more. It does not even give us any of the advantages JSON could give the protocol.

One of the main issues with the current protocol of the inability to extend the protocol would still exist in these examples.

from ircv3-specifications.

If this is all we're going to do for JSON then I'd rather not see it done. It does nothing but create overhead as @prawnsalad has said.

from ircv3-specifications.

@jwheare Whilst that is true it doesn't really provide any major benefits to omit it.

from ircv3-specifications.

@SaberUK What about syntactic simplicity and bandwidth savings for by far the more
common case?

from ircv3-specifications.

The benefit of using this format over rfc1459-proto is that it avoids several pitfalls in rfc1459-proto which most people are completely unaware of (one of which caused EFnet to get knocked offline last year).

from ircv3-specifications.

What about syntactic simplicity

The problem we have at the moment is clients make incorrect assumptions about how to parse the protocol. I would say its considerably simpler to not have clients check the types of arguments as some people will inevitably mess it up.

and bandwidth savings

You're talking about saving three bytes.

from ircv3-specifications.

(Hi. I'm the original founder of the Twisted project, which contains an IRC implementation, both client and server, so I'm not just here as a tourist; just wanted to get that out of the way first :).)

I think it's worth noting that clients don't just make incorrect assumptions as to how to parse the protocol resulting in some minor formatting mistakes. This is not a minor issue; it leads to game-over remote code execution vulnerabilities like this and this. The right answer is to prevent anyone from writing networking software in C, of course, but that's probably a bit harder than just unifying wire-level syntax formats.

On the bandwidth-savings side of things, there are a number of JSON compression algorithms that could be used to crush these messages down.

Both for compression and extensibility, I'd strongly suggest an object-style syntax rather than a list-style syntax. For example, if the format is

    [{}, "kaniini!~kaniini@rabbit", "PRIVMSG", "#ircv3", "hello world"]\r\n

then you need placeholders for every value that you want to compress out, like so:

    [null, null, null, null, "hello world"]\r\n

but if you have an object-based syntax, like:

    {"tags": {}, "from": "kaniini!~kaniini@rabbit", "command": "PRIVMSG", "channel": "#ircv3", "content": "hello world"}\r\n

then repeated messages could omit the keys that were the same, without having to come up with positional placeholders:

    {"tags": {}, "from": "kaniini!~kaniini@rabbit", "command": "PRIVMSG", "channel": "#ircv3", "content": "hello world"}\r\n
    {"repeat": true, "content": "hello again from kaniini on #ircv3"}\r\n
    {"repeat": true, "content": "hello again from kaniini on #ircv3"}\r\n

not to mention the fact that tags could simply be omitted rather than presented as an empty object.

from ircv3-specifications.

One of the main points behind using JSON is to make the protocol easier to extend. However in doing so we must also keep low bandwidth and simplicity in mind as to keep the benefits of the current protocol.
While the previous JSON thread was structured and easy to read it was far more bandwidth heavy. And while the examples in this thread are lighter on bandwidth it does not make it any more extendable and ignores the benefits that JSON gives with its relaxed structure that makes
it extendable.

If, and that is a big if as adding a sub protocol could be very dangerous and tiresome for developers on both the server and client side of things, the core IRC functions will need to be simple and very low bandwidth. This would need to be a CAP option as the thread by @DarthGandalf mentioned, but to throw ideas out building on the examples above:

General format:
Array with 3 items. Command, Parameters, and Optional data.
[COMMAND, [PARAMS], {OPTIONALS}]

Common format:
[ "PRIVMSG", ["#irc", "hello world"] ]

Optional tags:
[ "PRIVMSG", ["#irc", "hello world"], {"tags": {"tag1": "a", "tag2": "b"}} ]

Optional IRCd/network specific data (web-client only networks would love this):
[ "PRIVMSG", ["#irc", "hello world"], {"tags": {"tag1": "a", "tag2": "b"}, "arbitrary_data": {"ircd_specific": "a", "attached": "http://example.com/book.pdf"}} ]

Something along this format would keep the most common messages small in bandwidth, but also keeps the data required in each message at minimum. An optional compression option also as @glyph stated would be advantageous for this.

To take it one step further (and improve IMO) but one step further away from low bandwidth albeit small, the suggestion from @glyph does stand well to make the PARAMS section a JSON object with named values rather than an array. This will solve any parameter ordering issues and far more open to extension in future.

Eg:
[ "PRIVMSG", {"to": "#irc", "data": "hello world"} ]

Note: I have literally just pulled these names and format from my ass. They could be renamed or juggled

from ircv3-specifications.

@glyph Repeated messages doesn't sound like a good idea (it's lots more statefulness and the thing marking a repeat is longer than the "{},null,null" that you'd be removing at most.

@prawnsalad I'd prefer for to to be outside of the arguments in that case. Makes it easier to find programmatically (when sending, in particular). Besides, naming all the parameters is for a later revision, I think. Let's kill the 1459 serialisation first.

from ircv3-specifications.

Perhaps a repeat message could be denoted by being an object instead of an array. It would override the object contained in the previous regular message.

[ "PRIVMSG", {"to": "#irc", "data": "hello world"} ]
{"to": "#irc", "data": "hello again"}

from ircv3-specifications.

Keep in mind: RFC 1459 serialization is not going away any time soon. Until most major client and server implementations support JSON, 1459 is here to stay.

That said, I think the first thing to focus on is a simple, solid, 1459-inspired baseline to build on. Something that clients and servers can implement without too much extra work.

from ircv3-specifications.

I like the object-based syntax that @glyph mentioned.
{"tags": {}, "from": "kaniini!~kaniini@rabbit", "command": "PRIVMSG", "channel": "#ircv3", "content": "hello world"}\r\n

Specifically because I like the things being able to be named, and it's identical to what was posted in the issue topic, just with objects instead of an array, so turning that into something RFC1459 friendly would be relatively the same.

from ircv3-specifications.

That requires the first revision of the standard to name every argument to every command, and is a much bigger burden on server implementers.

from ircv3-specifications.

@aji If we don't do it that way now, later down the road if we want to implement it then we're going to be supporting 3 different variations. RFC1459, JSON-1, and JSON-2.

I think a basic naming scheme would be fine for the first revision.
source = Where the message came from
destination = Where the message is going
command = PRIVMSG/JOIN/NAMES/etc
content = The payload of the command.

So joins for example:
Psudo: source command :content
RFC1459: :Shawn!test@host JOIN :#Testing1
JSON: {'tags': {}, 'source': 'Shawn!test@host', 'command': 'JOIN', 'content': '#testing'}

And privmsg
Psudo: source command destination content
RFC1459: :Shawn!test@host PRIVMSG #test :Hello #test!
JSON: {'tags': {}, 'source': 'Shawn!test@host', 'command': 'PRIVMSG', 'destination': '#test', 'content': 'Hello #test!'}

from ircv3-specifications.

@shawn-smith At least the JSON-1 parser and serialiser would be pretty simple. Also, for what you propose, content should be an array, because part of the point of this was getting rid of the RFC1459 argument parsing.

from ircv3-specifications.

@kythyria I don't get what you mean. The example I gave is 100% identical to the example @kaniini gave. Except that it is an object so things could be named.

His example
[{"tag1": "a", "tag2": "b"}, "[email protected]", "PRIVMSG", "#ircv3", "hello"]\r\n
My example
{'tags': { 'tag1': 'a', 'tag2': 'b'}, 'source': 'Shawn!test@host', 'command': 'PRIVMSG', 'destination': '#test', 'content': 'Hello #test!'}\r\n

from ircv3-specifications.

@shawn-smith It's identical for PRIVMSG. However, consider a message with more than one positional argument other than the destination, MONITOR perhaps:
MONITOR + grawity,Shawn
In your proposal, it seems like that would be
{"command": "MONITOR", "content": "+ grawity,shawn"}
when it would be better to have
{"command": "MONITOR", "content": ["+", "grawity,shawn"]}
It's even more noticeable for numerics, such as WHO replies.

from ircv3-specifications.

@kythyria How would it be different if it was inside an array instead of an object? It seems like it would still be identical to me.

Eg: We can do this in an array:
['MONITOR', ['+', 'grawity,shawn']]
or we can do this inside an object
{'command: 'MONITOR', 'content': ['+', 'grawity,shawn']}

The end result is basically the same, except that with the latter one we have the option to name things.

Also splitting 'content' into an array like that is something that will need to be addressed by people implementing it. Having the whole thing inside an array or inside an object wouldn't change that.

from ircv3-specifications.

The problem is that introducing labels results in the chunks possibly being out of order, which requires somewhat of a different codepath versus rfc1459-proto, which is entirely positional.

from ircv3-specifications.

Would the chunks being out of order be an issue for the JSON-specific stuff, or just the RFC1459?

Because JSON should not care what order things are in inside the object, since it's all named. and translating that back to RFC1459 wouldn't be hard no matter what order it was in.

Example in JSON: {'content': ['+', 'name1,name2'], 'source': 'shawn!test@host', 'command': 'MONITOR'}

And it should be ordered as :source command content
So we simply do that before sending it out to RFC1459 clients.
:shawn!test@host MONITOR + name1,name2

(Obviously MONITOR wasn't a great example for this because you wont be sending the same MONITOR line to both JSON and RFC1459 clients, I used it because it gave an example of how 'content' was split into an array.)

Edit :: The order of things are generally going to be the same, source command destination content, with destination being omitted for some things like MONITOR, etc. So I guess what I'm trying to say is that insuring it is in the proper order before being sent to RFC1459 clients doesn't seem like it would be an issue.

from ircv3-specifications.

@shawn-smith The issue wasn't whether the outermost part is an array, it's whether the content is array or string. I think it should always be array even if it's a single string, otherwise it looked like you wanted to have an array in the form of a space-delimited string or something.

from ircv3-specifications.

Ahh! I see now. I'm sorry for that. I'm perfectly fine with content being an array as long as the outermost part is an object so we name things.

{tags: {}, 'source': 'nick!ident@host', 'command': 'TEST', 'content': [ 'Some stuff' ]}
Does this look better?

from ircv3-specifications.

@shawn-smith Yeah, looks pretty good.

from ircv3-specifications.

Just a note that the properties that have no value, ie. the tags property, should be optional to include. No point on having them there if not they're not needed and the client can easily check if it exists or not.

from ircv3-specifications.

@prawnsalad I agree, that's another reason for wanting the outermost part to be an object. Omitting the tags if there aren't any, or in the case of JOIN and MONITOR, removing the destination.

from ircv3-specifications.

Great, we got back to what I wrote in #55 (though with different names). So, 👍 for this

from ircv3-specifications.

@shawn-smith JOIN has a destination: the channel you're joining.

from ircv3-specifications.

@kythyria In the example I gave higher up the channel you were joining was in content
{'tags': {}, 'source': 'Shawn!test@host', 'command': 'JOIN', 'content': '#testing'}
Was the example I gave, but I guess the channel could be moved to destination, it doesn't matter to me where it goes.

@aji I'm fine with that.
So now after all the discussion so far we've come up with the names:

tags - the server tags
source - where the message came from
destination - where the message is going
command - the command privmsg/join/etc
args - everything after the command/destination

from was changed to source in the last issue
verb was changed to command in the last issue.
content was changed to args in this issue

from ircv3-specifications.

If shorter names are desired, one might consider src, dest, and cmd.

from ircv3-specifications.

@prestonsumner src, dest, and cmd seem to me like a sort of unpleasant nobody-wins compromise between readability and brevity.

I suggest that we have source command and arguments as the full proper names for these attributes, and then a compression option (maybe even a separate cap?) that abbreviates them as much as possible for bandwidth concerns; s, c, and a, for example.

from ircv3-specifications.

@glyph Would destination also be abbreviated to d? You missed that one in your post.

from ircv3-specifications.

@shawn-smith Right.

The full list would be:

source, destination, command, arguments → s, d, c, a

I should also note that this translation should ideally not be manual. Compression is better implemented by algorithms which do things automatically, like something that looks at a list of keys (the whole point is that one would be able to extend the IRC protocol, right?) and choose the shortest distinct prefix.

from ircv3-specifications.

@glyph Would that mean the shortest distinct prefix for a key would change if a key with the same prefix is later added to the protocol? That seems contrary to the goal of being able to add properties without affecting known ones, unless I'm misunderstanding you.

from ircv3-specifications.

I'm not sure that's entirely correct.

Both the client and the server know what capabilities have been enabled
on the connection, and thus the keys that may be sent for each command.

The extensibility comes from moving away from position being important,
once that's done multiple capabilities can extend a command without
clashing with each other (as long as they don't try and use the same key).

Only sending keys that are enabled by a cap will also reduce bandwidth.

On 11/03/14 18:11, prestonsumner wrote:

@glyph https://github.com/glyph Would that mean the shortest
distinct prefix for a key would change if a key with the same prefix
is later added to the protocol? That seems contrary to the goal of
being able to add properties without affecting known ones, unless I'm
misunderstanding you.

—
Reply to this email directly or view it on GitHub
#57 (comment).

from ircv3-specifications.

Just wrap the socket into zip stream, if you wish compression...

2014-03-11 18:11 GMT+00:00 prestonsumner [email protected]:

@glyph https://github.com/glyph Would that mean the shortest distinct
prefix for a key would change if a key with the same prefix is later added
to the protocol? That seems contrary to the goal of being able to add
properties without affecting known ones, unless I'm misunderstanding you.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/57#issuecomment-37329635
.

from ircv3-specifications.

Let me be clear about the intent of my comments about compression; I'm not saying that my proposed compression strategy is necessarily the best one. My point is that if we choose some clear, whole words with easy to read and understand names, the concern of bandwidth savings may be put off to a separate discussion. If we try to come up with names that are both as short as possible and also as clear as possible, we'll end up with names which are neither. So the initial discussion here should focus on clarity and ease of understanding for implementors, and once we have that we can discuss bandwidth costs.

from ircv3-specifications.

@prestonsumner It would, but that doesn't necessarily mean it's an obstacle. Some forms of binary XML get increased coding efficiency if the two ends can agree on the schema in use, but work without it, for instance. And in principle you could exchange a dictionary at connection start or something.

Where it gets fun is that if we have "MONITOR", "MOTD", and "METADATA" in our list of symbols to shorten, they could be "MO", "MOT", and "M" respectively without being a clash.

from ircv3-specifications.

Let me interject here. We want to minimize the amount of code that has to change to process the new wire format. The initial proposal, or a modification thereof, is the best avenue for this. Named arguments don't make sense since we then have to name everything and it reeks of overspecification.

One suggestion, mentioned on IRC, is that the source come presplit if it exists, with the resulting format being:

[tags={}, [nick, ident=null, host=null] OR [server] OR null, command, params=[]]

(with the name=value bits meaning 'default value if none').

Does anyone have an argument for making the message format an object rather than an array that isn't "lol properties are easier than positional elements"?

from ircv3-specifications.

For the outermost level (source/command/args): Positional elements would be fine, as there are always just four, and extra metadata can easily go in tags or at the end (after args array).

For arguments/parameters themselves: Allows clearly marking a "dest" parameter, is easier to extend with new parameters in case of multiple caps that add their own parameters (compare: the mess that is Unreal's s2s NICK command). Backwards compatibility not a huge issue, as a plain array already allows messages incompatible with old framing (e.g. ["foo bar", "baz"]) so this won't make it worse.

from ircv3-specifications.

@grawity It seems cleaner to have the dest parameter be outside the args, but I can see tremendous amounts of silliness stemming from actually calling it destination, not to mention people complaining about the bandwidth usage from setting it to null for things that genuinely don't have a destination.

That's a thought: Should numerics have the command actually be a number?

from ircv3-specifications.

For the moment, we're not trying to do anything more than stuff the existing protocol into JSON with as few modifications as possible while still being sensible. This is part of why there's no 'dest' element in the array: it's in params just like it would be if we were parsing rfc1459-proto.

@kythyria Numerics can't be numbers; they're not octal and numerics below 100 start with 0.

from ircv3-specifications.

@Aerdan What's the logic behind stuffing the existing protocol into JSON without actually taking advantage of JSON itself? Since parsers will have to adapt to JSON in what ever structure is decided upon (if any), I don't see why we would need to fall short in a first implementation?

Reading back over some of the previous messages it sounds like some people are thinking of generating the JSON string manually instead of using an existing library - which I really hope I'm wrong on as this is just asking for invalid JSON strings being thrown around. The order of a JSON object is irrelevant and IRC parsers will not have to deal with this.

I can understand using a simple array for the most common parts of an IRC line as this would keep bandwidth down but that would be the only advantage. But parts that are optional should not always be in the in JSON, such as tags: {}. No tags? Don't include it as there is no point. JSON libraries can easily check if a JSON objects property exists or not.

from ircv3-specifications.

@prawnsalad This is not a parsing issue, this is a "clients would have to adopt a new codepath to handle any changes which further distance the new wire format from the old one" issue, which the simple array avoids.

As to 'keeping bandwidth down', which is shorter?

[{},null,"PRIVMSG",["#foo","hello world"]]

or

{"command":"PRIVMSG","destination":"#foo","params":["hello world"]}

or even

{"c":"PRIVMSG","p":["#foo","hello world"]}

The object form only gets worse the more you stuff into it, even with the unreadably short version (which only produces a net savings if the command parameters are unaltered).

from ircv3-specifications.

The basic set of tags, source, destination, message type, payload, will probably never change no matter what happens to the protocol. IRC considers destination to be part of the payload, while other systems merge payload and tags in some way.

So there's no reason to use an object over an array, really, because the set of members is unlikely to change and in the most popular language where JSON objects become language objects (Javascript) it's the work of one line to convert one to the other if it really bothers you. Thus, may as well adopt the shorter and easier to read form.

@prawnsalad Why on earth would you generate the JSON manually. Unless you were using C, I guess, and couldn't stand the thought of anything more complicated than fprintf.

from ircv3-specifications.

Clients may convert new wire format {"source":"nick!u@h", "command": "PRIVMSG", "params": ["#channel", "hello world"]} (or with separate
destination, whatever...) to an array similar to [{}, "nick!u@h", "PRIVMSG", ["#channel", "hello world"]] if they wish so and then use old
RFC1459 code path.
Likewise, when sending stuff, client are free to make the message in any
internal representation they want (e.g. [{},null,"PRIVMSG",["#foo","hello world"] using old code path), then convert it to new wire format, and send
it.

2014-03-14 15:01 GMT+00:00 Kiyoshi Aman [email protected]:

@prawnsalad https://github.com/prawnsalad This is not a parsing issue,
this is a "clients would have to adopt a new codepath to handle any changes
which further distance the new wire format from the old one" issue, which
the simple array avoids.

As to 'keeping bandwidth down', which is shorter?

[{},null,"PRIVMSG",["#foo","hello world"]

or

{"command":"PRIVMSG","destination":"#foo","params":"hello world"}

or even

{"c":"PRIVMSG","d":"#foo","p":"hello world"}

The object form only gets worse the more you stuff into it, even with the
unreadably short version.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/57#issuecomment-37656469
.

from ircv3-specifications.

Or we could just use the array format to begin with. There is no benefit to using objects here. We have an avenue of expansion already, which is the only justification for objects that makes sense. I already outlined why bandwidth is a poor argument, particularly since you get a less-readable result if you opt to compress property names in order to outperform the array format.

And... conversion? Really? The array format requires less work all around and if you really need to remind yourself what each element represents, well, that's what comments are for.

from ircv3-specifications.

@Aerdan I'm not arguing for the bandwidth argument here, that's a non-issue for shaving a small number of characters here but I mentioned it as many people here believes it is.

What I am saying though is that if the RFC1459 format was to be converted to JSON but only throwing the values into an array, what is the point? Other than the ability to include spaces in any parameter there is no huge advantage other than saying "hey, it's unextendable JSON now".

If you look at the examples I had given further up in this thread I had suggested using an array format similar to how you suggest but with named parameters and tags. It's a nice mix of RFC1459 and named properties in the JSON object IMO.

from ircv3-specifications.

But it is extensible, in a way that is compatible with IRCv3.2 message tags. There is no need for other means of extending the wire format at this point in time, and if one is needed later we will discuss it then. Fretting about it now, and insisting that objects be involved beyond just for tags, is completely pointless and reeks of bikeshedding.

from ircv3-specifications.

But it is extensible
[...]
There is no need for other means of extending the wire format at this point in time, and if one is needed later we will discuss it then.

Or it could be kept more open than you propose so we don't end up with a second version of a JSON format in future.

What is it you find that doesn't work with this format that incorporates many of the views mentioned in this thread:
["source", "command", {named: params}, {named: tags}]

1. The optional data at the end so it doesn't get filled with pointless nulls when there is no data to include.
2. Every single IRC line will include a command, so that does not need to be a named property.
3. Clients and servers do not need to worry about parameter ordering or extraneous parameters, though they may do so if they still wish.
4. IRC commands can actually be extended without worry of ordering or other servers already using a parameter position (An issue I remember you having some time ago with the MONITOR command IIRC)

from ircv3-specifications.

@prawnsalad The vast majority of messages that are placed onto the wire have a sender, so it's actually a false economy to make that completely disappear unless you're paying far more for inbound bandwidth than outbound. So overall it would be far more efficient to have

[null, "command", {"named": "params"}, {"named": "tags"}]
["[email protected]", "command",{"named": "params"}, {"named": "tags"}]

than

["command", {"named": "params"}, {"named": "tags"}]
["[email protected]", "command",{"named": "params"}, {"sender":"[email protected]","named": "tags"}]

Indeed, the first pair is 137 bytes, while the second is 161. And for many messages the second line will be sent many times (any message sent to a channel, for instance).

And finally, how do you distinguish between a message that has only parameters and one that has only tags?

from ircv3-specifications.

The only thing I would offer is in response to the goal of minimizing the amount of code that must change. Whichever way the format is designed, the fact it's JSON means people will use a JSON parsing library regardless.

from ircv3-specifications.

How about keeping the existing parameters (RFC 1459 and others) in an array but allowing an object with named parameters to be added later, for extensibility?

[{},"foo","FROB",["#chan","+x"],{"named_param": "baz"}]

from ircv3-specifications.

@prestonsumner Importing a library is only an additional 2-3 lines. Maybe 4-5 more for mapping the array format I suggested into whatever internal structure the client uses.

The problem comes in when e.g. the target/destination is separated out from the payload, as the client then has to check whether the command has one and then frobnicate the parameters to put it where it belongs before continuing to work from the internal structure.

@grawity This is the most sensible approach, and would be what I'd suggest later if we actually needed to extend the protocol format (which I rather doubt).

from ircv3-specifications.

@prestonsumner +1

@kythyria Yes I'd updated the prefix after I'd sent the message - must not have have done it fast enough :) But you're first corrected example got that how I intended.

If a message has no parameters but does include tags, then it is perfectly fine to include a null or empty {} object for the parameters as that wouldn't happen too often I don't expect. The idea is purely just to reduce the number of pointless nulls or empty objects being sent. The only difference with the last array value (named tags in this case) is that this can be completely omitted if not in use.

from ircv3-specifications.

@Aerdan It's always possible; just look at extended-join. If two caps add fields to an existing command, I wouldn't like to end up with something like Unreal's NICKv2+NICKIP+VHP+... combinations of the s2s command, which really adds complexity.

And yes, there are tags, but they aren't always a good fit. They're a bit more to the side of metadata than data.

(Idly wondering about modifying the rfc1457 framing to urlencode parameters instead of using the trailer hack, and to allow for key=value parameters...)

from ircv3-specifications.

extended-join is a special case and only exists in the way that it does because IRCv3.1 did not have message tags; if it did, we'd have put an account tag on it. Moot.

The only reason to have a non-tags mechanism for expansion would be if we for some reason wanted JSON-only extensions, which obviously would not be useful for or relevant to rfc1459-proto clients. I really can't imagine us having any of those any time soon, given the need to continue supporting rfc1459-proto clients for the foreseeable future.

from ircv3-specifications.

I really can't imagine us having any of those any time soon, given the need to continue supporting rfc1459-proto clients for the foreseeable future.

On the other hand I don't see how it is different from introducing @tag. Old clients simply could not use any new caps that add tags, but remained compatible with all existing commands and caps. Same here.

from ircv3-specifications.

I fail to see how this
[{},null,"PRIVMSG",["#foo","hello world"]]
Is any different than this:
{"command":"PRIVMSG","destination":"#foo","params":["hello world"]}

In respects to this reasoning to use the first one: "clients would have to adopt a new codepath to handle any changes which further distance the new wire format from the old one"

Either way you're going to have to adopt a new standard if you wish to use the JSON stuff.

Using an object over an array is better because:

• You can name the tokens
• You can add new tokens for future things without worrying if the client understands them or not, if they don't they will just ignore it
• There is zero difference between this and an array other than the fact you can name things, and things don't have to be given in a forced order.

On the topic of "If we use an object things might not be in the proper order.":

This isn't really an issue either because it's not hard to put those into the correct order, source, then command, then destination, and finally params/arguments.

As for bandwidth, this is what compression is for, there's no need to compress the names ourselves.

from ircv3-specifications.

1. We don't need named tokens.
2. We have tags for such future expansion already.
3. Pulling the destination out means every single command with one (PRIVMSG, MODE, NOTICE, numerics, ...) has to have their arguments modified to put the destination back in the proper place or else clients have to modify every single handler for those commands to take an optional destination argument. Both of these are more invasive than the switch to JSON-proto itself is going to be.

from ircv3-specifications.

@Aerdan The original example given by @kaniini had the destination pulled out.
[{"tag1": "a", "tag2": "b"}, "[email protected]", "PRIVMSG", "#ircv3", "hello"]\r\n

I was simply following his outline and using an object instead of an array.

I'm not sure what you mean by pulling the destination out means we need to modify the arguments of every command that uses one.

If you have this: {'tags': {}, 'source': 'nick!ident@host', 'command': 'PRIVMSG', 'destination':'#test', 'params': [ 'Some stuff' ]}

Turning it into this is minimal effort at best: @tags :nick!ident@host PRIVMSG #test :Some stuff

And it's not an 'optional' destination. The destination would still be required, it's just being sent as 'json[destination]' instead of 'json[params][1]'

Edit :: Okay I see the issue you meant with destination, but that has nothing to do with using an object. It would be just as easy to include the destination in the last parameter with an object as it would with an array.
Eg: {'tags': {}, 'source': 'nick!ident@host', 'command': 'PRIVMSG', 'params': '#test :Some stuff'}

from ircv3-specifications.

Pulling values out by magic indexes would be an unusual use of JSON. This is unordered data that only has an order in the current format for parsing reasons. People will just #define the indexes into readable tokens and treat it like a key-value container anyway.

from ircv3-specifications.

@shawn-smith Not really, that was having one entry in the outer array for each entry in the argument list. Whether [3] is actually a destination is unspecified by his proposal, since he's going for minimum difference between RFC encapsulation and JSON.

from ircv3-specifications.

@kythyria Then the actual size of the array would differ depending on what kind of a line you were sending.

Eg: A PART or a QUIT has less arguments a PRIVMSG.

If you simply want to send RFC1459 messages over JSON then nobody is going to implement this because

• It's adding nothing beneficial. Just overhead.
• It's identical to RFC1459, but requires you to rewrite your parser for JSON instead of RFC1459
• There's no reason to add it. What we have works and what you're adding changes nothing.

So far the arguments against an object instead of an array seem to be this:

• You'll have to name everything
• You can have things in a different order than RFC1459 allows
• Bandwidth

The first is not an issue, you don't have to name more than a couple of things.
tags - Message tags
source - Where the message came from
destination - Where the message is going
command - What the message is doing
arguments - Information from the command

The second is not an issue either, putting things back into RFC1459 order is not hard.
tags - source - command - destination - arguments

Bandwidth is not an issue, we have compression for a reason. Adding another 50 or so characters for naming things is minimal at best in terms of arguing for bandwidth.

The argument for pulling the destination out has nothing to do with the object-over-array debate, you can just as easily put the destination in the arguments area in an object as you can in an array.

from ircv3-specifications.

May I remind you that the purpose of this exercise is replacing a more-complicated-than-it-looks wire format with one that everybody consistently and reliably implements, thereby moving parsing from the hands of teenage idiots to well-written libraries available for every language anyone would sensibly use.

So what does using an object over an array buy us? Named properties aren't that much more useful than knowing a precise order of elements that won't change ever, and many languages provide utilities such as Python's namedtuple if you really want named properties.

from ircv3-specifications.

May I remind you that the purpose of this exercise is replacing a more-complicated-than-it-looks wire format with one that everybody consistently and reliably implements

This has nothing to do with an object vs array.

So what does using an object over an array buy us?

An object can be expanded upon without breaking existing clients. They'll simply ignore named properties they don't understand, with an array if you want to change the size of the array you may upset clients who are dependent on the size.

Named properties aren't that much more useful than knowing a precise order of elements that won't change ever

You're thinking for the immediate future, not farther down the road. I'm sure whoever originally wrote the RFC1459 spec didn't think we'd want to change that either, with an object you have the OPTION to change this later without breaking existing clients.

and many languages provide utilities such as Python's namedtuple if you really want named properties.

If some people actually want named properties, myself included, what would it hurt by using an object and giving the properties names over using an array and making people use another utility for getting named properties?

from ircv3-specifications.

"What if someone writes a broken client that doesn't handle adding a new element on the end of the array? What then?"

is basically what that post is boiling down to for me, for which the only sensible is "who cares if some moron's client breaks because they insisted on the array always being four elements long?" Are you suggesting that anyone in the working group is stupid enough to insert new elements in the middle of an array?

I'm still not seeing a good argument for objects here, because all of them are basically "but... but... named properties!", as far as I can tell.

from ircv3-specifications.

@Aerdan I know you've always done it, but lets try to leave the 'lol developers are morons' out of the discussion, it's not constructive and is just pure childish. Also to remember, anything that makes development easier is always a plus.

from ircv3-specifications.

There's little reason to use JSON for messages if they're just going to be indexed data. It would essentially be a glorified CSV. At that point, you may as well go all-in with CSV and only deliver JSON in an extensible field:

{ "tag": "value"}, nick!ident@host, PRIVMSG, #test, :Well, this sure is something

from ircv3-specifications.

Can someone let me know when we've gotten back to the core of the original proposal instead of this retardation?

Thanks.

from ircv3-specifications.

To be reopened later (maybe)

from ircv3-specifications.

@shawn Before I forget: One reason against putting named command parameters directly in the root is separation between command-level and protocol-level parameters. That is, those which mean roughly the same thing for all commands and replies, and those which have different meanings for each command (and are dealt with by the command's handler). Like TCP/IP layers, for example. So even if the root was an object, it should still have a sub-object containing the named parameters.

from ircv3-specifications.

May I remind you that the purpose of this exercise is replacing a more-complicated-than-it-looks wire format with one that everybody consistently and reliably implements,

Yes, this is by far the most important thing here. Can we please get this reopened? This is a serious, critical issue in the RFC1459 serialization format which, as I noted before, causes real world security problems all the time.

I was offering my thoughts on objects versus lists before because I do think that objects are generally more sensible, more extensible, and more idiomatic with how JSON is usually used. However, I would be very happy to be overruled on all of those points if we could just get a non-ad-hoc and properly extensible serialization format for IRC messages.

thereby moving parsing from the hands of teenage idiots to well-written libraries available for every language anyone would sensibly use.

This is unnecessary ad-hominem. Sometimes the "teenage idiots" actually implement secure and sane systems using appropriate tools; sometimes very serious-looking grown-ups with multiple degrees write garbage code with stack smashers. The problem is that this is too hard for everybody.

from ircv3-specifications.

@glyph JSON discussion has been dropped until IRCv3.2 is released. It'll be brought back up again after that.

from ircv3-specifications.

@shawn-smith Thanks for clearing up the timeline.

from ircv3-specifications.

Shawn-Smith commented 19 hours ago:
@glyph JSON discussion has been dropped until IRCv3.2 is released. It'll be brought back up again after that.

This is a complete joke, now we have people who are not on the core group defining timelines.

Guys, IRCv3 is dead. They've basically (defacto-wise) handed it over to this @shawn-smith guy who doesn't know what he's doing.

from ircv3-specifications.

@kaniini by "they" you mean "you"? I don't think IRCv3 should be closed to someone, just because someone else doesn't like them.

from ircv3-specifications.

@kaniini Several people on IRC gave that as the timeline, I was simply relaying information.

from ircv3-specifications.

@glyph According to the original proposal, it's this or nothing, as this is the only format that will be considered for Atheme's IRCv3 daemon. This working group is a project of Atheme. I, too, welcome any improvement to RFC1459.

from ircv3-specifications.

We are willing to consider other formats, however, the internal object encapsulation proposed is something we are unwilling to do.

If people want semantic data as to field meanings, I propose a vector of key-value pairs, that way ordering is guaranteed and the semantic data is available.

However, this bug is trashed, so I propose doing so on another one. The main thing is, any new format we would want to preserve argument ordering and have any semantic data about each argument inline.

So, for example:

[{source: "nick!user@host"}, {command: "PRIVMSG"}, {dest: "#ircv3"}, {message: "hello world"}]

from ircv3-specifications.

If the intent is simply to replace <SPACE> in RFC1459's format, than the original proposal is simpler. When a new issue is opened, it might be useful to expound on the rationale for the preservation of RFC1459's argument ordering, since the question is likely to come up.

from ircv3-specifications.

I suggest that we use DSON instead.

so such "account" is "tester" wow next "tester!~tester@unaffiliated/tester" next "PRIVMSG" next so "#ircv3" next "Hello, world!" many many

^{Disclaimer: this is a joke, do not take it seriously.}

from ircv3-specifications.

DSON 👍

from ircv3-specifications.