New protocol about ssl-refbox HOT 4 CLOSED

I'd try to avoid protobuffer at least in the beginning. The so called legacy packet can be extended without breaking compatibility, as long as the current information is left as it is now, at the beginning of the packet, adding more bytes only to the trailing. This would give teams the option to capture the new info or not, and give more time to work on the pros/cons of using protobuffer.

from ssl-refbox.

Not necessarily true. How do you know teams don't check that the packet is exactly the length they expect it to be? If they do, then an extended packet will prevent their code from ever receiving anything. The problem with avoiding protobuf is that you need some way to encode variable-length content: the yellow card time counters.

In my opinion, we shouldn't be asking whether we can avoid protobuf, we should be asking why we want to. It's a dependency that every team has already figured out how to deal with. There is literally no way a team can possibly have not figured out how to do protobuf!

In any case, I said we should leave the legacy packet exactly as it is and add a protobuf packet stream on a separate multicast group and port number. This actually has better backwards compatibility (as explained in my first two sentences). If teams don't want to upgrade, they don't get the extra data, but then, if a team doesn't want to modify their refbox-receiving code, then it doesn't matter what we do, they'll never get the benefit of that extra data.

from ssl-refbox.

I see your point, but still believe it's a canon to kill a mosquito. 2 bytes at the end (size and checksum) would deal with the variable length and packet integrity. Anyway no need to detail that too much, as keeping the backward compatibility is more important, and leaves room for exploring options (including the use of protobuf) and allows devs to see how it goes.

cheers

from ssl-refbox.

Packet integrity isn't a problem; both UDP and Ethernet include checksums. However, note that it's not sufficient to just solve the variable-length packet problem (which, again, UDP solves for you, having a total length indicated in the UDP header and indicated by return value of recv()). The more relevant point is that different pieces of the packet are variable-length; at the very least, we have two separate arrays of yellow card timers, one for each team, both of which are variable-length.

More to the point, though, is that while it's still fairly simple, it's still more work (for every team) to handle this. Personally I'm OK hand-writing the parsing code for these packets. That's not the point. You can hand-encode and hand-parse any imaginable data structure no matter how complex, and you will nearly always get a more efficient encoding than you would manage using a tool. The point is, hand-writing the parsing code is more work. Protobuf has already solved this, so IMO the proper questions to ask are "what benefits do we get by using protobuf?" and "how much does it cost?".

To answer the first question, we get two basic benefits. First, we get the ability to avoid writing a bunch of code, by using a tool that everyone is already using. The second benefit is easier extensibility. Protobuf gives us the option to include things like optional fields. Of course that can be done by hand—everything can—but the more you add, the more work it is to do it by hand. Protobuf also guarantees a sane migration path to new protocols with new data added; hand-written parsers can be written to have a migration path, but there's no systematic guarantee that they all will be written that way.

To answer the second question, the cost is very low. Today, each protobuf packet uses up 22 (Ethernet header) + 20 (IPv4 header) + 8 (UDP header) + 6 (packet payload) + 4 (Ethernet CRC) + 12 (Ethernet interframe gap) = 72 bytes on the wire. According to my calculations, a Protobuf encoding of the current refbox payload would use a total of 11 bytes, or 5 more than the current 6 byte payload. This is an increase of 5 ÷ 72 = 6.9% in refbox bandwidth usage, and much less than that in terms of total bandwidth usage given that the vast majority is eaten by SSL-Vision—in fact, this would be a fraction of the increased bandwidth usage from SSL-Vision due to adding a sixth robot!

I absolutely agree 100% with keeping backwards compatibility. That's why I suggest that any new packets we deliver—whether in protobuf format or anything else—be in addition to, not instead of, the existing data stream (at least for a few years) and be sent on a different multicast group to a different port number. Thus our discussion here—we all agree we won't touch the existing stream, but isn't "exploring options" exactly what we're doing?

from ssl-refbox.