XML parser about trial.protocol HOT 11 CLOSED

That is a good question for which I have no good answer.

First of all, I think that a push-based XML parser is a very good idea. Actually, it was the XmlTextReader/Writer classes that first inspired me to follow the push-parser track.

However, I am undecided whether such an XML parser should be part of trial.protocol. While I have not documented this, I designed trial.protocol around formats that maps directly to basic C++ data types and containers (e.g. JSON, BinToken, MsgPack, CBOR, UBJSON.) XML is different. It is basically sequence of nested tags and (string) payloads. Mapping an XML document to C++ types requires some kind of schema, whether handwritten as in XML-RPC or general with the use of XML Schema.

from trial.protocol.

First of all, I think that a push-based XML parser is a very good idea. Actually, it was the XmlTextReader/Writer classes that first inspired me to follow the push-parser track.

Qt has one of the more complete XML libraries I've seen:

• http://doc.qt.io/qt-5/qtxml-index.html
• http://doc.qt.io/qt-5/qtxmlpatterns-index.html

And its base are two classes quite similar to a pull parser:

• http://doc.qt.io/qt-5/qxmlstreamreader.html
• http://doc.qt.io/qt-5/qxmlstreamwriter.html

I may have a bias towards this parser model and maybe we'd like a different choice for XML, but I felt I should share this info here.

At least to parse JSON without memory requirements of a "DOM parser", there were several situations on the code I'm writing that the lack of callbacks (or, more explicitly, the property of not stealing the control flow away from me) allowed me to properly validate the document with simplified code in several places. A similar situation happened with HTTP when I thrown NodeJS parser away. XML is different enough to possibly change the situation, but I want to make this point anyway.

However, I am undecided whether such an XML parser should be part of trial.protocol. [...]

I see. Okay then. I'll keep that in mind.

BinToken

I found no reference for this protocol on Google. Could you put a link to it in Trial.Protocol README.md?

MsgPack, CBOR, UBJSON

Nice.

CBOR was based on MsgPack. Maybe we don't need MsgPack after all.

I like both, CBOR and UBJSON.

from trial.protocol.

I designed bintoken after I was unable to find a binary protocol where we can skip over unknown tokens.

from trial.protocol.

I forgot to add that I consider other schema/IDL-based protocols, such as Apache Thrift or Google Protocol Buffers, to be in the same category as XML.

from trial.protocol.

Hey. Would Toml be in the targets as well?

I'm currently using it in a project and it maps pretty well to JSON. I'm using cpptoml, but the dynamic::variable is much more convenient to use than the builtin solution from cpptoml.

from trial.protocol.

TOML looks like a mixture between .INI and YAML.

You could start by developing an external TOML <-> dynamic::variable converter. This does not have to support incremental processing from the start.

Looking briefly at the TOML specification, I can see at least three challenges.

1. Date-time: dynamic::variable has no direct support for compounded types like date-time. These will have to be stored as strings or as an array of elements.
2. Comments: dynamic::variable has no support for comments (or white-spaces.) Either comments have to be dropped during parsing, or they must be stored as a type-value pair -- e.g. in JSON syntax: [\"comment\", \"# Here is my comment\"].
3. Tables: From my initial reading, it looks like tables and array-of-tables requires some context-specific parsing, making it more difficult to build and incremental parser.

The lack of date-time support in dynamic::variable is quite deliberate, because there is no agreement on date-time formats. For example, one protocol I worked on had 73 different date-time formats (e.g. year, year-month, year-month-day, year-month-weekday, month, month-day, etc.)

from trial.protocol.

I agree with #1. If I call JSON.stringify with some date, this is the solution chosen.

> JSON.stringify({a: new Date()})
'{"a":"2018-02-13T14:02:12.413Z"}'

#2 is not really a problem because we should just ignore comments.

#3 is a challenge. This format would benefit from some sort of partial parsing as it allows you to build tools that would find the proper point in the file to insert new fields (e.g. tools like npm install <name> which will modify the config file).

For my use, I only need the dynamic::variable integration.

Can I add a separate open issue so I have a reminder of this feature?

from trial.protocol.

Please do (especially since this issue was closed a long time ago :)

from trial.protocol.

First of all, I think that a push-based XML parser is a very good idea. Actually, it was the XmlTextReader/Writer classes that first inspired me to follow the push-parser track.

Sorry about the necro-bumping, but now it occurred to me that maybe you wrote "push parser" when you meant "pull parser". I mean, in the rest of the sentence, you confirm that you're following the "push-parser model" because this inspiration, but here you're writing a pull parser. Could you clarify if it was a typo?

from trial.protocol.

Yes, I meant to say pull parser in the above.

from trial.protocol.

Okay, so... just to you know, half of my previous comment was to actually convince you about it 😅😅😅.

Anyway, given dynamic::variable and the rest of the abstractions, I agree that XML parsing belongs to another library.

from trial.protocol.