Hi, I really like your work with comms and great documentation about how to use it and how it works. I want to use it for handling messages in embedded environment, but I've encountered one problem. Your library explicitly state MsgSizeLayer which obtains data about size of message from some values inside the message.
Now, let's talk about messages which I have to send/receive:
They are something similar to ByteStuffed messages from this slide 9.
So I have several differences in messages:

1. There is no explicit size known at the start of the message, the message is finished if some suffix bytes are encountered in byte stream.
2. Message suffix consist of two bytes, and if in original payload we encounter byte with same start value it should be stuffed - changed with two bytes, for example if suffix is 0xFF 0xFE we must change all bytes in payload from 0xFF to 0xFF 0xFA - so we can differ - is this really end of message, or this is only single byte from payload.
3. Because of this bytestuffing - in the worst case - obtained message can be x2 the size of the original, and I can't store the whole message before unstuffing it - I should unstuff elements and store them "on fly" - when message bytes arriving.
4. Message couldn't be obtained in single piece it come in chunks which are 2-8 times smaller than the whole thing.
5. When sending message - byte stuffing and suffixing must be applied as well.
6. Message have prefix which can differ in second byte, for example 0xFF 0x01 and 0xFF 0x02 which mean different "type" of message - is it incoming command or is it answer to my command.
7. This is not COBS, I can't use it, or change protocol to have explicit size field.

Other than that - I have as usual id, sync, payload, and crc elements inside the message.

I've encounter messaging task for the first time, really new to the field of messaging, and doesn't want to handle the problem from scratch, so I've tried to find some building blocks solutions for that problem and encountered your library, which looks very promising to me, but I'm worried if it is possible to handle such types of messages using your library, or it doesn't fit well in terms of your library.

Could you estimate what should be done to adapt your library for handling such byte stuffed/size-less messages and which steps it will require?
I would really appreciate any help.

Just out of interest, I've tried compiling this codebase with Clang.

There are some interesting diagnostics, that you might want to check out.

Using clang:

sudo apt-get install clang                     # clang
export CC=clang; export CXX=clang++

mkdir build
cd    build
cmake ..

To get clang to work, add another elseif here with something along these lines:

elseif (CMAKE_CXX_COMPILER_ID MATCHES "Clang")
    set (extra_flags_list
        "--std=c++11"
        # "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wctor-dtor-privacy"
        # "-Wmissing-include-dirs"
        # "-Woverloaded-virtual" "-Wredundant-decls" "-Wshadow" "-Wundef" "-Wunused"
        # "-Wno-unknown-pragmas" "-fdiagnostics-show-option"

	## "-Wnoexcept" "-Wlogical-op" "-Wstrict-null-sentinel" # not supported under clang
    )
    
    if (NOT CC_NO_WARN_AS_ERR)
        list (APPEND extra_flags_list "-Werror")
    endif ()
    
    string(REPLACE ";" " " extra_flags "${extra_flags_list}")
    set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${extra_flags}")
    
    if (CC_STATIC_RUNTIME)
        SET(CMAKE_EXE_LINKER_FLAGS  "${CMAKE_EXE_LINKER_FLAGS} -static-libstdc++ -static-libgcc")
    endif ()

As you can see, I've commented out a lot of warnings above, which can be put back later...

Even without these warnings, clang detects a lot of things... such as...

• comms/include/comms/util/alloc.h line 283

-        return iter->alloc<TObj>(std::forward<TArgs>(args)...);
+        return iter->template alloc<TObj>(std::forward<TArgs>(args)...);

• comms/test/CommsTestCommon.h line 44

-    using Base::Message;
+    using typename Base::Message;

etc.
You'll notice that Clang is quite strict (or "restrictive"). (Example regarding this line in embxx, ...)
Perhaps some things are somewhat more manageable, if c++14 is allowed...

I have tried to copy some data into a data field. (ArrayView). But since all member functions are const, I cannot change the underlying data.

Now there are two possibilities: Either I have missed something or it is an error that the ArrayView.h#L77
Iterator is marked as a const_iterator.

For me this is a bit counterintuitive, because an iterator of some data should be modifiable.

First of all thanks for the awesome library and a nice read!

Trying to compile using the latest MSVC (19.15.26729.0). Getting an error:

FieldsTestRunner.cpp
c:\test\comms_champion\build\comms\test......\comms\test\Fields.th(32): error C2338: Invalid function presence detection

Any ideas?

Hello!

I'm a big fan of this library. I've successfully used it in my project. However, I ran into one problem: how to read only one, known at compile time, message type, using ProtocolStack? In the documentation it is written, that messages can be handled using dispatch method of Message. It is very useful, I know, because of polymorphic behaviour, but brings unnecessary overhead in my case.

In the protocol I'm working at, I can send to server some Command, and then it must send an Answer. I know at compile time, which type of Answer it should be, because It comes from this protocol specification. It would be useful, in my opinion, to write code like this:

ConcreteAnswerMsg concreteAnswerMsg;
const auto errorStatus = protocolStack.tryRead(inputBuffer, concreteAnswerMsg);
if(errorStatus == Success)
{
    // good, input buffer contains message needed by us
    // read finished successfully, so call the success callback
    assert(concreteAnswerMsg.field_data().valid());
    const auto& data = concreteAnswerMsg.field_data().value();
    callback(data);
}

instead of writing code, like at the moment:

MessagePtr msgPtr;
...
const auto errorStatus = protocolStack.read(msgPtr, ...);
if(errorStatus == Success)
{
    assert(msgPtr);
    msgPtr->dispatch(handler);
    // message handler has to accept only `ConcreteAnswerMsg` type (by calling the success callback), 
    // and reject all others(by calling the error callback)
}

Hi,

Thank you for you great job, the library design is very smart and we are projecting to use it in commercial projects...

Just one issue we cannot solve with Variant and VariantHandler (following documentation exemples):

using MyFieldBase = comms::Field<comms::option::LittleEndian>;
using Value1 = comms::field::IntValue<MyFieldBase, std::uint8_t>;
using Value2 = comms::field::IntValue<MyFieldBase, std::uint32_t>;
using Value3 = comms::field::String<MyFieldBase,comms::option::SequenceSizeFieldPrefix<comms::field::IntValue<MyFieldBase,std::uint8_t>>>;

enum class KeyId : std::uint8_t
{
    Key1,
    Key2,
    Key3,
    NumOfValues
};

template <KeyId TId>
using KeyField = comms::field::EnumValue<MyFieldBase,KeyId,comms::option::DefaultNumValue<(int)TId>,comms::option::ValidNumValueRange<(int)TId, (int)TId>,comms::option::FailOnInvalid<>>;

using Key1 = KeyField<KeyId::Key1>;
using Key2 = KeyField<KeyId::Key2>;
using Key3 = KeyField<KeyId::Key3>;

using Property1 = comms::field::Bundle<MyFieldBase, std::tuple<Key1, Value1> >;
using Property2 = comms::field::Bundle<MyFieldBase, std::tuple<Key2, Value2> >;
using Property3 = comms::field::Bundle<MyFieldBase, std::tuple<Key3, Value3> >;

struct MyVariant
: public comms::field::Variant<MyFieldBase,std::tuple<Property1,Property2,Property3>,comms::option::DefaultVariantIndex<0>>
{
    COMMS_VARIANT_MEMBERS_ACCESS(prop1, prop2, prop3);
};

struct MyVariantHandler
{
    void operator()(Property1& prop) {}
    void operator()(Property2& prop) {}
    void operator()(Property3& prop) {}
};

void handleVariant(MyVariant& var)
{
    var.currentFieldExec(MyVariantHandler());
}

This code doesn't compile with the following error :
error: no member named 'operator()' in 'MyVariantHandler' func_.template operator()<TIdx>(*(reinterpret_cast<TField*>(storage_)))

I'm using Xcode with LLVM 9.0, clang-900.0.39.2.
I have also try with gcc 5.4.1 with the same error.
It seems the template resolution doesn't match, but how to implement MyVariantHandler to make this work?

Regards,

https://github.com/arobenko/comms_champion/blob/9187a4e39f19767d53e6206ad4bb29ff6a523a11/comms_champion/app/cc_view/src/widget/ProtocolsStackWidget.cpp#L179

should be replaced with item->isSelected()

https://doc.qt.io/qt-5/qlistwidget-obsolete.html#isItemSelected

Thanks for such a great system, I've been looking at it for quite a while and I felt really inspired once I started catching on 😄 I'm trying to implement a protocol which defines its basic stack like so:

I think I have the basic protocol stack layer properly built to handle this (SyncPrefix -> MsgId -> Size -> Payload) but then the parameter value can be any one of several data types. The primitive ones I think I've got down pat (basic ints, a string, etc). But parameters can come/go in any order and be any one of those data types or a group of them (i.e. the bottom right section of that table repeated all over again for the Value field). I've created Bundle fields for each specific pairing of param/length/data type, but I don't think there is a provision for a bundled field to have its own length, unlike ArrayList (if I understood ArrayList correctly). I saw Variant fields, but unless I'm wrong, for each message with, say, 3 parameters is going to look like this:

// header.h
using MyFieldBase = comms::Field<comms::options::LittleEndian>
using UInt16Field = comms::field::IntValue<MyFieldBase, uint16_t>;

template <int Num>
using ParameterIDField =
    comms::field::IntValue<MyFieldBase,
                           uint16_t,
                           comms::option::DefaultNumValue<Num>,
                           comms::option::ValidNumValueRange<Num, Num>,
                           comms::option::FailOnInvalid<>>;

template <int Num, typename T>
using Parameter =
    comms::field::Bundle<MyFieldBase,
                         std::tuple<ParameterIDField<Num>, UInt16Field, T>>;

template <int Num>
using EmptyParameter =
    comms::field::Bundle<MyFieldBase,
                         std::tuple<ParameterIDField<Num>, comms::field::IntValue<MyFieldBase, uint16_t>>>;

enum MsgIds : uint16_t
{
    MsgId_Message1 = 0x1234
};


// Message1.h
enum Message1ParameterIDs : uint16_t
{
    X = 0,       /* u16 */
    Y,           /* u16 */
    Button = 3   /* empty value, assume "true" aka "pressed" if present */
};

using Message1ParamVariant =
    comms::field::Variant<MyFieldBase,
                          std::tuple<Parameter<Message1ParameterIDs::X, UInt16Field>,
                                     Parameter<Message1ParameterIDs::Y, UInt16Field>,
                                     EmptyParameter<Message1ParameterIDs::Button>>>;

struct Message1Parameters : public Message1ParamVariant
{
public:
    COMMS_VARIANT_MEMBERS_ACCESS_NOTEMPLATE(X, Y, Button);
};

struct Message1Fields
{
    using PropertiesList =
        comms::field::ArrayList<MyFieldBase, Message1ParamVariant>;

    using All = std::tuple<PropertiesList>;
};

template <typename TBase>
class Message1
    : public comms::MessageBase<TBase,
                                comms::option::StaticNumIdImpl<MsgId_Message1>,
                                comms::option::FieldsImpl<Message1Fields::All>,
                                comms::option::MsgType<Message1<TBase>>,
                                comms::option::HasName>
{
public:
    // COMMS_MSG_FIELDS_ACCESS()     /* what do I even put here to get X, Y, and Button? */
};

My other thought was just run the entire message payload layer through another protocol stack definition (MsgId -> Size -> Payload) and so on and so forth for any subgroups of parameters within, but 1) that'd be several new protocol stacks for each message I think?, 2) I don't see an example of how to do that, 3) not sure where to even begin to try to wrap my head around fitting that in (parameter dispatch?) during message dispatch

am I trying too hard to look for something to avoid boilerplate code in every message? The example I wrote above already looks pretty hefty just for a single message, and I've got...oh, about 100 messages to implement 😬

Thoughts?