gobysoft / goby Goto Github PK

Warnings to become errors in future releases, using c++17.

[libprotobuf WARNING google/protobuf/compiler/parser.cc:547] No syntax specified for the proto file: dccl/protobuf/option_extensions.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.)

If the GobyMOOSApp config has optional sub messages, e.g.

message AppConfig
{
  optional GobyMOOSAppConfig common = 1; 
  optional SubMessageType submessage = 12;
}
message SubMessageType
{
  optional int32 foo = 1;
}

the submessages will always get populated, due to the call to fetch_moos_globals. We need to check if any of the subfields of submessage are actually populated by the moos globals, and if not, erase the newly added field.

Merge of pull request #7 introduced the following runtime error in protobuf messages importing from goby/acomms/protobuf/dccl_option_extensions.proto:

./a.out: symbol lookup error: /opt/goby/lib/libgoby_acomms.so.26: undefined symbol: _ZN4goby4util22DynamicProtobufManager5inst_E

You can reproduce the problem using the following script:

# Save as test.sh
cat <<EOF > simple_message.proto 
//simple_message.proto
//Works using this import
//import "dccl/protobuf/option_extensions.proto";

// Using this import creates the problem
import "goby/acomms/protobuf/dccl_option_extensions.proto";

message SomeMessage {
    required int32 some_field=1;
}
EOF

/usr/bin/protoc -I/usr/include -I$PWD --cpp_out=$PWD $PWD/simple_message.proto

echo "int main() { return 0; }" | /usr/bin/c++  simple_message.pb.cc -lgoby_common -lgoby_acomms -lgoby_pb -lgoby_util -ldccl -lprotobuf -x c++ -

./a.out && echo "OK"

The script runs fine using the first protobuf import line import "dccl/protobuf/option_extensions.proto":

$./test.sh
OK

However the 2nd import line produces the following:

$./test.sh
./a.out: symbol lookup error: /opt/goby/lib/libgoby_acomms.so.26: undefined symbol: _ZN4goby4util22DynamicProtobufManager5inst_E

Reverting back to 3cf85e9 fixes the problem.

These two classes started life as copies of each other (currently I think they are almost identical), so going forward Goby should just use the version in DCCL to avoid duplication of fixes.

[ 2018-Sep-07 20:19:57.375855 ]               {benthosatm900::fsm}: D: ReceiveData
[ 2018-Sep-07 20:19:57.375909 ]               {goby::acomms::modemdriver::in::1}: D: a7 b4 5d a8  9c e4 43 2a    27 3c 7e bc  d3 f6 01 0d
[ 2018-Sep-07 20:19:57.375954 ]               {goby::acomms::modemdriver::in::1}: D: 
[ 2018-Sep-07 20:19:57.375987 ]               {goby::acomms::modemdriver::in::1}: D: 
[ 2018-Sep-07 20:19:57.376026 ]               {goby::acomms::modemdriver::in::1}: D: Source:000  Destination:001
[ 2018-Sep-07 20:19:57.376120 ]: (Warning): Invalid data received, ignoring. Reason: bad lexical cast: source type value could not be interpreted as target
[ 2018-Sep-07 20:19:57.376163 ]               {benthosatm900::fsm}: D: ~ReceiveData

This appears to be caused by line 155 of benthos_atm_driver_fsm.cpp, which trims away the entire 000.

                boost::trim_left_if(id_str, boost::is_any_of("0"));

I seem unable to split messages across frames for when using the MicroModem driver with a QueueManager/MACManager. This specific case uses a MAC slot w/ rate 1, which is a 192B packet split into 3 64B frames. The encoded message is 71 bytes and it gets refused because it is too big.

Here is the snippet where we see the message get pushed into the queue:

Example [2016-May-27 20:00:34.603099] {goby::acomms::queue::push::1}: D: dslcore.acomms.protobuf.DslTextMessage (16386): attempting to push message (destination: 2)
Example [2016-May-27 20:00:34.603194] {goby::acomms::queue::push::1}: D: pushed to send stack (queue size 2/100)
Example [2016-May-27 20:00:34.603290] {goby::acomms::queue::push::1}: D2: Message: [[DslTextMessage]] header {
Example [2016-May-27 20:00:34.603364] {goby::acomms::queue::push::1}:   src: 1
Example [2016-May-27 20:00:34.603433] {goby::acomms::queue::push::1}:   dst: 2
Example [2016-May-27 20:00:34.603501] {goby::acomms::queue::push::1}:   timestamp: 1464379234601972
Example [2016-May-27 20:00:34.603568] {goby::acomms::queue::push::1}:   msg: 2
Example [2016-May-27 20:00:34.603635] {goby::acomms::queue::push::1}: }
Example [2016-May-27 20:00:34.603701] {goby::acomms::queue::push::1}: message: "HELLO FROM USB0  THIS IS A MUCH LONGER MESSAGE THAT WONT FIT"
Example [2016-May-27 20:00:34.603768] {goby::acomms::queue::push::1}: 
Example [2016-May-27 20:00:34.603849] {goby::acomms::queue::push::1}: D2: Meta: [[QueuedMessageMeta]] non_repeated_size: 71

And here is the portion where goby tries to send it:

Example [2016-May-27 20:01:00.054759] {goby::acomms::amac::1}: D: Cycle order: [ d1/-1@1 d2/-1@1  ]
Example [2016-May-27 20:01:00.055118] {goby::acomms::amac::1}: D: Starting slot: src: 1 dest: -1 time: 1464379260000000 rate: 1 type: DATA slot_seconds: 15 slot_index: 0
Example [2016-May-27 20:01:00.055415] {goby::acomms::modemdriver::out::1}: D: Beginning to initiate transmission.
Example [2016-May-27 20:01:00.055685] {goby::acomms::modemdriver::out::1}: D:   this is a DATA transmission
Example [2016-May-27 20:01:00.056026] {goby::acomms::queue::priority::1}: D2: Finding next sender: [[ModemTransmission]] src: 1
Example [2016-May-27 20:01:00.056265] {goby::acomms::queue::priority::1}: dest: -1
Example [2016-May-27 20:01:00.056474] {goby::acomms::queue::priority::1}: time: 1464379260000000
Example [2016-May-27 20:01:00.056677] {goby::acomms::queue::priority::1}: rate: 1
Example [2016-May-27 20:01:00.056876] {goby::acomms::queue::priority::1}: type: DATA
Example [2016-May-27 20:01:00.057074] {goby::acomms::queue::priority::1}: max_num_frames: 3
Example [2016-May-27 20:01:00.057271] {goby::acomms::queue::priority::1}: max_frame_bytes: 64
Example [2016-May-27 20:01:00.057468] {goby::acomms::queue::priority::1}: frame: ""
Example [2016-May-27 20:01:00.057665] {goby::acomms::queue::priority::1}: slot_seconds: 15
Example [2016-May-27 20:01:00.057863] {goby::acomms::queue::priority::1}: slot_index: 0
Example [2016-May-27 20:01:00.058088] {goby::acomms::queue::priority::1}: D: Starting priority contest
Example [2016-May-27 20:01:00.058294] {goby::acomms::queue::priority::1}:       Requesting 3 frame(s), have 0/64B
Example [2016-May-27 20:01:00.058562] {goby::acomms::queue::priority::1}: D:    dslcore.acomms.protobuf.DslTextMessage next message is too large {71}
Example [2016-May-27 20:01:00.058790] {goby::acomms::queue::priority::1}: D:    all other queues have no messages
Example [2016-May-27 20:01:00.059380] {goby::acomms::queue::priority::1}: D: ending priority contest
Example [2016-May-27 20:01:00.059844] {goby::acomms::queue::out::1}: D: No data found. sending empty message to modem driver.

The rejection notice is from goby/src/acomms/queue/queue.cpp:

    // wrong size
    else if(request_msg.has_max_frame_bytes() &&
            (next_msg.non_repeated_size() > (request_msg.max_frame_bytes() - data.size())))
    {
        glog.is(DEBUG1) && glog << group(parent_->glog_priority_group()) << "\t" << name() << " next message is too large {" << next_msg.non_repeated_size() << "}" << std::endl;
        return false;
    }

I would expect messages to get rejected if they're too big to fit in a transmission packet, but you should be able to span frames without issues. Or am I misunderstanding something here?

With multiple drivers, pAcommsHandler may not read the mail often enough while running ModemDriverBase::startup() and thus the CMOOSApp gives the following warning:

Too many unread incoming messages [2055] : purging

Thus, data are lost at this startup phase. We need to run the ModemDriverBase::startup() in a separate thread, or some other such mitigation method to continue reading mail.

We need a stack to handle the current group and verbosity settings when glog is called recursively, e.g.

void write()
{
  glog.is(DEBUG1) && glog << group("bar") << "testing" << std::endl;
}

int main()
{
   glog.is(DEBUG2) && glog << group("foo") << write() << std::endl;
}

In this case, the embedded call to write() calls glog again and when it is flushed (std::endl), the group and verbosity are erased. This means that in the case of multithreaded use, the logger mutex is never released, and other threads will wait forever on locking the mutex.

I'm experiencing some problems in the build process of goby.
I've tried by myself to fix them, but unfortunately I've a problem during the linking of goby_moos which fails. In the following I summarize what I added.

In the main CMakeLists.txt I've added few lines to include the support for homebrew package-manager:

## Homebrew
include_directories(/usr/local/include)
link_directories(/usr/local/lib) 

In the main CMakeLists.txt of moos submodule I've added:

find_package(MOOSGeodesy QUIET)
include_directories(${MOOSGeodesy_INCLUDE_DIRS})

target_link_libraries(goby_moos
  goby_util
  goby_common 
  goby_acomms
  ${MOOS_LIBRARIES} ${Boost_LIBRARIES} ${PROTOBUF_LIBRARY} 
  ${MOOSGeodesy_DEPEND_LIBRARIES}
)

I'm trying to build Goby 2.1.8 using DCCL 3.0.4 on Enterprise Linux 6.9, but it appears that protoc-gen-dccl is not being built. It's not obvious where this is attempted, as DCCL builds cleanly and a simple grep of its tree didn't reveal any eurekas.

What is required to generate protoc-gen-dccl ?

Note that I'm using an older version of protoc (v2.3) at this point. It is provided by the EPEL repository.

ERROR MESSAGE

$ ./build.sh
Configuring Goby
cmake ..
Compiling in Git source tree.
libprotobuf < 2.5.0
Using DCCL in /opt/dccl/build
Including protobuf dir: /opt/dccl/include
-- Boost version: 1.41.0
-- Found the following Boost libraries:
-- signals
-- GMP libs: /usr/lib64/libgmp.so /usr/lib64/libgmpxx.so
-- Boost version: 1.41.0
-- Found the following Boost libraries:
-- system
-- date_time
-- thread
-- program_options
-- filesystem
-- regex
-- Boost version: 1.41.0
-- Found the following Boost libraries:
-- signals
-- Configuring done
-- Generating done
-- Build files have been written to: /opt/goby/build
Building Goby
make
[ 1%] Generating BinLink
[ 1%] Built target bin_link
[ 1%] Generating IncludeLink
[ 1%] Built target inc_link
[ 1%] Generating LibLink
[ 1%] Built target lib_link
[ 1%] Generating ShareLink
[ 1%] Built target share_link
[ 2%] Generating WtResourcesLink
[ 2%] Built target wt_link
[ 2%] Running C++ protocol buffer compiler on /opt/goby/src/common/protobuf/pubsub_node_config.proto
protoc-gen-dccl: program not found or is not executable
--dccl_out: protoc-gen-dccl: Plugin failed with status code 1.
make[2]: *** [include/goby/common/protobuf/pubsub_node_config.pb.cc] Error 1
make[1]: *** [src/common/CMakeFiles/goby_common.dir/all] Error 2
make: *** [all] Error 2

SOFTWARE VERSIONS

$ g++ --version
g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ protoc --version
libprotoc 2.3.0

$ pwd
/opt/moos-ivp
$ git log -1
commit 62dca1c06aa532c984420283b7aba4103b769879
Author: Mohamed Saad Ibn Seddik [email protected]
Date: Thu Mar 3 15:13:35 2016 -0500

15.5

$ pwd
/opt/dccl
$ git log -1
commit 7abb906be5b705cdf876f4324cf5fd8c887c117e
Author: Toby Schneider [email protected]
Date: Fri Jun 24 11:36:08 2016 -0400

Release 3.0.4

$ ls bin/
analyze_dccl dccl
$ ls lib/
libdccl_arithmetic.so libdccl_ccl_compat.so libdccl.so
libdccl_arithmetic.so.30 libdccl_ccl_compat.so.30 libdccl.so.30
libdccl_arithmetic.so.3.0.4 libdccl_ccl_compat.so.3.0.4 libdccl.so.3.0.4

$ echo $DCCL_DIR
/opt/dccl/build

$ pwd
/opt/goby
$ git log -1
commit a3a92af
Author: Toby Schneider [email protected]
Date: Tue May 9 16:43:42 2017 -0400

Release 2.1.8

Unable to build goby2 on RHEL/CentOS 8.

There are three main issues with compiling on RHEL/CentOS 8:

1. boost::asio::io_service was deprecated in 1.66. Replaced with boost::asio::io_context.
2. boost::asio::basic_stream_socket native() method deprecated in 1.47. Replaced with native_handle().
3. Using proj_api.h is deprecated. Can be enabled by defining ACCEPT_USE_OF_DEPRECATED_PROJ_API_H.

signal_raw_incoming and signal_raw_outgoing are not implemented in UDPDriver.

protobuf_to_inverse_moos fails with a Protobuf out of range error on the repeated TranslatorEntry::publish(int index) call if the number of publish blocks does not equal the number of create blocks.

MOOSTimeWarp multiplies the system clock, and "register_timer" users a normal int to store the next time value. This will overflow easily on time warp values (and will eventually on the normal system clock).