neilstephens / opendatacon Goto Github PK

After enabling the console plugin when quitting using Ctrl-\ the ODC console appeared to continue operation and broke Gnome Terminal 3.18.3.

andrew@Computer:~/opendatacon-0.3.4-Linux-x86_64-Release$` bin/opendatacon -c opendatacon-serialmaster.conf 
This is opendatacon version 0.3.4 ge1a42ab 
Loading configuration... 
Initialising Interfaces
Initialising DataPorts
Initialising DataConnectors
Enabling DataConnectors... 
Enabling DataPorts... 
Enabling Interfaces... 
Up and running.
odc> quit
Unknown command: quit
odc> Disabling user interfaces... 
Disabling data connectors... 
Disabling data ports... 
Finishing asynchronous tasks... 
Destoying Interfaces... 
Destoying DataConnectors... 
Destoying DataPorts... 
Shutting down DNP3 manager... 
opendatacon version 0.3.4 ge1a42ab  shutdown cleanly.
andrew@Computer:~/opendatacon-0.3.4-Linux-x86_64-Release$ PARSE ERROR: Argument: dssd
             Couldn't find match for argument

Brief USAGE: 
   bin/opendatacon  [-r] [-d] [-i] [-p <string>] [-c <string>] [--]
                    [--version] [-h]

For complete USAGE and HELP type: 
   bin/opendatacon --help

Consider a check for duplicate port names in conf file.

It would be nice to embed the version info for the 3rd party code that is compiled in/linked against
Eg.
asio (code)
jsoncpp (code)
libmodbus (lib)
libmicrohttpd (lib)
tclap (code)
etc.

When performing a console 'shutdown' the program does not always fully shut down, it intermittently gets stuck on the 'Finishing asynchronous tasks...' message and goes no further. This only seems to be a problem when data is being generated by the simulator. I didn't notice it happening when I had disabled data generation, and it was happening all the time when data generation was very fast.

Or generalise JSON port to be either client or server

ODC/ConfigParser.cpp: In member function ‘const Json::Value ConfigParser::GetConfiguration() const’:
ODC/ConfigParser.cpp:128:15: warning: ‘Reader’ is deprecated: Use CharReader and CharReaderBuilder instead [-Wdeprecated-declarations]
Json::Reader reader;
^~~~~~
In file included from /ConfigParser.h:31,
from ODC/ConfigParser.cpp:30:
ODC/../JSON/json/json.h:1389:83: note: declared here
class JSONCPP_DEPRECATED("Use CharReader and CharReaderBuilder instead") JSON_API Reader {

Changed to using a single JSON stream writer in commit 96326e5 . Turns out it's not a thread safe object (because it copies the stream pointer to a local member in BuiltStyledStreamWriter::write() )

Short term, I'll revert the change, so it'll be back to creating and destroying a new stream writer for every write :-(. Long term there must be a better way. I'll ask the jsoncpp devs if it's meant to be thread safe...

I was able to build the opendatacon without tests, however, when I attempt to build the tests I get a bunch of undefined references to several DNP3 functions:
openpal:LogRoot:LogRoot
openpal:LogRoot:GetLogger

What version of DNP3 should I be using?
I am using the DNP3 secauth branch because I was not able to compile the main opendatacon package with their master branch. The reason is the same, undefined functions. DNP3 seems to have been updated a lot.

been burnt a few times now when API interface classes change their abstract members, and it goes unnoticed.
Need to look into c++11 migration tools that may be able to add the override keyword automagically

1. Start the with basic example which have json and console port.
2. When you will shutdown from the system it will crash on windows.

On start up of log files are not being created. Config below:

{
    "LogName" : "ODC_Log_new",
    "LogFileSizekB" : 50000,
    "NumLogFiles":  1,
    "LOG_LEVEL":    "NORMAL",

    "Plugins" : 
    [
        {
            "Name" : "ConcoleUI",
            "Type" : "ConsoleUI",
            "Library" : "ConsoleUI",
            "ConfFilename" : "",
            "ConfOverrides" : {} 
        }
    ],

    "Ports" :
    [
        {
            "Name" : "DNP3_1",
            "Type" : "DNP3Master",
            "Library" : "DNP3Port",
            "ConfFilename" : "",
            "ConfOverrides" : 
            {
                "SerialDevice" : "/dev/ttyUSB0",
                "BaudRate" : 9600,
                "Parity" : "NONE",
                "DataBits" : 8,
                "StopBits" : 1,
                "MasterAddr" : 0,
                "OutstationAddr" : 2,

                //------- DNP3 Link Configuration--------#
                "LinkKeepAlivems" : 60000,
                "LinkNumRetry" : 0,
                "LinkTimeoutms" : 5000,
                "LinkUseConfirms" : false,

                //-------- DNP3 Common Application Configuration -------------#
                "EnableUnsol": true,
                "UnsolClass1": false,
                "UnsolClass2": false,
                "UnsolClass3": false,

                //-------Outstation Stack conf--------#
                "SelectTimeoutms" : 30000,
                "SolConfirmTimeoutms" : 30000,
                "UnsolConfirmTimeoutms" : 30000,
                "EventBinaryResponse": "Group2Var2",
                "EventAnalogResponse": "Group32Var5",
                "EventCounterResponse": "Group22Var1",

                "StaticBinaryResponse": "Group1Var2",
                "StaticAnalogResponse": "Group30Var5",
                "StaticCounterResponse": "Group20Var1",

                "WaitForCommandResponses": false,

                //-------Point conf--------#
                "Binaries" : [{"Index": 0},{"Index": 1},{"Index": 5},{"Index": 6},{"Index": 7},{"Index": 8},{"Index": 10},{"Index": 11},{"Index": 12},{"Index": 13},{"Index": 14},{"Index": 15}],
                "Analogs" : [{"Range" : {"Start" : 0, "Stop" : 5}}],
                "BinaryControls" : [{"Range" : {"Start" : 0, "Stop" : 4}}]
            }
        }

    ]
}

On OEL the if the JSON port is not overridden in the main config file the and is only specified in the "ConfFilename" it does not run on IPv4 only IPv6. But if it is overridden then it works! Build# g3fa348a

JSON.conf
{
"ConfOverrides" :
{
"IP" : "0.0.0.0", "Port" : 2598,
//"OutputTemplate" : {"Context":{"Time":"","Quality" : ""},"Val":"","Name":"","Index" : ""},
//"StyleOutput" : true,
"JSONPointConf" :
[
]
}
}
ODC.conf

{
"LogName" : "ODC_Log",
"LogFileSizekB" : 50000,
"NumLogFiles": 1,
"LOG_LEVEL": "ALL",

"Ports" :
[
	{
		"Name" : "DNP3Master",
		"Type" : "DNP3Master",
		"Library" : "DNP3Port",
		"ConfFilename" : "DNP3_Master.conf",
		"ConfOverrides" : { "IP" : "192.168.1.11", "Port" : 20000, "MasterAddr" : 0, "OutstationAddr" : 1}
	},
	{
		"Name" : "JSONTEST",
		"Type" : "JSONServer",
		"Library" : "JSONPort",
		"ConfFilename" : "JSON.conf",
		"ConfOverrides" : { "Port" : 2598}
	},	
],

"Connectors" :
[
	{
		"Name" : "TEST1",
		"ConfFilename" : "",
		"ConfOverrides" :
		{
			"Connections" :
			[
				{
					"Name" : "TEST1",
					"Port1" : "TEST1DNP3Master",
					"Port2" : "JSONTEST1"
				}
			]
		}
	}
	
]

}

Opendnp3 has a TLS mode. It would be very useful to have a implementation of the TLS port in opendatacon to secure data comms over hostile networks.

I see that there are several scripts in the install directory to run the opendatacon daemon.

When I run the init script I get the following error:
/etc/init.d/functions: No such file or directory

My assumption is that I have to do something else other than building and installing the opendatacon in order to be able to run it. What am I missing?

Most importantly TCP keepalives.

This should be easy enough for the ports that use the native TCP socket manager, but need to investigate if it's possible with opendnp3...

Add Rpi and Oracle/CentOS/RHEL builds to Travis

• use docker images
• use qemu for Rpi (docker w qemu saves messing about with sysroot for cross-compilation)
• possibly cache docker images if installing build tools/deps slow

Add 32bit linux builds using -m32 and multilib

Caching for incremental builds (same approach on Travis and Appveyor)

• cache source and build dirs, then git pull from the fresh clone - this should result in incr. build
• do a full build from fresh clone on tagged commits

Caching for homebrew packages on OSX

• this should speed things up a lot because I think homebrew builds from source

In a situation where multiple connections are made to a single JSON port there is no way to identify which connector sent the data.

Please allow the output of the sender in the JSON port.

Exit using SIGQUIT (instead of shutdown command) when using ConsoleUI on Linux leaves terminal in a bad state. Suspect tinycon code puts terminal in a different 'mode?' on ctrl press...

It's been observed that high latency on a TCP DNP3 connection can be enough for the link status to timeout (and master will set comms point failed and point quality comm_lost).
BUT the TCP connection doesn't necessarily drop out, and eventually the link status reply comes through (so the comms point is set good).
The problem is that if the TCP connection didn't drop out, the master didn't have to do an integrity scan, and that's what we rely on to reset the point quality.

In summary, opendatacon has an invalid assumption built in that there's always an integrity scan after a comms fail, but this is a false assumption.

ACTIONS:
Reconsider forcing the TCP connection closed on link status timeout; this is what the DNP3 spec says in an optional section, but I was avoiding doing that because it effects multidropped stations on the same connection.

Consider forcing an integrity scan after link-timeout, but this might result in redundant scans if the TCP connection did go down (or messy state logic to avoid it).

When running example non-demand and the WebUI is open in a browser the shutdown is very slow. It hangs at:
odc> shutdown Disabling user interfaces...
for a very long time.

Interestingly a refresh of the WebUI causes the process to quit cleanly.

In addition to that if you try to kill the process it dies messily with the following error:
`Disabling user interfaces...
Fatal error in GNU libmicrohttpd daemon.c:4153: Failed to join a thread

Disabling user interfaces...
Fatal error in GNU libmicrohttpd daemon.c:4153: Failed to join a thread

Disabling user interfaces...
Fatal error in GNU libmicrohttpd daemon.c:4153: Failed to join a thread

Disabling user interfaces...
Fatal error in GNU libmicrohttpd daemon.c:4153: Failed to join a thread

Disabling user interfaces...
Fatal error in GNU libmicrohttpd daemon.c:4153: Failed to join a thread

Disabling user interfaces...
Fatal error in GNU libmicrohttpd daemon.c:4153: Failed to join a thread

Disabling user interfaces...
Fatal error in GNU libmicrohttpd daemon.c:4153: Failed to join a thread

Disabling user interfaces...
Fatal error in GNU libmicrohttpd daemon.c:4153: Failed to join a thread

Disabling user interfaces...
Fatal error in GNU libmicrohttpd daemon.c:4153: Failed to join a thread

Disabling user interfaces...
Fatal error in GNU libmicrohttpd daemon.c:4153: Failed to join a thread

Disabling user interfaces...
Segmentation fault
`

Good Luck
Andrew

1. Right now the Web UI plugin is not working correctly (may be it broken).
2. Even if it is working we don't throw all the functionalities through Web UI plugin, what we provide through console.

Need to split out other control types in config. At the moment, all controls use the list of indexes called "BinaryControls" in the point config

I am trying to send a command to a JSON port that is connected to DNP3.

I have the json port open with netcat and I tried pasting either of the following into nc to get a command issued.

{"Controls":{"0":1}, "ControlMode": "Latch"}
{"Controls":{"1":1}}

The JSONPointConf looks like the following.

{
"PointType" : "Control",
"Points" :
[
{"Name":"CRTL1", "Index": 0, "JSONPath" : ["controls","0"]},
{"Name":"CRTL2", "Index": 1, "JSONPath" : ["controls","1"]},
{"Name":"CRTL3", "Index": 2, "JSONPath" : ["controls","2"]},
{"Name":"CRTL4", "Index": 3, "JSONPath" : ["controls","3"]},
{"Name":"CRTL5", "Index": 4, "JSONPath" : ["controls","4"]}
]
},

Are they correct as I do not see anything in the log or the connected DNP3 device which is scanning.

Andrew

DNP3 outstation and master ports have the ability to re-timestamp events. They default to adding a timestamp if the event doesn't have its timestamp set already (value zero). In this case, probably log a warning. In the case of explicit overrides, probably log an info message...

opendatacon consumes close to 100% CPU when run as a linux service if the opendatacon.conf includes the ConsoleUI plugin. Using init script in /etc/rc.d/init.d

e.g.

	"Plugins" :
	[
		{
			"Name" : "ConsoleUI-1",
			"Type" : "ConsoleUI",
			"Library" : "ConsoleUI",
			"ConfFilename" : "",
			"ConfOverrides" : { }
		}
	],

Confirmed in opendatacon versions:

• 1.3.0 g3c5a8b4(20 ahead) (OEL7)
• 0.5.1 gca0fdf8 (OEL6)

This is opendatacon version 0.4.0 g3fa348a
Loading configuration...
Caught exception: in Json::Value::resolveReference(key, end): requires objectValue

Could be a little more descriptive... It was an ill defined JSON object in a file brought in via "ConfFilename". The object was not enclosed in {}.

The divide and conquer method of working out what was wrong did not lead in the right direction. It was by chance that it was worked out.

In an attempt to provide mitigation of #45 , there's a DNP3 Outstation option (WaitForCommandResponses) that's meant to allow immediate return of control status without waiting for the actual result:

auto future_results = PublishCommand(arCommand, aIndex);

	auto pConf = static_cast<DNP3PortConf*>(this->pConf.get());
	if (!pConf->pPointConf->WaitForCommandResponses)
	{
		return CommandStatus::SUCCESS;
	}

This doesn't actually work, because the d'tors of the futures block. This actually makes the situation worse than if the option isn't used, because the thread is actually blocked, whereas we'd normally call poll_one() (still a problem - see #45 )

It's outside the scope of DNP3, but we could implement a forced integrity scan

In the case of a broken channel, we need a(n optional) way to close and reopen it. This either needs some coordination to disable all the stations on the channel at the same time, or a mod to opendnp3 to expose Enable/Disable methods on the channel (only has Shutdown at the moment), and the stations just get the usual channel state notifications.

A real life example: TCP connection gets broken by network issue (no FINs, so the server endpoint remains established and doesn't re-listen). The client can't reconnect until TCP times out (hours), all the while link keepalives are failing, but there's currently no way to close and reopen the connection.

In addition to #18 ODC is not locking the serial port. The same config applies.

I was testing this out with the latest release on here 0.5.1 and I can't seem to get the WebUI plugin or the Modbus plugin to load on Windows or Ubuntu. I always get something like the following

This is opendatacon version 0.5.1 gca0fdf8
Loading configuration...
WebUI Info: dynamic library load failed 'WebUI.dll' skipping plugin...
WebUI Error: failed to load plugin, skipping...
Warning: failed to load library 'ModbusPort.dll' mapping to null port...
Initialising Interfaces
Initialising DataPorts
Initialising DataConnectors
Enabling DataConnectors...
Enabling DataPorts...
Enabling Interfaces...
Up and running.

The weird thing is that if in Windows I watch opendatacon load the dll files with procmon it shows them loading successfully as can be seen in the image below.

I'd like to request a feature to be added. Can a startup delay can be added for each RTU/outstation, so the initial unsolicited message sent to the master station can be delayed for an arbitrary period of time? This would assist with debugging.

See comment/review on pull request: #60 (comment) @cpp-rakesh

Need to update to the latest opendnp3 API, then use cmake to keep up-to-date by using sub-repository dependence

Please update

I have been playing with a SDM120C Power meter.

I have used this program to verify that the meter and RS485 interface work no a RPI, and it reads the meter correctly.
https://github.com/gianfrdp/SDM120C by @gianfrdp

This is an packet capture of the SDM120C software requesting only the voltage register.
Device --> (01)(04)(00)(00)(00)(02)(71)(cb)
Host --> (01)(04)(04)(43)(6e)(99)(9a)(65)(e6)

I found that setting the Modbus config as below:

{
  // MODBUS Configuration for the SDM120c


  "PollGroups" : [{"PollRate" : 10000, "ID" : 1}],
  "InputRegIndicies" :
  [

                // Power Meter Quantities (Voltage (high 16 bits), Voltage (low 16 bits))
                        {"IndexOffset":0,"Range":{"Start":0,"Stop":1},"PollGroup":1}




  ]


  }

Gave this output:

Device --> (01)(04)(00)(00)(00)(02)(71)(cb)
Host   --> (01)(04)(04)(43)(6c)(80)(00)(4f)(dd)

After which ODC promptly quit with the following output.

opendatacon -c opedatacon.conf 
This is opendatacon version 0.3.3 g091b827 
Loading configuration... 
WARNING: WebUI port 10443: The key/certificate files could not be read. Reverting to default certificate.
Initialising Interfaces
Initialising DataPorts
Initialising DataConnectors
Enabling DataConnectors... 
Enabling DataPorts... 
Enabling Interfaces... 
Up and running.
odc> opendatacon: /home/pi/dnp3/./cpp/libs/include/openpal/container/ArrayView.h:66: const ValueType& openpal::ArrayView<ValueType, IndexType>::operator[](IndexType) const [with ValueType = opendnp3::Cell<opendnp3::Analog>; IndexType = short unsigned int]: Assertion `index < this->size' failed.
Disabling user interfaces... 
Disabling data connectors... 
Disabling data ports... 
Finishing asynchronous tasks... 
Destoying Interfaces... 
Destoying Interfaces... 
Aborted

Is there any documentation on how to compile / use the opendatacon? Thanks.

To avoid blocking a thread when waiting on std::futures to be set, poll_one() is being used. This succeeds in not blocking the thread, but it still blocks the progress of a strand.

• add explaination of git flow branching
• list cmake vars in build instructions
• explain (after a inventing a method to deal with) dependency versions ie. opendnp3 libmodbus etc.
• fix the broken/incomplete example config files to be in line with latest version and show new features
• document the webui and new console ui

Check index of received events to make sure a point with that index is configured, if not, log a warning and drop the event, otherwise an assert in opendnp3 will fail and opendatacon will stop.

Would be worthwhile adding support for octet strings (opendnp3::OctetString).

See IEEE 1815 - section 11.9.7

From SimControl, when we try to generate an event it sends successfully but twice not once.

Adopt the opendnp3 astyle template to get started?
http://dnp3.github.io/doc/2.0.x/contributing/formatting.html

Should re-timestamping functionality be core functionality?

It could be added to the IOHandler base class, but does it belong there, or should it stay as a port specific implementation?