Hi, I'm trying to install the runtime with the docker file to try out the linux runtime.
docker build -t openplc:v3 .

During buiding I get the error

Setting up swapspace version 1, size = 1000 MiB (1048571904 bytes)
no label, UUID=f66f37e1-41f8-4579-b0c1-4287c3c81be5
+ swapon swapfile
swapon: /workdir/utils/dnp3_src/swapfile: insecure permissions 0644, 0600 suggested.
swapon: /workdir/utils/dnp3_src/swapfile: swapon failed: Operation not permitted
+ cmake ../dnp3_src
CMake Error: The current CMakeCache.txt directory /workdir/utils/dnp3_src/CMakeCache.txt is different than the directory /home/bdb/projects/OpenPLC_v3/utils/dnp3_src where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt
+ make
/bin/sh: 1: cd: can't cd to /home/bdb/projects/OpenPLC_v3/utils/dnp3_src
make[2]: *** [CMakeFiles/openpal.dir/depend] Error 2
CMakeFiles/openpal.dir/build.make:543: recipe for target 'CMakeFiles/openpal.dir/depend' failed
CMakeFiles/Makefile2:212: recipe for target 'CMakeFiles/openpal.dir/all' failed
make[1]: *** [CMakeFiles/openpal.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2
+ make install
/bin/sh: 1: cd: can't cd to /home/bdb/projects/OpenPLC_v3/utils/dnp3_src
make[2]: *** [CMakeFiles/openpal.dir/depend] Error 2
CMakeFiles/openpal.dir/build.make:543: recipe for target 'CMakeFiles/openpal.dir/depend' failed
make[1]: *** [CMakeFiles/openpal.dir/all] Error 2
CMakeFiles/Makefile2:212: recipe for target 'CMakeFiles/openpal.dir/all' failed
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2
+ '[' 2 -ne 0 ']'
+ echo 'Error installing OpenDNP3'
+ echo 'OpenPLC was NOT installed!'
+ exit 1

since i'm not sure what the purpose of this is, am I doing something wrong?

I want to try an experiment,
of building openplc agains different libs version..

Can u give me a hint as to which are "externals"
eg

dnp3 is here https://github.com/automatak/dnp3 and all the "tags"

which are the other externals.. ?

libmodbus = https://github.com/stephane/libmodbus
matiec = https://bitbucket.org/mjsousa/matiec (NO TAGS)
?

Hi @thiagoralves

Got some time so hacking away at last part of www server merge.. #3

Am also looking at dumping sqlite and using files instead (json etc)

You mention csv files for storage,
so for hacking purposes do u have a few samples or the format ??

Problem

Currently, we pass messages between the runtime and the webserver for control operations using carefully written strings send over sockets. This has caused problems during development because either only one side of the message is updated or I would like to safely change the message. Furthermore, it is easy to "break the build" because of an incompatibility. This also makes it harder to develop a new web server since it must conform to an undocumented API.

Solution

We will define the allowed messages between the web server and runtime using a language independent format (IDL). The candidates here are protobuf and flatbuffers. This allow you to define the messages in a special syntax and then you use a special compiler to convert that into platform specific code.

While protobuf are definitely more popular, flatbuffers may be more appropriate in this case because they a nicer to the heap which is important for embedded systems.

Approach

The plan for this is to define the message that we want to allow. There are some improvements that would be very nice (for example querying a subset of the logs) that would also improve overall performance.

@thiagoralves what is your opinion on fixing the Python dependencies through a requirements.txt file?

This way we won't break if for instance the Flask project has some changes in the latest version. We can just issue a pip install -r requirements.txt command for installation of the dependencies.

I can submit a PR if you're interested.

Using openplc_v3 how do you upload a modbus configuration file? I tried following the directions found here but didn't see a place in the GUI to upload the new config (as there was in openplc v2).

I followed the install directions for v3, but unlike in v2, there was no option to activate the modbus driver.

Hi,

I'm working on making some changes to the C++ code for DNP3. I've noticed that there are no unit tests for any of the C++ code (or at least as far as I can tell).

I'd like to propose the following:

catch2 as a test framework
CMake as a build system for tests

I haven't used catch2, but it seems to be a more popular than google test (the last C++ testing framework I used).

I have some mixed feelings on this proposal - CMake is a heavy dependency, however, that dependency is already there from OpenDNP3. It is also very possible that this would only be necessary for tests.

Any thoughts on these approaches?

@pedromorgan and @thiagoralves
As we discussed in #14 and #26 we should consider a CI pipeline on Windows as well. I know for a fact that appveyor is the most popular CI tool for Windows builds but I personally have no experience with it.

If it's fine with you guys, can we start an experiment in my fork to add appveyor support?

Or, maybe @pedromorgan do you know of any other (free) CI option for Windows?

So here's what pete's up to as others busy..

main reasoning for moving to gitlab is

• it got a project instead of a personal thing gone mad (linplc)
• the ci and diffferent build intergrated on host (easier)
• MAINLY for me.. build docs from src in ci and publish ;-))) api docs updated
• this is a serieouly limiting factor for projects and maintaining incorrect/correct docs

doxygen is good for c++ and stuff

mainly doxygen for c/c+= in partucular

IMPORTANT, each project contains its own sub

openplcproject.com

• currently its on wix (or some hosted)
• https://openplcproject.gitlab.io/ is replacement DNS target
• It is jekyll powered and src autobuild from master (like github)
• need some work but mainly replaces current

Hello @thiagoralves ,

I've been using OpenPLC with the rpi3 and everything worked nicely. But today I tried to install it on another one and the install script failed. I noticed that there was a swapfile among the files, that wasn't there on the other rpi3. After looking through the install script, I found the lines that create the swapfile. I removed them and OpenPLC installed nicely. Basically, there wasn't enough space on my SD card for the swapfile, but there was enough for OpenPLC.

Is there any particular reason for creating the swapfile? If no, I would suggest removing these lines, or if yes, I think it should be made conditional and skipped if there isn't enough disk space to create them. If you agree, I would be happy create a pull request with whatever solution we come up with.

In refactoring, bumped into issue where "my settings" in DB here didn't have enip etc..

so refactoring with defaultSettings in py to keep running..

Main Question ?

is there somewhere we can save the default settings, and machine readable for c/c++ etc and py et all

• Snag is that with the db, its in the db and query crashes,,,
• prefer a runtime/etc/settings.ini or whatever format work on most plats instead of having this problem
• eg a txt file with key=val

defaultSettings = dict(
    modbus_enabled="false",
    modbus_port=502,

    dnp3_enabled="false",
    dnp3_port=20000,

    enip_enabled="false",
    enip_port=20000,

    Pstorage_enabled="false",
    Pstorage_polling=10,

    Start_run_mode="false",
    Slave_polling=100,
    Slave_timeout=1000
)

I add two BYTE number, compile it, it errors.

Docs = generate api and stuff from source code..

Its a complex project with c++, py, sh et all..
To skin the cos cat,
I suggest firing doxygen at the source..

Doxygen is cool cos it parses cpp + c initially,
but also md and many others...

And automate it..

This would give the api docs..

So am new to .st

But all this seems to do is ??
https://github.com/thiagoralves/OpenPLC_v3/blob/master/utils/st_optimizer_src/st_optimizer.cpp

Loking at it, this would be simple few lines function in python..
def st_optimize(inp_file)

am so happy I dont code c++ anymore ;-)))))

@pedromorgan says..

• we take our hats of to @thiagoralves the BDFL
• I download checkout compile and run, snag no pc equip etc..
• sniff around, look at commits (tags) etc
• so its a fantastic project, and needs help to make it better_better ;-) wanna help

@pedromorgan @thiagoralves

Not so fast... we need to first...

• Create a working mirror of thiagoralves/OpenPLC_v3 with CI/CD pipleline on Gitlab
• Test the generated pages to satisfaction
• Replace the Build Badge from Travis

When using the Hello World example on a Raspberry Pi 3B, Linux raspberrypi 4.14.98-v7+ #1200,
when executing the example, the output %QX0.0 nicely switches off after 2 seconds in the OpenPLC Monitoring page.
The physical pin 8 on the PI connector stays 3.3V for various time between 2 and 11 seconds though, before returning to 0V level.

Currently, the DNP3 console logger does not log into the SPD logs. It needs to.

Investigate using Cpack to make binaries and installers, eg .deb files

https://cmake.org/cmake/help/v3.0/module/CPack.html

https://nowardev.wordpress.com/2009/12/27/how-to-create-a-debian-package-for-script-and-for-project-that-use-cmake/

One of the snags a the moment is that data is in sqlite.db,
and this makes it difficult to change etc and maintaing, update

So sql.db is what I would have gone for also..
but the only snag from experience is where one want to update shema etc and olde db.. arghh!!!

So can we introduce "workspace" or "work" directory..

This is required when u start a server...
and the idea is all your stuff in one external directory (can be gitable, dropable, zipable)

and instead of having "tables" for data,
we have files and directories instead.

openplc -w ~/my_plc_stuff```

Suggested Layout

/config.yaml - main config

so looking at recent development branch there are a couple of issues I wish to raise...

The root of the problem/snag is that the build process rewrites files within the repos itself, eg the openplc.db file. I only truly realised this problem when i started refactoring the www server.

The recent restructure with the 'runtime/' , scripts and cmake etc is cool, but doesnt solve the whole problem. Its also confusing for a new dev like myself.

To really solve it, I think there needs to be a "openplc_workspace/", play/, my_stuff/ or whatever its called outside of openplc file structure completely.. In fact the idea imho is that its a git.repo itself for versioning as project.

so the structure with the "workspace" dir could have

/bin/ = all binary exe's in here
/build = cmake compile and stuff
/config/ = opelplc_platform, hardware, curr_program // etc
/db/* = all yaml files replacing the db
/etc/* = other stuff
/vendor/* = thirdparty stuff
/secrets/* = ssh keys etc

This means all the "scripts" + www etc need to have a "workspace" config somehow somewhere...

Ok so the c++ and stuff makes me nervous..

But the project should be spit up anyway..

• the runtime
• the monitor..

So this is the way I got in on #46

OpenPLC_v3/*

• install.sh
• start_openplc.py
• /etc/
  • core/
    • *.cpp/h - main.cpp, app and header files (dnp, modbus server etc)
    • /hardware_layers/*.cpp - platform files
    • /lib/*.h - iec_* files etc
  • docs/* - doxygen config+files (also add api stuff here)
  • lib/*.txt - pile of .txt files
  • st_files/
    • blank_program.st
  • openplc_default.db - default db (soon to be abandoned)
• /scripts/
  • common.sh - common vars to all scripts
  • change_hardware_layer.sh
  • compile_program.sh
• /utils/ - all the sources and some third party libs
• /webserver/ - all python and html templates only
• /build/ -
  • bin/* - where executables are build to
  • src_gen/* - where src code is copied to and compiled

So I realise that the

• sqlite db is in version control
• and cannot see any schema creation scripts ( do they exists )

Guess the db needs to be created on setup and places in users home dir etc

Is there a risk of stack overflow? As a function, large structures can be transferred.

Problem

The current design uses socket-based messaging. This works well provided that the components are running in the same context and that you use a runtime that supports sockets (e.g. not Windows). However, this approach would not work if you want to run different parts of the system in different Docker containers (for example, to improve security).

Solution

This proposal is to replace the current socket-based messaging with nng (https://github.com/nanomsg/nng). nng can use socket-based messaging, but also provides abstractions that permit using different transport mechanisms that can work across containers.
It offers some other nice things in that it works on Windows without MinGW/Cygwin. It has implementations in a variety of languages, including C, Go, Rust, Python.

As the snapshot，I make the first bite to another number, it could not support.

Hello,
I got the following error, when trying to install OpenPLC on raspberry pi.
g++ -g -Wall -Wpointer-arith -Wwrite-strings -Wno-unused -DHGVERSION="\"\"" -o iec2c main.o stage1_2/libstage1_2.a stage3/libstage3.a stage4/generate_c/libstage4_c.a absyntax/libabsyntax.a absyntax_utils/libabsyntax_utils.a /usr/bin/ld: stage3/libstage3.a(stage3.o): in function `case_elements_check(symbol_c*)': /home/pi/OpenPLC_v3/utils/matiec_src/stage3/stage3.cc:134: undefined reference to `case_elements_check_c::case_elements_check_c(symbol_c*)' /usr/bin/ld: /home/pi/OpenPLC_v3/utils/matiec_src/stage3/stage3.cc:136: undefined reference to `case_elements_check_c::get_error_count()' /usr/bin/ld: /home/pi/OpenPLC_v3/utils/matiec_src/stage3/stage3.cc:134: undefined reference to `case_elements_check_c::~case_elements_check_c()' /usr/bin/ld: /home/pi/OpenPLC_v3/utils/matiec_src/stage3/stage3.cc:134: undefined reference to `case_elements_check_c::~case_elements_check_c()' collect2: error: ld returned 1 exit status make[1]: *** [Makefile:453: iec2c] Error 1 make[1]: Leaving directory '/home/pi/OpenPLC_v3/utils/matiec_src' make: *** [Makefile:494: all-recursive] Error 1

This has to go..

all uploads atmo go into the etc/et_file/*

and file same need overwrittern..

the upload atmo goes into etc/st_files/prog.X.st where X is the pk from db, and later just a randy hash.hash

Just notice that there's a cmake of gluegenerator..
so being a muppet to cmake kinda

mkdir build/glue_evo
cd build/glue_evo
cmake ../../utils/glue_generator_src/

and it compiled ;-))
but ended up up with 2 files

utils/glue_generator_src/bin/glue_generator
utils/glue_generator_src/bin/glue_generator_test

So it works..
Main question is whats strategic plan ??

@thiagoralves would it be a good idea to put the source-code for the HMI builder and editor projects on GitHub?

The bottom of the http://www.openplcproject.com webpage says OpenPLC is licensed under CC BY-SA, but it doesn't say that anywhere in the git repo.

The C++ code in webserver/core says OpenPLC is GPLv3+.

The libraries in utils/ each contain their license declarations, but the core openplc python code in webserver/ doesn't mention any license.

Problem

As a production environment, it is important to have well constructed and reliable logs so that I can diagnose issues that happen in the field (postmortem) and analyse logs in real time.

The current OpenPLC application generates logs in two places:

• printed to stdout
• supplied to the web front end

However, this does not give me the configuration that I need. For example, I want to be able to enable debugging to get additional logs. I also want to be able to easily send the logs somewhere else. It is also important that I can get more detailed information from each entry, in particular time stamps.

Additionally, the in-memory logs currently store up to 1MB of data. Hitting this endpoint would cause a substantial amount of data to be transferred. Thus I want to reduce the risk of performance impacts from this endpoint.

Solution

We will integrate the spdlog library as the core of logging. This is a very popular and high-performance logging library that supports multi-threading. This will enable logs messages to target different loggers and log levels, thus providing configuration. The currently socket-based query can be achieved with a custom sink.

In the future, we will enable configuration of logging at the command line or possibly through a web front end. These would need to support at least the following capabilities:

• Specify the log level, eg. trace, debug, info, etc
• Specify log targets (the current circular buffer, stdout and syslog

When i use the newest MatIEC， some problem, as when i have a st that include a functionblock variable， the error line is add to 1，when i have two functionblock variable, the error line is add 2.
like my st file, could you help me?
plc_1.zip
plc_2.zip

There seems to be two files

• https://github.com/thiagoralves/OpenPLC_v3/blob/master/webserver/core/dnp3.cfg
• https://github.com/thiagoralves/OpenPLC_v3/blob/master/webserver/dnp3.cfg

Both files are different ?
so which is the correct one, or is is "generated" somewhere ??

Ok so this might be crazy..

But the more I dig into the code,
I realise there is a lot of comms stuff.. and alike
going on etc etc..

This smacks of what golang is good at, concurrency, multi things going on, websockets et all..

Would be intrigued to start a project as altername, for fun of course

Am looking at the webserver code.

Can I suggest using templating (eg Jinja2 is popular), as this would make things easier.

I can implement if the maintainers think its a good idea.

So this had to go into the same directory as where the openplc command is ?

ie /bin/openplc

is that correct ??

Ok so hacked a bit,
and finding where the project is at..
and helping with a future..
so am adding ideas and issues in this thread. for fun

I am having this issue where the third version of OpenPLC maxes out two cores on my machine. I have run it in many different environments. An 8 core xeon processor, a two core VM, and an i7 machine all of them, while running Openplc have two cores maxed out. To me it seems like this shouldn't happen? Have you seen the same results? Do you have a solution? Installed using the linux option on the install script and running the blank program

I code below snapshot, compile it ,errors.

Problem

The existing web server works reasonably well, but has a few problems.

1. It is absent tests so it is exceptionally easy to break things.
2. It depends on Python, and not all of the systems that I'd like to use OpenPLC on will have Python
3. From a performance perspective, the current web server behaves in a way can negatively affect the critical runtime

Solution

The proposed solution is to write a web server in Rust. This project would initially be separate from OpenPLC, but if it works, could be treated as a complimentary web server (or perhaps replace the current one).

This solution would use the messaging proposed in #68 and avoid the socket implementation completely. It would also depend on an IDL for communicating with both the web front end and the runtime (although those are inherently different). The server and design also needs to handle important things including:

1. Authentication
2. Rate throttling
3. Unit testing

Solution Alternatives

@pedromorgan has already don't work to make a web server in Go, so why not Go? The reasons is that Go is less portable than Rust and requires more of the system. Why not C? Because no one should write a web server in C.

Hi
When I type cmd with ./install.sh linux. on my ubuntu 16.04 pc.
I have encountered some error messages during installation. Like the following.

glue_generator.cpp:269:48: error: no matching function for call to ‘std::basic_ofstream<char>::basic_ofstream(std::__cxx11::string&, const openmode&)’
  ofstream glueVars(output_file_name, ios::trunc);

I think it might be related to the compiled parameters.
So I replaced g++ in install.sh file with g++ -std=c++11.
And it working normally.

Just to hit on..

Namespace,

so in url's an using
/programs
/program//start|stop

We need to namspace the stuff..

so maybe
oplc is good for me..

Currently, files that form the OpenPLC platform and user provided files are written to the same directory. A result of that is git status shows a lot of changes, and thus creating a complex history. It can also cause merge issues since they may write to the same files.

Thus, I would like to separate user supplied files from those that are part of the platform. There is already some work that moves the core files into etc. This change takes that a little further to move those files into etc and have user files in usr. In the usr folder, propose to have the following:

usr\st_files <= uploaded ST files
usr\core <= generated by iec2c
usr\cfg <= configuration files

So this is a side project in a way
https://openplcproject.gitlab.io/openplc_go/

Please test by downloading and run

• the sqlite import is a pain in the butt

This is adding to a previous comment from @thiagoralves somewhere

So can i suggest that the firmware is moved into main repos to avoid downloading etc..

all in the box=git

Problem

There are several developers who are contributing code to this project. We each have our own preferences for how things are structured, but that's bad. We should have a common style to make the code easier to read, to eliminate potential bugs, and improve overall quality.

Solution

Improve the CI build scripts to include automated code quality checks using domain-specific style checkers. For the C++ code, this would be cppcheck as a start.

I'm trying to fix some issues with DNP3 and in the process was trying to get things to compile locally on my machine with Visual Studio (yes, I know, I'm doing it the hard way).

In any case, I noticed that the tm struct (iec_std_lib.h, line 275) does not match the standard definition from C90 (see http://www.cplusplus.com/reference/ctime/tm/).

Is there a particular reason for this difference? It is part of the IEC standard?

The reason I'm asking is I'm running in definition errors and trying to figure out what is the best way to resolve them.

Problem

The current DNP3 implementation is based off of Modbus and has some limitations.
Mapping from IO addresses (for example %IW0) is based off of the data type,
which is different from the DNP3 model of making a point available in different
representations (classes). The current approach results in data loss due to
truncation of values. For example, it is possible to have a IEC variable of REAL
type, but that cannot be mapped to a DNP3 point without first converting to fixed
point. This proposal is to address those issues.

Outside of Scope

For the purpose of this work, we assume the following it out of scope

• Mapping variables to different communcation drivers. That is, we will continue
  to use a global address space and assume the user only utilizes communcation
  protocol.
• Multiple indpendent instances of a communcation driver. For example, you can
  only have one DNP3 Outstation.

Solution

In IEC languages, PLC addresses define a direction (I, Q, M) and a data width
(see below). The data width should not be confused with data type.

Currently, OpenPLC generates glue arrays for some of these types, however that support
is incomplete. In order to support each of these data types without precision loss,
OpenPLC needs to support "glueing" to these types.

Adding support for all of these would substantially increase the memory footprint. Furthmore,
iterating over the arrays would often iterate over values are are un-mapped. Thus, a new glue
structure is proposed where we track both an index and the pointer to the value

template<T>
struct GlueVariable {
   std::uint16_t index;
   T* value;
   ... // See below for additional members
};

The index here defines the index for DNP3.

Declaration of the glue arrays need only allocate sufficent space for the mapped values and
writing to the communcation protocol is simply a matter of iterating over the all glue arrays, which
is known at compile time.

Receipt of a new value can be in different formats, for example receving a AnalogOutputDouble64 for
a value that is mapped to an IEC SINT. In this case, we must iterate over all glue arrays, find a matching
index. Because we know which glue array it is in, we can appropriately cast to the IEC data type. The
GlueVariable structure can then define cast operations for each data width available from DNP3.

template<T>
struct GlueVariable {
   std::uint16_t index;
   T* value;
   T as(const std::uint8_t v) { return (T)v; }
   ... // and so on for other DNP3 types
};

Searching over the arrays incurs a performance penalty which can be addressed in the future by with an
additional array of pointers to appropriate glue variable slots. I believe it is possible to handle the
casting in this case, but that is for future work.

Intergration with Modbus and Periperals

The current design for integration Modbus and periperals is based on glue variables that are of fixed
size (and interation over those in order to discover mappings to values). In order to limit the scope
of this work, we will keep that approach in place and allow DNP3 to work with variables in either
format.

Offsets

The current DNP3 mapping allows for the user to supply an offset - offsets define a runtime operation
to change the mapping index to the glue arrays. This will continue to be supported, however, only for
the data types (not data widths) that current support this type.

DNP3 Types

DNP3 supports different representations for a single data point. For example object group 40 (analog output
status) may be represented as a 32-bit signed integer, 16-bit signed integer, single-precision floating point,
and double-precision floating point. The same logical point can also be represented with object group 41, 42
and 43.

For "simplicity", we define the following as the default object groups when publishing data over DNP3:

(1) BOOL values that are only partially mapped are supported for each item in the 8-bit field that has an address.

The following types will not be mapped to DNP3 - consequently there is no glue mapping required for these.

Future Work

This proposal clearly does not address all of the needs for a complete DNP3 implementation. The goal is to enable sane defaults with the option of allowing further user-customization in the future either from the OpenPLC web front end, configuration files, or the PLCOpen project file. Furthermore, the focus is on a subset of data types that define some useful applications - these won't address all needs, but can be improved upon in the future.