emodbus / emodbus Goto Github PK

As I wrote I am planning to add a push service, that is to collect responses in defined intervals and send those out via UDP. I am considering some design details I would like to get your opinion.

All your comments are welcome indeed!

Structure

The push service will be attached to a ModbusServer - regardless which type it is. It will use the server's localRequest() interface to gather data from the local server, or, if the server in fact is a ModbusBridge, from all servers attached to the bridge as well.

It will use one single UDP client (WiFi or Ethernet) to push, as no connections have to be maintained.

Pushes will be organized in "jobs", that have a target host/port, a run interval and a number of repetitions as attributes, plus a unique token to identify the job. Each job has a list of requests whose responses are collected to be pushed as one packet.

A push packet has the job token, the repetition count and the current time, a list of response lengths and the responses proper as data. Thus a recipient can tell the order, reference time etc. of the push packet received and f.i. cope with packet losses with this information.

The size of a push packet is determined when defining the job - but must not be larger than the MTU size minus UDP header data. A response not fitting any more in the push packet will have a response length of 0 in the table.

We will not know in advance the size of all the responses in a job, if we do not restrict the function codes. I do not know yet if that is sensible, but I tend to think so - on one hand I cannot see what sense a repeated WRITE_HOLD_REGISTER with necessarily constant values may make, but on the other hand we do not know what a user-defined FC will do at all, that makes pretty sense when called again and again.

Two tasks will be run when the push service is started, one maintaining the jobs list and doing the UDP pushes (the "pusher" task), another gathering the responses for requests that are due (the "requester" task).

Job request

The smallest unit of a push "job" is a single request. It has a serverID, functionCode, data and len data set, that exactly fits the signature of the localRequest() call. The response is a part of the job request, as it may be used multiply (described below).

A request has a "due" flag, protected by mutex, that is used to signal a new response is required. The response is protected by the mutex as well to prevent a job reading the response data while it is refreshed.

A request may be used in more than one job; if the identical request is required a second time, it will have its "user count" incremented. If a job is deleted, the requests used by it are either deleted as well (if only used by the one job) or the count decremented.

To avoid unnecessary duplicate request execution, a request will have to be called with the shortest interval of the jobs using it. In addition, if a job is requiring the response, no new request will be started, if the last response is not older than some definable latency time, but the last response will be used.

To give you an example: assume three jobs J3, J5 and J10, running with intervals of 3, 5 and 10 seconds, respectively, that all will need some request R. The latency shall be 2 seconds.
When starting the job, R is run once for J3. J5 and J10 will use the response, as time has not advanced yet.
At second 3, R is run again for J3, since the last call is 3 seconds ago - longer than the 2 seconds latency.
At second 5 the request is not run for J5, as the last response is 2 seconds old only.
Second 9 sees another run for J3, but in second 10 J10 will get the same response again.
Second 12 R is run for J3, second 15 again, but not for J5 etc.

In effect the job with the shortest interval will have the most actual data, whereas those with longer intervals will have data outdated up to the latency time as a maximum. In return you will save quite a number of requests.

Pusher task

The pusher task will loop over the jobs list to see if one is due. If so, it will take the currently held responses of the requests for the job and build up the push packet. This way the jobs will have a push almost at the time their interval mandates, for the cost of slightly outdated data (as described above)

The pusher task will also set requests to 'due' if the responses need to be refreshed.

Requester task

The requester task will loop over the requests to see if one is marked "due", then call ``localRequest()```to get a fresh response. The response will be swapped with the old one upon completion. This swap and the reset of the "due" flag is protected by the mutex.

Main task

The push service instance will create new jobs, add requests to it, start and stop jobs and delete finished jobs with their further unused requests. The whole service can be started and stopped.
Requests will only be accepted if they are addressing one of the serverIDs served by the ModbusServer/ModbusBridge the push service is associated to.

Jobs options

• A common application will be a device that, once configured, will have just a few fixed push jobs throughout its lifetime. It would be stupid to have to recompile the firmware every time a job may have to be modified, so it could be handy to have a job definition as data. This job definition could be saved in a SPIFFS file to be loaded at startup, or be received over air.
• My bridge has a special "local" job that is used to store preset requests. These requests are used to continuously display current values of attached servers on the tiny OLED screen. One can modify the requests on the device and store the changes.
• A dedicated user function code is used to start new jobs, transporting the job definitions in the request data. For security reasons I restricted the jobs to a defined repeat count (no unlimited repeats), the interval to at least 2 seconds and to only have the sender of the request as the target for the push packets.

I will work on ModbusServerRTU next. This will require a different setup with another RTU Client doing requests I do not have yet.

• The code is not yet cleanly formatted, cpplint will have lots of complaints.
• Test statements, Serial output and bad language have to be taken out

Our code has been using the data pointer/data length idiom for the base classes designed earlier, while the more recent code makes more use of the std::vector container.
The overhead of vector is minimal, while providing standard access methods as well as the data() / size() construct that is identical to the older idiom.
So I plead for a refactored ModbusMessage as a std::vector<uint8_t> and all derived from that.
Major task, but IMHO worth it.

What's your opinion?

The name "TheBase" I chose initially is not very lucky, so I used ModbusClient in the meantime in the files to denote the library.

Where should this reside in the end? We have a few options:

• keep the ESP32ModbusMasterUnified organization an create a new public repository ModbusClient for it, that will hold all the public files, knowingly keeping the "Master"... 😁
• create a new repository in your Github account
• do likewise in my Github account.

The library.properties and library.json files both are expecting a mail address of the maintainer at least. I have a matching @[email protected] address I could provide for that. That account is mostly used to deflect spam, though, so I might be missing mails.

What is it used for?

I do see usage for ModbusTCP::errorResponse but not for ModbusTCP::generateErrorResponse.

The front page README.MD is still missing.

It should contain:

• Disclaimers
• Basic description of ModbusRTU and ModbusTCP
• Example code snippets
• API description in detail (public only)
  • ModbusRTU myObj(HardwareSerial& serial, int8_t rtsPin = -1, uint16_t queueLimit = 100)
  • void myObj.begin(int coreID = -1)
  • ModbusTCP myObj(Client& client, uint16_t queueLimit = 100)
  • ModbusTCP myObj(Client& client, IPAddress host, uint16_t port, uint16_t queueLimit = 100)
  • TCP: bool myObj.setTarget(IPAddress host, uint16_t port)``
  • bool myObj.setTimeOut(uint32_t TOV)
  • void myObj.onDataHandler(MBOnData handler)
  • void myObj.onGenerateHandler(MBOnGenerate handler)
  • void myObj.onErrorHandler(MBOnError handler)
  • uint32_t myObj.getMessageCount()
  • enum ModbusClient::Error
  • enum ModbusClient::FunctionCode
  • ModbusError myErr = addRequest(...
  • MBOnData and MBOnError
  • Error myObj.addRequest(uint8_t serverID, uint8_t functionCode, uint32_t token = 0)
  • Error myObj.addRequest(uint8_t serverID, uint8_t functionCode, uint16_t p1, uint32_t token = 0)
  • Error myObj.addRequest(uint8_t serverID, uint8_t functionCode, uint16_t p1, uint16_t p2, uint32_t token = 0)
  • Error myObj.addRequest(uint8_t serverID, uint8_t functionCode, uint16_t p1, uint16_t p2, uint16_t p3, uint32_t token = 0)
  • Error myObj.addRequest(uint8_t serverID, uint8_t functionCode, uint16_t p1, uint16_t p2, uint8_t count, uint16_t *arrayOfWords, uint32_t token = 0)
  • Error myObj.addRequest(uint8_t serverID, uint8_t functionCode, uint16_t p1, uint16_t p2, uint8_t count, uint8_t *arrayOfBytes, uint32_t token = 0)
  • Error myObj.addRequest(uint8_t serverID, uint8_t functionCode, uint16_t count, uint8_t *arrayOfBytes, uint32_t token = 0)
  • uint16_t ModbusMessage::addValue(uint8_t *target, uint16_t targetLength, T v)

The RTU version requires a feature currently not available in the Arduino framework. A workaround is suggested in the documentation but this requires changing Arduino source files. (https://emodbus.github.io/requirements#important-note-for-using-the-rtu-bridge-server-and-client)

Platformio IDE has the possibility to update the file automatically: https://docs.platformio.org/en/latest/projectconf/advanced_scripting.html#override-package-files

Feature: create sample scripts for eModbus.

Set up a working skeleton class for ModbusServerTCP.

The ECHO_RESPONSE constant does not fit the Modbus standard response to function code 0x10 (write multiple registers), as it will return the complete request. The standard requires only the address and number of registers being responded.

Just an idea...

Modbus servers mostly are responding to a writing FC by echoing the request back. This always was nagging me, as I need to check the response against the request in full length.

How about an option to have that done in the lib and bring back an Error response instead?

One could have a call like translateWriteResponses(Error e) ; to have the lib compare the received response to the request and send back the chosen Error code (SUCCESS, ACKNOWLEDGE or whatever seems appropriate) in an Error response. This could be stopped by another call noTranslateWriteResponses();

This will catch any response being identical to the request, though, even unintentionally, but I reckon that will not happen in reality.

There is a memory leak in arduino-esp32's WiFi library - documented here.

I did not yet understand why it will not be fixed, but it definitely is still there and is eating up 100-400 bytes at each connect().

When a bridge receives the response from a remote server, the server id is not masked with the server id known to the original requester.

I am out of means at the moment; perhaps you can shed some light on me.

I am running my test sketch with both RTU server and RTU client on one device, running different tasks. In loop() I am firing two test requests every twenty seconds:

    uint8_t sid = random(1,3);
    uint8_t fc = random(3,5);
    uint16_t addr = random(1,120);
    uint16_t wrds = random(0, 20);
    Serial.printf("Request: %02X %02X %04X %04X\n", sid, fc, addr, wrds);
    e = MBclient.addRequest(sid, fc, addr, wrds, (uint32_t)0x12345678);
    if (e != SUCCESS) {
      ModbusError me(e);
      Serial.printf("addRequest error %02X - %s\n", (int)me, (const char *)me);
    }
    delay(5);
    e = MBclient.addRequest(sid, 0x07, (uint32_t)0xFFEEDDCC);
    if (e != SUCCESS) {
      ModbusError me(e);
      Serial.printf("addRequest error %02X - %s\n", (int)me, (const char *)me);
    }

The first is using some random parameters to eventually provoke error responses, the second is a fixed one for serverID/FC=0x07, that is not served on that server. The expected response would be an ILLEGAL_FUNCTION error delivered to the onError callback.

I spiced the whole code with numerous print statements. Both send and receive functions got an additional label parameter to know if client or server had been calling them.

An example result is the following. I annotated each line to let you know when it is printed out.

SRV REQ timeout=20000

This is the server receive start telling us that the timeout is 20 seconds for it.
Now the main loop is generating a request:

Request: 01 03 003B 0005      
01 03 00 3B 00 05 Request sent

The second line is printed out immediately after picking the request from the queue. Next is the begin of the send function in RTUutils, stating the time since the last message was seen (20+ seconds) and the length of the packet to be written to Serial:

20050668µs, len=6 - CLN REQ   

Since the send does append the CRC, the next message from the server's receive is that it has caught 8 bytes, starting with 0x01 - our request obviously:

SRV REQ rv: 8 01

The server gets it, calls the callback for FC 0x03 and invokes send to give back the response - 13 bytes in length (5 words requested plus length byte plus FC plus serverID):

01 03 00 3B 00 05 F4 04 Requested

send gets it and sends it.

5780µs, len=13 - SRV RSP

The client in the meantime has called receive, indicated by a shorter timeout of 2 seconds. The server has started the receive process again (20 seconds timeout):

CLN RSP timeout=2000
SRV REQ timeout=20000

The client's receive call returns 15 bytes - the 13 of the response plus the CRC the server's send has added:

CLN RSP rv: 15 01

The client takes the response and calls the onData callback, that prints it:

     onData:  server:01, FC:03, token=12345678
     01 03 0A 74 75 76 77 78 79 7A 7B 7C 7D 

The client processes the next queue entry - the 0x07 FC that can not be served. It is given to send as 2 bytes, send adds the CRC and the server's receive gets 4 bytes:

01 07 Request sent
2240µs, len=2 - CLN REQ
SRV REQ rv: 4 01

The server checks the request and decides on ILLEGAL_FUNCTION:

01 07 41 E2 Requested
01 87 01 illegal FC

It sends the 3-byte error response:

3236µs, len=3 - SRV RSP

The client has started again the receive call (timeout 2 seconds), whereas the server again starts listenimng (timeout 20 seconds):

CLN RSP timeout=2000
SRV REQ timeout=20000

Now something has happened that slips me!
The client's receive returns just one byte: 0xE0=TIMEOUT:

CLN RSP rv: 1 E0

Since the message has been sent, as we saw above, the client's receive must have missed it - but why?
Next oddity: at this point I let the Client print out the error packet it has just constructed. Code is:

response = new RTUResponse(3);
        response->add(request->getServerID());
        response->add(request->getFunctionCode() | 0x80);
        response->add(rv[0]);
        Serial.printf("SRV got error: ");
        for (uint16_t i = 0; i < 3; ++i) {
          Serial.printf("%02X ", response->data()[i]);
        }
        Serial.println();

But as you can see, The FC in the request it should have been sending (and has sent - see above) out of a sudden ist the error code! The next byte 0xFB is something arbitrary, most probably found in the heap at that time:

SRV got error: 01 E0 FB

This then is not recognized as Error message, although the "FC" has the 0x80 bit set, but given to the onData calback instead.

     onData:  server:01, FC:E0, token=FFEEDDCC
     01 E0

It looks like the request the client is processing is destroyed at some point, that closely is related to the erroneous function code in the request. This does not happen if I am sending servable requests in sequence.

I committed the code as is, with all print statements in etc., to give you a chance to check it.

I cannot use the "ModbusBridgeRTU", the packets are sent correctly but the reply message that is sent by the TCP server does not reach the RTU client.
Please help me, and congratulations for this very useful library!!

MY CODE:

  #define LOCAL_LOG_LEVEL LOG_LEVEL_6
  #include "Logging.h"
  #include "modbusBrigeTasks.h"
  
  char ssid[] = "****";                     // SSID and ...
  char pass[] = "****";                     // password for the WiFi network used

  IPAddress serverIP = {192,168,43,100};
  uint16_t port = 502;                      // port of modbus server
  
  // Create a ModbusTCP client instance
  WiFiClient theClient;
  ModbusClientTCP myClient(theClient,5000);
  // Create server
  ModbusBridgeRTU myBridge(Serial2, RS485_EN);
  
//function in the setup()
  void modbusTasks_begin(){
  
      // Connect to WiFi
      WiFi.begin(ssid, pass);
      delay(200);
      while (WiFi.status() != WL_CONNECTED) {
          Serial.print(". ");
          delay(500);
      }
      IPAddress wIP = WiFi.localIP();
      Serial.printf("WIFi IP address: %u.%u.%u.%u\n", wIP[0], wIP[1], wIP[2], wIP[3]);
  
      // Set TCP Modbus message timeout to 2000ms
        myClient.setTimeout(2000,0);
  
      // Start ModbusTCP background task
        myClient.begin();
  
      // Define and start RTU bridge
        myBridge.attachServer(1, 1, ANY_FUNCTION_CODE, &myClient, serverIP, port);
  
      // Check: print out all combinations served to Serial
        myBridge.listServer();
  
      // Start the bridge. Port 502, 4 simultaneous clients allowed, 600ms inactivity to disconnect client
        myBridge.start();
  
  }

There is also a video of the test;

Provide user-defined float and double orders: I can imagine there will be Modbus devices in the wild not taking care to use a portable format.
There could be an optional bitmap as a third parameter in the add and get functions, consisting of an ORed map from

`SWAP_NIBBLE`
`SWAP_BYTE`
`SWAP_REGISTER`
`SWAP_WORD` (double only)

The default will be 0, resulting in the "pure IEEE754" sequence being used.

With these, from the normalized IEEE754 order any user-defined sequence can be achieved per call.

float Aa Bb Cc Dd for instance with SWAP_BYTE | SWAP_REGISTER applied to it would end up as Dd Cc Bb Aa.

Can you have a look at a problem that I am not currently able to understand or solve?

I added a retry mechanism to handleConnection() in ModbusTCP.cpp. In case of TIMEOUT or IP_CONNECTION_FAILED I give the same request another round by simply not popping it from the queue, so that the next loop will fetch the same entry again.

Unfortunately this seems to crash the ESP for a reason I do not see. In the following Serial Monitor output you can see the execution of the first 10 of 16 consecutive requests, fired with maximum possible speed. The tenth (Token: A) is running into a TIMEOUT and is tried again. That is successful (the response is shown), but then disaster strikes:

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:8896
load:0x40080400,len:5828
entry 0x400806ac
__ OK __
Resetting Wiz W5500 Ethernet Board...  Done.
My IP address: 192.168.178.73
.WIFi IP address: 192.168.178.74
TCP request bridge, 20, 0x03, continuous 32 byte blocks
   Token: 1 Length: 6
-----
Request 14 03 00 01 00 10
   Token: 1 Length: 35
Response 14 03 10 48 21 00 02 66 92 00 00 05 1E 00 09 00 03 C0 A8 C7 28 01 F6 FF FF FF 00 C0 A8 C7 01 C0 A8 C7 01
   ServerID: 20, FC: 3 Token: 1 Length: 35
   Message: 14 03 10 48 21 00 02 66 92 00 00 05 1E 00 09 00 03 C0 A8 C7 28 01 F6 FF FF FF 00 C0 A8 C7 01 C0 A8 C7 01
   Token: 2 Length: 6
Request 14 03 00 11 00 10
   Token: 2 Length: 3
Response 14 83 EB
   Error: TCP header mismatch Token: 2
   Token: 3 Length: 6
Request 14 03 00 21 00 10
   Token: 3 Length: 35
Response 14 03 10 01 02 0C 25 00 02 00 03 01 02 0B DB 00 02 00 03 14 07 00 14 00 02 00 07 01 02 0C 25 00 02 00 03
   ServerID: 20, FC: 3 Token: 3 Length: 35
   Message: 14 03 10 01 02 0C 25 00 02 00 03 01 02 0B DB 00 02 00 03 14 07 00 14 00 02 00 07 01 02 0C 25 00 02 00 03
   Token: 4 Length: 6
Request 14 03 00 31 00 10
   Token: 4 Length: 35
Response 14 03 10 00 00 00 00 00 00 00 00 04 02 00 02 00 02 00 03 04 02 00 04 00 02 00 03 04 02 00 06 00 02 00 03
   ServerID: 20, FC: 3 Token: 4 Length: 35
   Message: 14 03 10 00 00 00 00 00 00 00 00 04 02 00 02 00 02 00 03 04 02 00 04 00 02 00 03 04 02 00 06 00 02 00 03
   Token: 5 Length: 6
Request 14 03 00 41 00 10
   Token: 5 Length: 35
Response 14 03 10 04 03 00 10 00 02 00 08 04 01 00 12 00 04 00 08 04 01 00 19 00 04 00 03 00 00 00 00 00 00 00 00
   ServerID: 20, FC: 3 Token: 5 Length: 35
   Message: 14 03 10 04 03 00 10 00 02 00 08 04 01 00 12 00 04 00 08 04 01 00 19 00 04 00 03 00 00 00 00 00 00 00 00
   Token: 6 Length: 6
Request 14 03 00 51 00 10
   Token: 6 Length: 35
Response 14 03 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 06 00 60 00 02 00 03 00 01 5F 62 37 96 5F 64
   ServerID: 20, FC: 3 Token: 6 Length: 35
   Message: 14 03 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 06 00 60 00 02 00 03 00 01 5F 62 37 96 5F 64
   Token: 7 Length: 6
Request 14 03 00 61 00 10
   Token: 7 Length: 35
Response 14 03 10 B8 94 00 02 53 65 70 20 31 36 20 32 30 32 30 00 52 53 34 38 35 2F 54 43 50 2D 42 72 69 64 67 65
   ServerID: 20, FC: 3 Token: 7 Length: 35
   Message: 14 03 10 B8 94 00 02 53 65 70 20 31 36 20 32 30 32 30 00 52 53 34 38 35 2F 54 43 50 2D 42 72 69 64 67 65
   Token: 8 Length: 6
Request 14 03 00 71 00 10
   Token: 8 Length: 35
Response 14 03 10 20 20 20 00 6D 69 71 31 40 67 6D 78 2E 64 65 20 28 63 29 32 30 32 30 00 00 00 07 D0 68 74 66 62
   ServerID: 20, FC: 3 Token: 8 Length: 35
   Message: 14 03 10 20 20 20 00 6D 69 71 31 40 67 6D 78 2E 64 65 20 28 63 29 32 30 32 30 00 00 00 07 D0 68 74 66 62
   Token: 9 Length: 6
Request 14 03 00 81 00 10
   Token: 9 Length: 35
Response 14 03 10 73 76 2E 73 70 64 6E 73 2E 6F 72 67 00 74 65 6D 65 2E 64 65 00 00 00 00 00 00 00 00 21 36 FF E0
   ServerID: 20, FC: 3 Token: 9 Length: 35
   Message: 14 03 10 73 76 2E 73 70 64 6E 73 2E 6F 72 67 00 74 65 6D 65 2E 64 65 00 00 00 00 00 00 00 00 21 36 FF E0
   Token: A Length: 6
Request 14 03 00 91 00 10
   Token: A Length: 3
Response 14 83 E0
Retry on timeout...
   Token: A Length: 6
Request 14 03 00 91 00 10
   Token: A Length: 35
Response 14 03 10 00 03 00 05 01 02 0C 03 00 02 00 03 01 02 0B C1 00 02 00 03 01 02 0C 25 00 02 00 03 01 02 B0 2B
   ServerID: 20, FC: 3 Token: A Length: 35
   Message: 14 03 10 00 03 00 05 01 02 0C 03 00 02 00 03 01 02 0B C1 00 02 00 03 01 02 0C 25 00 02 00 03 01 02 B0 2B
Guru Meditation Error: Core  0 panic'ed (InstrFetchProhibited). Exception was unhandled.
Core 0 register dump:
PC      : 0x00000000  PS      : 0x00060030  A0      : 0x800893a0  A1      : 0x3ffcf890
A2      : 0x3ffc1240  A3      : 0x3ffd2170  A4      : 0x00000000  A5      : 0x00000000
A6      : 0x400d8d8c  A7      : 0x00000014  A8      : 0x800d8574  A9      : 0x3ffcf870
A10     : 0x3ffd2170  A11     : 0x3ffc1c50  A12     : 0x3ffd0a24  A13     : 0x37350da2
A14     : 0x0000000a  A15     : 0x3ffc1864  SAR     : 0x00000010  EXCCAUSE: 0x00000014
EXCVADDR: 0x00000000  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xffffffe2

Backtrace: 0x00000000:0x3ffcf890 0x4008939d:0x3ffcf8d0
  #0  0x00000000:0x3ffcf890 in ?? ??:0
  #1  0x4008939d:0x3ffcf8d0 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

Rebooting...

The PC : 0x00000000 is an indication there might be a jump with an unset function pointer, but where would that be in the code?

// handleConnection: worker task
// This was created in begin() to handle the queue entries
void ModbusTCP::handleConnection(ModbusTCP *instance) {
  const uint8_t RETRIES(2);
  uint8_t retryCounter = RETRIES;
  bool doNotPop;
  uint32_t lastRequest = millis();

  // Loop forever - or until task is killed
  while (1) {
    // Do we have a reuest in queue?
    if (!instance->requests.empty()) {
      // Yes. pull it.
      TCPRequest *request = instance->requests.front();
      doNotPop = false;

      // onGenerate handler registered?
      if (instance->onGenerate) {
        // Yes. Send request packet
        instance->onGenerate("Request ", request->data(), request->len(), request->getToken());
      }
      // Empty the RX buffer - just in case...
      while (instance->MT_client.available()) instance->MT_client.read();

      // check if lastHost/lastPort!=host/port off the queued request
      if (instance->MT_lastHost != request->targetHost || instance->MT_lastPort != request->targetPort) {
        // It is different. If client is connected, disconnect
        if (instance->MT_client.connected()) {
          // Is connected - cut it
          instance->MT_client.stop();
          delay(1);  // Give scheduler room to breathe
        }
      }
      else {
        // it is the same host/port. Give it some slack to get ready again
        while (millis() - lastRequest < 100) {
          delay(1);
        }
      }
      // if client is disconnected (we will have to switch hosts)
      if (!instance->MT_client.connected()) {
        // It is disconnected. connect to host/port from queue
        instance->MT_client.connect(request->targetHost, request->targetPort);

        delay(1);  // Give scheduler room to breathe
      }
      // Are we connected (again)?
      if (instance->MT_client.connected()) {
        // Yes. Send the request via IP
        instance->send(request);

        // Get the response - if any
        TCPResponse *response = instance->receive(request);

        // onGenerate handler registered?
        if (instance->onGenerate) {
          // Yes. Send request packet
          instance->onGenerate("Response ", response->data(), response->len(), request->getToken());
        }
        // Did we get a normal response?
        if (response->getError()==SUCCESS) {
          // Yes. Do we have an onData handler registered?
          if(instance->onData) {
            // Yes. call it
            instance->onData(response->getServerID(), response->getFunctionCode(), response->data(), response->len(), request->getToken());
          }
        }
        else {
          // No, something went wrong. All we have is an error
          if (response->getError() == TIMEOUT && retryCounter--) {
            Serial.println("Retry on timeout...");
            doNotPop = true;
          }
          else {
            // Do we have an onError handler?
            if(instance->onError) {
              // Yes. Forward the error code to it
              instance->onError(response->getError(), request->getToken());
            }
          }
        }
        //   set lastHost/lastPort tp host/port
        instance->MT_lastHost = request->targetHost;
        instance->MT_lastPort = request->targetPort;
        delete response;  // object created in receive()
      }
      else {
        // Oops. Connection failed
        // Retry, if attempts are left or report error.
        if (retryCounter--) {
          instance->MT_client.stop();
          delay(10);
          Serial.println("Retry on connect failure...");
          doNotPop = true;
        }
        else {
          // Do we have an onError handler?
          if(instance->onError) {
            // Yes. Forward the error code to it
            instance->onError(IP_CONNECTION_FAILED, request->getToken());
          }
        }
      }
      // Clean-up time. 
      if (!doNotPop)
      {
        // Safely lock the queue
        lock_guard<mutex> lockGuard(instance->qLock);
        // Remove the front queue entry
        instance->requests.pop();
        retryCounter = RETRIES;
      }
      // Delete request
      delete request;   // object created from addRequest()
      lastRequest = millis();
    }
    else {
      delay(1);  // Give scheduler room to breathe
    }
  }
}

Any idea?

Have to change all Serial.print.... code into log_x().

I am not satisfied at all with the TCP communications. Neither EthernetClient nor WiFiClient are working without problems.

I am using test code like this:

    Serial.println("TCP request bridge, 4, 0x03, 1/10");
    WiFiBus.setTarget(IPAddress(192, 168, 178, 77), 6502);
    ModbusError e = WiFiBus.addRequest(4, 3, 1, 10, token++);
    if(e!=SUCCESS) {
      Serial.print("Error: ");
      Serial.println((const char *)e);
    }
    Serial.println("-----");

WiFiBus is set up like

WiFiClient cB;
ModbusTCP WiFiBus(cB);
...
 WiFiBus.onDataHandler(&handleData);
  WiFiBus.onGenerateHandler(&handleRaw);
  WiFiBus.onErrorHandler(&handleError);
  WiFiBus.begin();

The Ethernet variant is likewise

EthernetClient cA;
ModbusTCP NetBus(cA);
...
  NetBus.onDataHandler(&handleData);
  NetBus.onGenerateHandler(&handleRaw);
  NetBus.onErrorHandler(&handleError);
  NetBus.begin();

After some successful requests initially, more and more requests are resulting in TIMEOUT or TCP_HEAD_MISMATCH. After a while that turns into IP_CONNECTION_FAILED and nothing works any more. I do not know what is happening there - may be the server is out of sockets, may be my test client is it. I suppose it is the former, since the WiFiClient is not able to connect as well if the EthernetClient was stuck before.

The connected() issue (connected() returns true even if the socket was closed on the server side) can be "solved" by sending a single character before the stop() call on the server side. Nobody seems to know why. I am trying to get @maxgerhardt to look into it - he is the author of the Ethernet library fork I am using.

To properly work in the Arduino IDE/PlatformIO, we need to add some files describing the library:

• keywords.txt for ArduinoIDE
• library.json for PlatformIO
• library.properties for ArduinoIDE

When issueing requests in quick sequence, TCPasync will send them as fast as can to the server. At least my servers will need a short time to work on the request and recover. As it is now, the first one or two requests are served, the remainder ends with timeouts.
I can see only the first requests being received from the WiFiClient, the rest is unnoticed. So I am afraid there may be a buffer overrun in the WiFiClient caused by the request storm.
I an few cases I received the next request's data with the current. I tried to notice the end of the packet at least by means of the len field in the TCP header, but that changed nothing.
Then I put an unconditional delay(200) in ModbusClientTCPasync::send() immediately before the send() and the problem was gone. With delay(100) I lost every second request to TIMEOUT, by the way.
It would be good to have a configurable recovery interval time with TCPasync as well. I have that in the plain TCP server code from the beginning, as I expected my devices to need it.

Euh, dunno what I've just done...

Is there anything screwed up by my last git actions?

Describe the bug

When using the RTU06Example with an ESP32-WROOM (NodeMCU) Development Board and Modsim64 as a server. The CRC validation fails when reading registers. It is however able to write to registers without issue.

The connection is made via UART (CP2102) to a Windows host.

Expected behavior

It is expected that CRC validation would succeed in both contexts.

To Reproduce

Install Modsim64 (demo or equivalent)
Run Example

Example code

RTU06Example (accessed 29/12/2020)
https://github.com/eModbus/eModbus/blob/d0e5329e93290076d271f374ed13c654fbf00ef7/examples/RTU06example/main.cpp

I have also attempted to add CRC to a Modbus Message manually with no difference in the outcome:

        ModbusMessage msg = ModbusMessage(1, READ_HOLD_REGISTER, 10, 1);
        RTUutils::addCRC(msg);
        err = MB.addRequest(msg, Token++);
        if (err != SUCCESS)
        {
            ModbusError e(err);
            Serial.printf("Error creating request: %02X - %s\n", (int)e, (const char *)e);
        }

Additional context

I have tried using alternative serial configurations (SERIAL8E1, SERIAL8N2, etc.) to no success.

Modbus Messages recorded via Modsim64 (over UART):

CRC Counters:

Register written to successfully:

MCU Output:

Response: serverID=1, FC=6, Token=00000457, length=6:
01 06 00 0A 12 34 
Error response: E0 - Timeout
Response: serverID=1, FC=6, Token=00000459, length=6:
01 06 00 0A BE EF 
Error response: E0 - Timeout

EDIT: UPDATED incorrect modscan references to modsim.

Hi,
I was using and following the older modbus library by mr.bert, but had faced a tons of issues figuring out to read the messages and make it more literate. now that i am using the emodbus, trust me im tired to figure out how to have a basic example where i can read few holding registers from a slave device, which are in 32bit floating format. perhaps the slave is also an arduino device and the code is written by me, it is reading data on the master software on laptop, but i am tired to figure out how on earth am i supposed to be combining the way too literate documentation for decoding the message. yes i am getting the data, but how on earth can i make it just read a simple message "25.786" thats the temperature reading on holding register 40000 and 40001 .. its split into 2 16bit int.

humble request to add some really basic and literate examples in the code so that it is useful.

here is my slave code..
https://pastebin.com/LUs2NhXR

Here is the ESP32 Master Code (Which gives data, but not a readable one)
https://pastebin.com/bsCUdysP

apologies if i sounded rude, but its just whole 3 days of frustration coming out as i couldn't figure out any way to make this work.

thanks in advance.

The interface to the ModbusServerTCP classes has to be added to README.md.

Just as a place holder: the implementation of the ModbusTCP variant using AsyncTCP is still open.

Not sure of the use case, but in the constructor of PhysicalInterface, the number is incremented. Shouldn't we have a destructor decrementing this number?

It would be great to have some automatic test sketch. We cannot expect any interface to be available for the testing, so I propose a reduced scenario. This at least will tell us if the internal functionalities (error handling, checks, message generation) are working okay.

• have the sketch define one instance for each interface, without initializing the Serial or Client instances
• no begin() is called
• have a switch structure with 16 case slots. Each numbered slot will have one of the 2 * (7 + 1) = 16 generateXXX calls, including a check on the returned message or error
• have a data (file?) block of tests. Each test has a slot number (which generateXXX call is to be used), a matching set of parameters and a byte array of the expected resulting message. This may be a complete message or a single byte for the expected error code.
• the "slotted" generateXXX call will execute, then the result will be compared to the given array and a passed/did not pass returned.

This way we can add further tests at will, run the test sketch and check if we had "did not pass" returns in it.

What do you think?

The mutex should be placed outside the if statement:

now:

1. check size
2. lock queue
3. add if possible

better:

1. lock queue
2. check size
3. add if possible

https://github.com/ESP32ModbusMasterUnified/TheBase/blob/fc1c8f532d08e2f086933deb0620ad576f8ecc1e/src/ModbusTCP.cpp#L357

I ran the modbus RTU example and it logs the modbus message response, I am interested in retrieving the actual register values from the modbus device. How do I convert the hex dump into values I can use in other parts of the program. Thanks

Currently the ModbusClientTCPasync has a single constructor only that takes the IP address and port of the target Modbus server.

This requires to know the target at compile time already. It would be nice to have a way to create a ModbusClientTCPasync object without a target first and have a call to set it later - potentially from some configuration variables (IP and port). The already existing connect() method would come handy for that. Of course the object then must be protected to be started target-less.

I was just trying out the modbus RTU06 example and I noticed in the example, RX2=17, TX2=16, whereas the board that I have has these pins swapped. I have TX2=17, RX2=16. I amusing espressif dev kit. Can you confirm if the pins in the example are with respect to the esp32 or RS485 to TTL converter ?

Since there are a number of commonly used functions in both TCPasync and TCP, it seems to be reasonable to create a new class, derived from ModbusClient, that takes over these functions. ModbusClientTCP and ModbusClientTCPasync will be derived from that new class.

Function candidates to be moved to the new class ("ModbusClientTCPutil"?) are

• all addRequest calls
• all generateXXX calls (request and error response)
• vectorize
• makeHead

What else?

Cpplint brought up the point of requiring a Copyright notice in the files (and a license file to complement it).

I personally am in favour ot either the MIT license or GNU GPLv3. 0.

The Copyright line could be like

// Copyright 2020 by Michael Harwerth, Bert Melis and the contributors of ModbusClient (MIT license)

to cover the elementary requirements.

Do you agree?

The default CodeQL scan script is failing due to being unable to build a target.

I have no experiences with the Github actions, so could you please have a look, @bertmelis ?

We need to have the eModbus lib listed in PlatformIO - Registry

We need to prevent the code being deployed on devices without FreeRTOS (that is, xTask etc.). This should be done by having the preprocessor checking by #ifdef target system-defined values.
Supported MCUs: https://www.freertos.org/RTOS_ports.html
Interestingly, the ESP32 is missing on that list. Might be one of the "Contributed ports" mentioned.

RS.485 rx/tx flow control does not work
I am using ModbusClientRTU() to read Modbus energy meters. My RS.485 hardware is using an ESP32 pin to control the receive / transmit. The implementation in RTUtils.cpp RTUutils::send() does not work, because the RTS pin is switched only on for less than a millisecond. I uncommented serial.flush()...

Code in RTUtils.cpp, line 110

// serial.flush();
// Toggle rtsPin, if necessary
if (rtsPin >= 0) digitalWrite(rtsPin, LOW);

exchange with:

// 
// Toggle rtsPin, if necessary
if (rtsPin >= 0) {
   serial.flush();
   digitalWrite(rtsPin, LOW);
}

I am using a USER_DEFINED_42 function code on my smart plug's ModbusServerTCPasync to trigger writing data to EEPROM.
The TCP request packet is like

00 01 00 00 00 02 01 42

This is just the FC, no more data, and a valid Modbus request AFAIK.

The ModbusServerTCPasync does not react on it at all, though. If I will add another (illegal) byte to the request:

00 01 00 00 00 02 01 42 01

the request will be processed, ignoring the obsolete byte.

I must confess I did not find where this may be caused, so I hand it over to you, @bertmelis 😁

Can you please add an example for Modbus slave in serial mode. I'm new to Modbus and it will help me alot.

replace size_t by uint16_t for consistent type use in ModbusMessage.

Sometimes a request is issued with dedicated timeout and same host interval times, but different times are used.

I took the TCPasync example and added a ModbusServerWiFi on port 502 to it. I let the async client request on IP 127.0.0.1:502, which in principle works nicely. It looks like the WiFiServer is listening to 0.0.0.0 by default.

The exception is the missing reaction of TCPasync on remote connection closing - plainly nil.

I got an idle timeout in the TCP server, after which the connection is closed with stop(). In my test sketch I am requesting every 15s from the server. As long as the server's idle timeout is above the 15s, all requests are responded to nicely.

But if I drop the idle timeout to a value below the 15s, the server will intentionally close the connection, before the next request comes in. TCPasync does not notice the connection has been closed and is continuing sending a request every 15s, all resulting in a TIMEOUT error code.

[Updated to slightly different example cases and keeping track of already finished examples)
We need to do example sketches to include with the library. These should be as simple as possible.

I suggest doing one example each for these cases:

• ModbusTCP, WiFi: FC 0x03 (read holding registers) done

• ModbusTCP, Ethernet: FC 0x03 (read holding registers) done

• ModbusRTU: FC 0x06 (write single register) done

• ModbusRTU: FC 0x16 (write multiple registers) done

• ModbusRTU: generateRequest for FC 0x03 and generateErrorResponse done

• ModbusTCP: generateRequest for FC 0x10 and generateErrorResponse done

Any additions?

Can't seem to find it.

When building the test sketch anew, I noticed that a bunch of TCP test cases ended in "EB TCP header mismatch" instead of the expected (and simulated) result.

A closer inspection revealed, that a previous timeout test case caused the delayed delivery of the response for that test case. The next test case took the response as if it was caused be the test case's request and found a transaction ID mismatch.

This was happening because I removed the "read before send" pattern in the TCP client in the course of schasing the loopback bug. Before, I emptied the RX buffer blindly before sending a new request. Now whatever remains in the RX buffer will be read after the request has been sent.

I put the read-before-send back in in a slightly refined way: only if a connection is already open to the same target host, I will empty the RX buffer.

All is running okay again now, but I cannot decide that this will be harmless in all cases. Any response sent out of sync with the client will be dropped unnoticed - can that cause additional troubles?

With RTU the case seems clear: The client asks and the server responds in time, all else is forbidden and may be dropped regardless. (Is it? may there be more than one client requesting on the bus?)

With TCP we may have delayed responses for reasons the server cannot influence. With the solution as described above in place we treat that as if the server had not responded at all.

How is that handled with the Async client? There the implicit assumption is that responses are out of sync - do you match requests just by the transaction ID, regardless of the time passed? What is defining a timeout then?

Do you know why when I have an error in the translation ID in modbus TCP sometimes a pointer goes out of the heap area and the ESP32 reboot?

Which one is the Ethernet lib?
For platformio, the dependencies are listed in the json file. But currently there are none defined.

Now this might be tricky because it depends on the architecture/chip. I'll not be using Ethernet and you'll not be using Async.

for now, I'll add both otherwise it won't compile at all.

We do have a spot where we are spending to much time without yielding to the supervisor, so it seems:

E (3104188) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (3104188) task_wdt:  - IDLE0 (CPU 0)
E (3104188) task_wdt: Tasks currently running:
E (3104188) task_wdt: CPU 0: Modbus01TCP
E (3104188) task_wdt: CPU 1: IDLE1
E (3104188) task_wdt: Aborting.
abort() was called at PC 0x400d8c4b on core 0

Backtrace: 0x4008b588:0x3ffbe170 0x4008b7b5:0x3ffbe190 0x400d8c4b:0x3ffbe1b0 0x400845ed:0x3ffbe1d0 0x400efc86:0x3ffb9380 0x400ef49f:0x3ffb93a0 0x400ef4ba:0x3ffb93c0 0x400d3e3b:0x3ffb93e0 
0x400d2bd9:0x3ffb9410 0x400d3319:0x3ffb9430 0x400d333c:0x3ffb9460 0x400d3a37:0x3ffb9480 0x400d2a48:0x3ffb94b0 0x400d4c4b:0x3ffb94d0 0x400d4f53:0x3ffb9500 0x40088215:0x3ffb9550
  #0  0x4008b588:0x3ffbe170 in invoke_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/panic.c:707
  #1  0x4008b7b5:0x3ffbe190 in abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/panic.c:707
  #2  0x400d8c4b:0x3ffbe1b0 in task_wdt_isr at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/task_wdt.c:252
  #3  0x400845ed:0x3ffbe1d0 in _xt_lowint1 at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/xtensa_vectors.S:1154
  #4  0x400efc86:0x3ffb9380 in spiTransferBytesNL at C:\Users\Micha\.platformio\packages\framework-arduinoespressif32\cores\esp32/esp32-hal-spi.c:283
  #5  0x400ef49f:0x3ffb93a0 in SPIClass::transferBytes(unsigned char*, unsigned char*, unsigned int) at C:\Users\Micha\.platformio\packages\framework-arduinoespressif32\libraries\SPI\src/SPI.cpp:269
  #6  0x400ef4ba:0x3ffb93c0 in SPIClass::transfer(unsigned char*, unsigned int) at C:\Users\Micha\.platformio\packages\framework-arduinoespressif32\libraries\SPI\src/SPI.cpp:269
  #7  0x400d3e3b:0x3ffb93e0 in W5100Class::read(unsigned short, unsigned char*, unsigned short) at .pio\libdeps\az-delivery-devkit-v4\Ethernet\src\utility/w5100.cpp:460
  #8  0x400d2bd9:0x3ffb9410 in W5100Class::readSn(unsigned char, unsigned short, unsigned char*, unsigned short) at .pio\libdeps\az-delivery-devkit-v4\Ethernet\src/EthernetClient.cpp:168 
  #9  0x400d3319:0x3ffb9430 in W5100Class::readSnTX_FSR(unsigned char) at .pio\libdeps\az-delivery-devkit-v4\Ethernet\src\utility/w5100.h:286
  #10 0x400d333c:0x3ffb9460 in getSnTX_FSR(unsigned char) at .pio\libdeps\az-delivery-devkit-v4\Ethernet\src/socket.cpp:388
  #11 0x400d3a37:0x3ffb9480 in EthernetClass::socketSendAvailable(unsigned char) at .pio\libdeps\az-delivery-devkit-v4\Ethernet\src/socket.cpp:474
  #12 0x400d2a48:0x3ffb94b0 in EthernetClient::flush() at .pio\libdeps\az-delivery-devkit-v4\Ethernet\src/EthernetClient.cpp:168
  #13 0x400d4c4b:0x3ffb94d0 in ModbusClientTCP::send(TCPRequest*) at lib\ModbusUnified\src/ModbusClientTCP.cpp:136
  #14 0x400d4f53:0x3ffb9500 in ModbusClientTCP::handleConnection(ModbusClientTCP*) at lib\ModbusUnified\src/ModbusClientTCP.cpp:136
  #15 0x40088215:0x3ffb9550 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

Rebooting...

This is way down in the Ethernet code as I read it, so I am undecided what to do about it. The target host I was addressing at that time is behind a Powerline connection that is sometimes unreliable. May be that was the reason, but it is not satisfying having the complete sketch going down on that.

The current implementation of the onGenerate callback is only useful while we are testing, since the callbacks will be served for every single request, enabling an easy printout of what is going out and coming in.

I would like to have calls similar to the addRequest ones, that will only do the parameter checking, set up of the message proper and the respective additional data (TCP header or CRC) and then serve a callback that will get the complete message in a uint8_t *data, uint16_t dataLen buffer. After the callback has ended, the message etc. is destroyed again and will not be placed on the request queue.

We may have the 7 generateRequest calls matching the addRequest ones, plus another generateErrorResponse call that will set up a correct response for a given Error. If there is no onGenerate callback registered, the calls will return with an error telling this.

The rationale behind is to give users a convenient way to generate correct Modbus messages to use elsewhere.