A better defined serial protocol for interacting with computer about t-962-improvements HOT 11 OPEN

Apparently the email reply did not go through late last night:

I agree, most definitely a really nice thing to have. Sending the periodic measurement/status report in some more defined format is definitely the easiest part, together with prefixing of other debug messages. Controlling the operations from a workstation requires something to actually listen to the serial port - that is currently not implemented.

Any ideas how such a protocol could look like? I haven't seen any standard protocol for this purpose.

Designing a TUI/GUI would be a completely different project (but definitely strongly connected).

from t-962-improvements.

I suppose there would be two possible routes to take: either the one I outlined above (ie. line-oriented protocol with short keywords, arguments separated with white space) or a more extensible hip protocol, such as one based on JSON.

The latter is actually pretty nifty as used by the TinyG CNC controller firmware, but I suppose it might create some opposition having the appearance of a bloaty web2.0 protocol ;). Though I must give it that probably the needs of this project are relatively small.

Perhaps at the very minimum the protocol could be something like (similar to G-code):

client->T962: [command] [rv] [rv]\r\n
T972->client: [response] [rv] [rv]\r\n

As the interaction is going to be relatively low-bandwidth and low-latency and rarely more than 1 message in-the-flight and no out-of-order processing, I think it's not required to use 'tags' for identifying which command the response is for. However, messages that are produced asynchronusly (say, current temperature indicated every x seconds) must be different from messages that are responses to commands, as this simplifies the client code that waits for the response to its particular command. All commands should have a response, even if it's just 'OK', so the client knows which of its commands have been processed.

Commands and responses could be single-letter as far as I can see, unless a more verbose protocol is desirable.

Commands of interest:
-> S0 p0 t0 v200.0 - Set profile 0, time slot 0, value 200.0
<- R - "ok"
-> G0 p0 t0 - Get profile 0, time slot 0
<- R v200
-> G1 t0 - Get temperature
<- R v25.0

Asynchronous responses:
<- A1 t0 v25.0 - Asynchronous temperature information
<- A2 s100 g120 t110.0 - Cycle in progress, time=100 seconds, goal temperature 120.0, temperatute 110.0

Other commands of interest with the basic R response:
-> C1 p0 - Start cycle, profile 0. This probably is best to require enabling from device itself..
-> C0 - Stop cycle
-> B1 t50 s3600 - Start baking at set temperature for one hour
-> B0 - Stop baking

Or is this too terse? Multi-value responses are probably a big awkward. R v25.0 v26.0?

One point is also whether the communication should be checksummed. If so, it could be the last argument in the line (say: c35134) indicating the checksum of the message prior to the c character. Messages that have an invalid checksum could be discarded with a reply indicating it. This approach isn't fool-proof if it doesn't require all messages to contain such a checksum - perhaps it could require a checksum after the first checksummed message has been received.

In addition to corrupted messages there could be fully missing messages. Those could be detected with message counters (ie. n42 before the checksum).

from t-962-improvements.

I'm still waiting on my oven (now at least in the courier's system) so can't contribute much code, but a few comments from similar projects:

Is ASCII a requirement? If you are willing to accept binary things are much easier IMHO... I've got some code that I've used for a variety of systems which manages building received messages without blocking the main event loop. This also might make downloading/editing the profiles much easier, since you could basically just copy byte-by-byte into the desired location (be EEPROM or SRAM).

The protocol basically uses 'start of frame' (SOF) and 'end of frame' (EOF) bytes to determine message locations. Thus you could still send ASCII data randomly in printfs() (which could never contain the SOF character as it's not printable ASCII) for debugging without causing the computer to consider this as valid data, unless it interrupts an ongoing transmission. In fact the system could avoid sending any binary data until it's talked to in the binary format, such that you aren't polluting your terminal with binary crap.

I had planned on modifying my own version of the code to do this for plotting to a GUI. I've already got the C & Python code for doing this from other projects so was the easiest for me since it's just a small porting effort!

from t-962-improvements.

I don't see ASCII as a hard requirement, though it would allow more easily implementing a client separately, as for regular testing purposes plain terminal program would be sufficient.
Ability to directly write to the memory does sound nice, but it would mean that the offsets would need to be either hardcoded or given symbolic names for the purposes of the protocol. In addition, at least the EOF would need to be escaped when sending binary streams, though it would of course complicate things only very slightly.
I agree that without a frame separator mixing plain printf output is more difficult, but not by much, as such messages could simply be prefixed with a special asynchronous message prefix, such as ! or the space.
As for sending/receiving profiles, there could be a much more succinct command for handling the whole profile in one go, for example one based on hexadecimal values. For instance, binary data could be expressed as b[hex string here] such as: s0 p0 b1020304050302010.

from t-962-improvements.

I agree with eras that ASCII is not a hard requirement, but the fact that it's so easy to parse (especially on the host-side), I would expect some ASCII protocol variant would be the easiest for people to interface with. The additional processing that ASCII imposes on the controller side is very slim. This is after all (for this purpose) a way overkill 32-bit ARM7 MCU clocked reasonably fast, so making the protocol defined by binary data representing the internal data structures does not make that much sense to me personally. It introduces unnecessary restrictions on how the data can be shuffled around in memory as well. As some sanity checking will have to be done on the data being sent over the serial port anyway it's not very difficult to massage the data during that process.

I was browsing for other projects doing graphing and some control host-side and one showed up in a few different places: MegunoLink (http://www.megunolink.com). This seems like a nice way to deal with both the control parts and graphing. Unfortunately it's not open source and it's Windows only. I couldn't find a matching open project, have any of you seen something similar?

from t-962-improvements.

I've coded a simple serial protocol in #51 (including plotting with python/matplotlib), but did not pay attention to this discussion. Any thoughts @eras @colinoflynn?

from t-962-improvements.

code > talk ;-)

I'd seen progress being made, and figured if you've got stuff working that's the best option. Can always hack onto it later if required.

from t-962-improvements.

Now that #51 is merged, lets do some talking. What kind of functionality should be added to the serial interface? Some thoughts:

• Bake should get timer functionality
• Dump profile to serial
• Update/load a profile

Besides that, replies of the oven are not really standardized now, sow polishing that would be another thing to consider doing.

from t-962-improvements.

The commands works well from the computer but has to be sent as one continuous stream, typing them in manually doesn't work as expected. Not a major issue when using the python wrapper but if talking to it using a terminal program you have to basically paste in what you want to send. Timer on bake is nice. That timer should ideally start ticking only when having reached the setpoint. This will make it possible to use this functionality as a pre-heat for the oven as well.

from t-962-improvements.

Good point, I tested with a program called cutecom which only sends the command after you press enter.

About the timer in bake function: If I pre-heat, the reflow profile will try to cool down the oven before heating it again. So pre-heat only makes sense if the profile starts with a relatively high temperature?

from t-962-improvements.

Aha, makes sense! I used my favorite terminal program "screen"!

Yes for pre-heat you don't want to heat the oven too much anyway, but letting the oven reach 50C and then staying there for a few minutes definitely makes a difference if the oven was very cold to begin with. The reflow profiles were all adjusted to start at 50C some time ago for the same reason, to avoid reflow starting to cool right from the start..

from t-962-improvements.