Different response from PICO in windows and ubuntu about rp2daq HOT 17 CLOSED

Thank you a lot. This is truly interesting; I will prioritize making it work on Windows reliably. BTW I already was testing rp2daq on Windows lately and encountered another issue (#5).

I didn't have to comment out portname line - could you please link what is the problem?

What do the msg headers look like when you get different bitwidth, as you write? The only supported bitwidths in the firmware are 8, 12 or 16 bits and the value should be hard-compiled, so any change in this value now rather indicates corrupted data flow...

from rp2daq.

Here is entire block header (24 byte) in windows and ubuntu, plus 10 first bytes of samples in each block:

It is weird, but randomly code works right on windows! I was wondering if you get same result as me on your windows machine too?
and here is the code

import time
import serial
import struct
from sys import platform
import sys
import tkinter


if platform == "linux" or platform == "linux2":
    port = "/dev/ttyACM0" # for ubuntu
elif platform == "win32":
    port = "COM6" # for windows
    
ser = serial.Serial(port=port, timeout=1)

ser.write(struct.pack(r'<BBB', 1, 0, 1)) 
time.sleep(.15) # 50ms round-trip time is enough

try_port = ser
   
id_data = ser.read(try_port.in_waiting)[4:] 

print ("connected to " + str(id_data))

dt = bytes([0x0A,0x04,0x10,0xe8,0x03,0x01,0x01,0x00,0x60,0x00])

ser.write(dt) 

time.sleep(.15) 

sizeToRead = 1500

cnt =0;

while True:
    header = ser.read(24)
    
    
    samples = ser.read(sizeToRead)
    print(f'{cnt:04d}'+ " th block, header (24 byte): "+ header.hex() + ", samples (first 10 bytes): "+ samples[:10].hex()  )
    
    cnt = cnt+1

Thanks again for the cool opensource project

from rp2daq.

As if the 9th block were not fully read on windows, so its payload data get wrongly interpreted as headers of the 10th and following messages. The firmware is probably the same as I use and it only depends on the OS - it might be buffer overflow problem.

From the last two bytes you transmit (0x60,0x00) I can see you are using fastest rate possible (500 ksps). Would you try changing these two bytes, e.g., to (0xC0,0x03) so that you measure only at 50 ksps?

from rp2daq.

As if the 9th block were not fully read on windows, so its payload data get wrongly interpreted as headers of the 10th and following messages. The firmware is probably the same as I use and it only depends on the OS - it might be buffer overflow problem.

From the last two bytes you transmit (0x60,0x00) I can see you are using fastest rate possible (500 ksps). Would you try changing these two bytes, e.g., to (0xC0,0x03) so that you measure only at 50 ksps?

I can confirm that after replacing last two bytes from 0x60,0x00 to 0xC0,0x03, the problem is fixed on windows.

from rp2daq.

OK, thanks.

I will devise a new solution to the inter-process communication, replacing the Queue object with a Pipe, and get back to you.

from rp2daq.

thanks,
I just found out if there is a small gap in reading data in PC side, this do happen. I think it is something with windows, not with your code. so changing the code will not do any help...
Is there any other way to connect to PICO with LibUSB etc. instead of serial port?

from rp2daq.

There are few questions to be resolved experimentally, and right now I have no time to run such experiments on Windows.

Where does the data corruption come from? It looks like buffer/queue overflow, which wouldn't occur without 1. data rate limits on the receiving side, and simultaneously 2. limited buffer size.

1. USB hardware as well as related OS subsystems on modern computers (5000 or at least 480 Mbit/s) are much faster than max data rate coming from pico (12 Mbit/s). But some objects in Python are fairly inefficient (while some are fine). Aside of this, a Python process can intermittently stop pulling USB data if any procedure [in any of its sub-threads] is busy, imagine re-plotting a complicated oscilloscope screen. To remedy this, I configured the rp2daq.py module to use a separate process to efficiently receive and enqueue data from USB.

2. On Linux, this works like a charm; if the computer is super busy, it accumulates some megabytes in the inter-process queue, but when it finally gets to pull them from the queue it never loses a single byte. I have little experience with Windows, though, and the queue there may have a bit different implementation or default parameters: Its limited size, or a stricter timeout are possible way how you lose data. Maybe I should try a multiprocessing.Pipe instead of multiprocessing.Queue.

Fixing any of these two problems (speed / queue size) would probably prevent data losses. Of course, the clean way is to make receiving data both fast & rock solid.

One only learns this by real-life testing! So thank you once again for reporting this early.

from rp2daq.

OK thanks.
let me check with CPP and CreateFile() api on windows and see if it works without packet loss. I'll post result here...

from rp2daq.

after some optimization, and separating reading and processing thread, the C# code finally worked. on my PC there is no need for lower level APIs like win32 CreateFile(), with the SerialPort itself it could solve problem.
Thanks

from rp2daq.

Thanks for noting. The task ahead of me is making it 500 ksps sampling reliably work with Python also on Windows, anyway. So it could be useful if you point out some tricks that you found effective.

from rp2daq.

Thanks for noting. The task ahead of me is making it 500 ksps sampling reliably work with Python also on Windows, anyway. So it could be useful if you point out some tricks that you found effective.

Sure, still working on it. here is the code that creates above result so far:

static void TestDirect()
      {
          var sport = new SerialPort("COM6", 268435456);

          {//https://stackoverflow.com/a/73668856
              sport.Handshake = Handshake.None;
              sport.DtrEnable = true;
              sport.RtsEnable = true;
              sport.StopBits = StopBits.One;
              sport.DataBits = 8;
              sport.Parity = Parity.None;
              sport.ReadBufferSize = 1024 * 10000;
          }

          sport.Open();
          
          string ver;

          //read device identifier
          {
              var dt = new byte[] { 1, 0, 1 };

              sport.Write(dt, 0, dt.Length);
              
              Thread.Sleep(100);

              var l = 34;

              if (sport.BytesToRead != l)
                  throw new Exception("Unexpected Resonse Length");

              dt = new byte[34];

              sport.Read(dt, 0, dt.Length);
          }

          {
              var dt = new byte[] { 0x0A, 0x04, 0x10, 0xe8, 0x03, 0x01, 0x01, 0x00, 0x60, 0x00 };

              sport.Write(dt, 0, dt.Length);
          }

          var str = sport.BaseStream;

          var header = new byte[24];
          var samples = new byte[1500];

          while (true)
          {
              //ReadExactAmount(sport, header.Length, header);
              //ReadExactAmount(sport, samples.Length, samples);
              ReadArray(sport.BaseStream, header);
              ReadArray(sport.BaseStream, samples);


              Console.WriteLine("header: " + BitConverter.ToString(header));
          }

          
      }

public static void ReadArray(this Stream stream, byte[] array)
      {
          var buf = array;

          var counter = 0;

          var l = array.Length;

          while (counter < l)
          {
              var remain = l - counter;

              var rdr = stream.Read(buf, counter, remain);
              counter += rdr;
          }
      }

from rp2daq.

Still am not able to have it in multy thread mode.
Do you see any mis configuration in this? for example DTR and RTS are ok for connecting to PICO? or stop bit etc.

{//https://stackoverflow.com/a/73668856
      sport.Handshake = Handshake.None;
      sport.DtrEnable = true;
      sport.RtsEnable = true;
      sport.StopBits = StopBits.One;
      sport.DataBits = 8;
      sport.Parity = Parity.None;
      sport.ReadBufferSize = 1024 * 10000;
}

from rp2daq.

Your solution indeed does not use multithreading nor multiprocessing. More complicated, real-world programs may need to separate data reception process from other tasks to avoid GUI lags and possible data missing. This depends on what you wish your application to accomplish - if it is super simple, you can be fine with this short code.

Please note that while I appreciate your effort, I have no capacity for developing any C# interface/programs right now. When I make a demo oscilloscope app in Python, I will notice you. Apparently a lot of concepts can be directly transferred from Python to C# code.

Regarding your latter post: USB is commonly treated as a "serial port", but probably none of these serial port parameters are relevant for it. You can change any of them to see they have totally no effect whatsoever. (Perhaps excepting the ReadBufferSize, which I don't know: you would have to set it to e.g. 1 byte to see if it somehow limits data rate.)

from rp2daq.

Finally i've made it working. The only problem i still facing, is the detecting of frequency with FFT. I've done some dummy algorithm to find frequency. the frequency is found without problem, but the phase keeps changing :) have look at the image which is a 25% pwm signal made by arduino with 490Hz frequency.

also code is updated in the repository so you can see how multi threading is working.
I'm not good at FFT and math stuff at all.
thanks anyways, i'll close the issue for now

from rp2daq.

Also here is the working code so far which is used in above gif image...
https://github.com/epsi1on/SimpleOscilloscope/blob/main/src/POC/SimpleOsciloscope.UI/HardwareInterface/RpiPicoDaqInterface.cs

from rp2daq.

Very nice.

I can implement edge trigger for the ADC acquisition like common oscilloscopes have. Then you will have a nice waveform that would not jump across your screen. OK?

from rp2daq.

Very nice.

I can implement edge trigger for the ADC acquisition like common oscilloscopes have. Then you will have a nice waveform that would not jump across your screen. OK?

i did the SimpleOsciloscope project as a POC (Proof of Concept). It meant to have limited features. like only 1 channel data, simple yet ugly display (as you can see in the image) and having a buggy frequency detection algorithm.
for the edge detection, if you can make the python or C version, then i probably am able to translate it into C#.
Here is a raw ADC dump of 500ksps sample for a 25% PWM signal you can simply parse with python. each line a short integer value of sample.

samples.csv

If you are ok, then we could continue this discussion in other repository (only for future reference)? I made an issue there:

epsi1on/RpiPicoOsciloscope#2

from rp2daq.