Giter Club home page Giter Club logo

rp2daq's Introduction

RP2DAQ - Raspberry Pi Pico for Data Acquisition (and much more)

Raspberry Pi Pico is a small, but quite powerful microcontroller board. When connected to a computer over USB, it can serve as an interface to hardware - which may be as simple as a digital thermometer, or as complicated as scientific experiments tend to be.

rp2daq overview

This project presents both precompiled firmware and a user-friendly Python module to control it. The firmware takes care of all technicalities at the microcontroller side including parallel task handling and reliable communication, and is optimized to harness Raspberry Pi's maximum performance. All actions of RP2DAQ are triggered by the Python script in the computer. This saves the user from programming in C and from error-prone hardware debugging. Even without any programming, one can try out few supplied Example programs.

If needed, entirely new capabilities can be added into the open source firmware. More is covered in the developer documentation for the C firmware. Contributing new code back is welcome.

Development status: basic features implemented, real-world testing underway

No programming: first steps

Getting things ready

  1. You will need a Raspberry Pi Pico (RP2) with a USB cable. And a computer.
  2. Make sure your computer has Python (3.6+)
    • On Windows, use a recent Python installer, and then issue pip install pyserial in Windows command line.
    • On Linux, Python3 should already be there, and pyserial can be installed through your package manager or with pip3
    • On Mac, follow a similar approach; version update may be needed
  3. Download the binary firmware from the latest release. No compilation is necessary.
  4. Holding the white "BOOTSEL" button on your RP2, connect it to your computer with the USB cable. Release the "BOOTSEL" button.
    • In few seconds the RP2 should appear as a fake flash drive, containing INDEX.HTM and INFO_UF2.TXT.
  5. Copy the rp2daq.uf2 firmware file directly to RP2.
    • The flashdrive should disconnect in a second.
    • The green diode on RP2 then flashes twice, indicating the firmware is running and awaiting commands.
  6. Download and unpack the stable Python+C source from the latest release.

If you have problems, please describe them in a new issue.

If don't fear having problems, you can get fresh code from this repository and compile the firmware according to Developer notes.

Run hello_world.py

Launch the hello_world.py script in the main project folder.

Two possible outcomes of the script

  • If a window like the one depicted left appears, rp2daq device is ready to be used! You can interactively control the onboard LED with the buttons.
  • If an error message appears (like depicted right) the device does not respond correctly. Check it your RP2 blinks twice when USB is re-connected, and make sure you uploaded fresh firmware. On Linux, Python may need to adjust permissions to access the USB port.
  • If no window appears, there is some deeper error with your Python installation.

Python programming: basic concepts

Controlling the LED with 3 lines of Python code

To check everything is ready, navigate to the unpacked project directory and launch the Python command console. Ipython3 is shown here, but spyder, idle or bare python3 console will work too.

import rp2daq          # import the module (must be available in your PYTHONPATH)
rp = rp2daq.Rp2daq()   # connect to the Pi Pico
rp.gpio_out(25, 1)     # sets GPIO no. 25 to logical 1

The GPIO (general-purpose input-output) 25 is connected to the green onboard LED on Raspberry Pi Pico - it should turn on when you paste these three lines. Turning the LED off is a trivial exercise for the reader.

Receiving analog data

Similarly, you can get a readout from the built-in analog/digital converter (ADC). With default configuration, it will measure 1000 voltage values on the GPIO 26:

import rp2daq
rp = rp2daq.Rp2daq()
print( rp.adc() )

The adc() command returns a standard pythonic dictionary, with several (more or less useful) key:value pairs. Among these, the ADC readouts are simply named data; value 0 corresponds to cca 0 V, and 4095 to cca 3.2 V.

Most commands take several named parameters which change their default behaviour; e.g. calling rp.adc(channel_mask=16) will switch the ADC to get raw readouts from the built-in thermometer. If the parameters are omitted, some reasonable default values are used.

Tip: Use TAB completion

The ipython3 interface has numerous user-friendly features. For instance, a list of commands is suggested by ipython when one hits TAB after writing rp.:

ipython console printout for rp.adc?

The docstring for any command is printed out when one adds ? and hits enter:

ipython console printout for rp.adc?

Alternately, you can find the same information extracted in the Python API reference. In contrast, none of the commands can be found in the python code, as they are generated dynamically by parsing the C code on startup. This eliminates redundancy between the C firmware and the Python module, and always guarantees their perfect binary compatibility.

Asynchronous commands

Consider the following ADC readout code, which looks different, but does almost the same as the previous example:

import rp2daq
rp = rp2daq.Rp2daq()

def my_callback(**kwargs):
	print(kwargs)

rp.adc(_callback=my_callback)     # non-blocking!

print("code does not wait for ADC data here")
import time
time.sleep(.5) # required for noninteractive script, to not terminate before data arrive

The important difference is that here, the rp.adc is provided with a callback function. This makes it asynchronous: it does no more block further program flow, no matter how long it takes to sample 1000 points. Only after the report is received from the device, your _callback function is called (in a separate thread) to process it.

Calling commands asynchronously allows one to simultaneously orchestrate multiple rp2daq commands. It is particularly useful for commands taking long time to finish, like extensive ADC acquisition or stepping motor movement.

Caveats of advanced asynchronous commands use

You can call synchronous rp2daq commands of one type and asynchronous commands of another type without problems.

It is not advisable, however, to issue two asynchronous commands of the same type with two different callback functions, if there is a risk of their temporal overlap. At most one callback function is assigned to one command type, not to each unique command you issued.

As a result, if you launch two long-duration commands of the same type in close succession (e.g. stepping motor movements), first one with _callback=A, second one with _callback=B, each motor reporting its move being finished will eventually result in calling the B function as their callback.

The clean and recommended approach is therefore to define one callback function for a given command type. It can easily tell apart the reports from multiple stepper motors, using information it receives as keyword arguments.

Likewise, an asynchronous command should never be followed by a synchronous one of the same type, as the latter erases the callback. In such a case, the first command would erroneously interrupt waiting for the second one.

Both synchronous and asynchronous commands can be issued even from within some command's callback. This allows for command chaining in an efficient event-driven loop.

Asynchronous command with multiple reports

Maybe the greatest strength of the asynchronous commands lies in their ability to transmit unlimited amount of data through subsequent reports.

The following example measures one million ADC samples; these would not fit into Pico's 264kB RAM, let alone into single report message (limited by 8k buffer). This is necessary to monitor slow processes, e.g., temperature changes or battery discharge.

If high temporal resolution is not necessary, each data packet can be averaged into a single number by not storing kwargs['data'], but [sum(kwargs["data"])/1000]. Note that averaging 1000 numbers improves signal to noise ratio sqrt(1000) ~ 31 times.

import rp2daq
rp = rp2daq.Rp2daq()

all_data = []

def my_callback(**kwargs):
    all_data.extend([sum(kwargs["data"])/1000])
    print(f"{len(all_data)} ADC samples received so far")
print(all_data)

rp.adc(
	blocks_to_send=1000, 
	_callback=my_callback)

print("code does not wait for ADC data here")
import time
time.sleep(.5)
rp.adc(blocks_to_send=0)

An alternative to blocks_to_send=1000 is setting infinite=1. The the ADC reports will keep coming, until they are stopped by another command rp.adc(blocks_to_send=0).

More elaborate uses of ADC, as well as other features, can be found in the example_ADC_async.py and other example scripts.

PAQ: Presumably Asked Questions

Q: How does Rp2daq differ from writing MicroPython scripts directly on RP2?

A: They are two fundamentally different paths that may lead to similar results. MicroPythonGitHub stars (and CircuitPython) interpret Python code directly on a microcontroller (including RP2), so they are are good choice for a stand-alone device (if speed of code execution is not critical, which may be better addressed by custom C firmware). There are many libraries that facilitate development in MicroPython.

In contrast, rp2daq assumes the microcontroller is constantly connected to computer via USB; then the precompiled firmware efficiently handles all actions and communication, so that you only need to write one Python script for your computer.

Q: Can I use Rp2daq with other boards than Raspberry Pi Pico?

A: Very likely it can be directly uploaded to all boards featuring the RP2040 microcontroller. Obviously the available GPIO number, as well as their assignment, may differ. For instance the colourful LED on the RP2040-zero is in fact a WS2812B chip, and its data bus is connected to GPIO 16.

However, the RP2040-zero randomly failed to connect over USB, as reported elsewhere (1) (2) ; we cannot consider it fully supported.

The Arduino family of boards is not supported. Neither the ESP/Espressif boards are. (Development of this project was started on the ESP32-WROOM module, but it suffered from its randomly failing (and consistently slow) USB communication, as well as somewhat lacking documentation.)

Q: Can a Rp2daq device be controlled from other language than Python 3.6+?

A: Perhaps, but it would be rather hard. The firmware and computer communicate over a binary interface that would have to be ported to this language. One of the advantages of RP2DAQ is that the interface on the computer side is autogenerated; the corresponding C-code parser would have to be rewritten. Hard-coding the messages in another language would be a quicker option, but it would be bound to a single firmware version.

Python is fine.

Q: Are there projects with similar scope?

A: Telemetrix also uses Raspberry Pi Pico as a device controlled from Python script in computer. Rp2daq started independently, with focus on higher performance and broader range of capabilities. However, the report handling subsystem in Rp2daq was inspired by Telemetrix.

PyFirmata does a similar job, but has not received an update for a while.

Belay

Digital I/O can similarly be performed with PyFtdi.

Q: Can Rp2daq communicate with scientific instruments, e.g. connected over GPIB/VISA?

A: No, although basic support e.g. for less common interfaces GPIB could be added in the future.

Digital interfacing with lab instrumentation is outside of Rp2daq's scope, but over 40 other projects provide Python interfaces for instrumentation and they can be imported into your scripts independently. While Rp2daq does not aim to provide such interfaces, capabilities of RP2 could substitute some commercial instruments in less demanding use cases.

Q: Why are no displays or user interaction devices supported?

A: The Python script has a much better display and user interaction interface - that is, your computer. Rp2daq only takes care for the hardware interaction that computer cannot do.

Q: Can Rp2daq control unipolar stepper motors using ULN2003?

A: No. Both bipolar and unipolar steppers seem to be supported by stepstick/A4988 modules, with better accuracy and efficiency than provided by ULN2003.

Legal

The firmware and software are released under the MIT license.

They are free as speech after drinking five beers, that is, with no warranty of usefulness or reliability. Rp2daq cannot be recommended for industrial process control.

rp2daq's People

Contributors

epsi1on avatar filipdominec avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

rp2daq's Issues

STM32

Hello there,
Do you know any firmware that utilities use stm32's ADC for using as sampler for osciloscope?
Not sure but i think stm32f4 can handle 3 channels of ADC, up to 2 Msps for each channel.

page 105 of datasheet
image

Thanks

Edge Detection

Hi,
How does the Edge Detection works? is it a trigger which tells the ADC to when take samples?
I would be thankful if you give me some info about it's logic.
Thank you

Interfacing With C#.NET code

Hello there...
Thanks for the great library.
I am trying to interface your firmware with my C# code. In C# I need to read the ADC values of RPI pico.
I was wondering if you please can give me some direction on:

  • How to find the device
  • How to start ADC reader on device
  • How to Read and interpret data

It is more about the communication protocol.
I'm trying to make a super simple oscilloscope with WPF and C#.

Thanks again.

Multiprocessing error on Windows 11: ctypes objects containing pointers cannot be pickled

The commit 4ee6530 (from Jun 9 2023) introduced multiprocessing; it works flawlessly on Linux, but on Windows 11 this throwed

C:\Users\ts\rp2daq-main> python .\hello_world.py
2023-08-29 13:48:14,459 (MainThread) connected to rp2daq device with manufacturer ID = E66118604B826B2A
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.1520.0_x64__qbz5n2kfra8p0\Lib\multiprocessing\spawn.py", line 113, in spawn_main
	new_handle = reduction.duplicate(pipe_handle,
				 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.1520.0_x64__qbz5n2kfra8p0\Lib\multiprocessing\reduction.py", line 79, in duplicate
	return _winapi.DuplicateHandle(
		   ^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [WinError 50] The request is not supported

This is critical, as it disables rp2daq use from Windows.

Most examples currently don't work, due to multiprocessing

TR;DR When performance optimisation made me to introduce multiprocessing in the rp2daq module, it has broken all example scripts here. Instead of expanding the code of the examples, I demonstrate how to patch the multiprocessing.Process class so that no changes in user code is required.

Rationale The multiprocessing module turns out necessary for fast & reliable USB communication, and its compatibility with both Windows & Linux stipulates the use of the "spawn" approach instead of "fork". This in turn allegedly requires that a part of the executed script code, excepting imports and some trivial definitions, is fully enclosed in a "guard":

if __name__ == "__main__": ...

.... clause, as discussed in multiprocessing module's Programming Guidelines. Otherwise spawning the secondary process would result in an infinite chain of further processes - a fork bomb, or rather a spawn bomb here.

This has been puzzling people for over 10 years without anybody actually bringing another solution: https://stackoverflow.com/questions/22595639/best-practices-for-using-multiprocessing-package-in-python?rq=4
https://stackoverflow.com/questions/48306228/how-to-stop-multiprocessing-in-python-running-for-the-full-script
https://stackoverflow.com/questions/48680134/how-to-avoid-double-imports-with-the-python-multiprocessing-module
https://stackoverflow.com/questions/50781216/in-python-multiprocessing-process-do-we-have-to-use-name-main
https://stackoverflow.com/questions/55057957/an-attempt-has-been-made-to-start-a-new-process-before-the-current-process-has-f
https://stackoverflow.com/questions/57191393/python-multiprocessing-with-start-method-spawn-doesnt-work
https://stackoverflow.com/questions/70263918/how-to-prevent-multiprocessing-from-inheriting-imports-and-globals
https://stackoverflow.com/questions/77220442/multiprocessing-pool-in-a-python-class-without-name-main-guard

With all due respect to the python's core module developers, I consider this necessity for the "guard" a very bad UX design decision, because it cannot be hidden in the module - instead it is propagated to all scripts that import it. It apparently breaks the principle of least astonishment, leads to repeating code, also it is ugly, complicated, and fairly hard to explain.

One of core principles of rp2daq is to resolve all necessary complexity inside its code, thus enabling the user to write very short, easy-to-read scripts as means for routine instrumentation control. Therefore, the "guard" clause is utterly unacceptable for this project.

Fortunately, a multiplatform, and arguably quite elegant solution exists.

Suggestion

First of all thanks and congrats for the innovative project.
Currently the compiled firmware (I mean /build/rp2daq.uf2 file) is placed inside the project files. I would like to suggest you to publish the compiled binary file as a release file. I think this is how usually the binary files are published in github.
also the version control would be easier and files can be visible from release section:
https://github.com/FilipDominec/rp2daq/releases

Different response from PICO in windows and ubuntu

Hi,
I was trying to diagnose the data problem i've got in the windows. so did try to get infinite blocks by ADC.
I simply send a command to PICO, which is :

0x0A,0x04,0x10,0xe8,0x03,0x01,0x01,0x00,0x60,0x00

equivalent to this I think

command

I assume after sending this code to PICO, i'll get a 24 byte of block info. The bitwidth is [3]th byte of header followed by 1500 bytes of sample data. i constantly read 24 byte then 1500 byte over and over. on ubuntu my bitwidth is always 12 but in windows, after block 10 or 11 it will start do get different bitwidth for each block. Note that i execute same code on windows and ubuntu, only need to comment out portname line.
I think is because of some transfer problem maybe.
do you have any idea what is the problem?

here is the code:

import time
import serial
import struct

## User options
#width, height = 1000, 1000
#channels = [0, 1, 2,] #,  3, 4]     # 0,1,2 are GPIO 26-28;  3 is V_ref and 4 is internal thermometer
#kSPS_per_ch = 100 * len(channels)  # one-second run

import rp2daq
import sys
import tkinter

#rp = rp2daq.Rp2daq()


#port = "/dev/ttyACM0"
port = "COM6"
ser = serial.Serial(port=port, timeout=1)

ser.write(struct.pack(r'<BBB', 1, 0, 1)) 
time.sleep(.15) # 50ms round-trip time is enough

try_port = ser
   
id_data = ser.read(try_port.in_waiting)[4:] 

print ("connected to " + str(id_data))

dt = bytes([0x0A,0x04,0x10,0xe8,0x03,0x01,0x01,0x00,0x60,0x00])

print(dt)

ser.write(dt) 

time.sleep(.15) 

sizeToRead = 1500

cnt =0;

while True:
    header = ser.read(24)
    
    print(str(cnt)+ " th block, bitwidth: " + str(header[3]))
    ser.read(sizeToRead)
    cnt = cnt+1

2023-09-01 05_55_40-Windows PowerShell
Screenshot from 2023-09-01 09-22-29

Digital Input

Hi,
I was trying to add a x10 attenuation button to the oscilloscope. using a 2-contact switch which one of those is unused.

image

Screenshot from 2024-04-15 08-25-09

image

switch have two contacts which one of them is free (as image). how is good for me to know that whether this switch is pressed or not (want to detect key press automatically on PC)?

Thanks

RPI Pico Carrier Board

Hi,
Thanks for great project. could you suggest a simple carrier board for RPI pico to turn it into oscilloscope?
A very simple one like THIS (with 2 resistor, 3 diode and 1 capacitor) could work with PICO version?
I am also wrapping up to make next version of C# oscilloscope.

Rewrite port handler to support software-controlled disconnect & reconnect

Double initialization of one device must be blocked.

The 2nd call somehow manages to connect to already connected device, but the identify callback
gets redirected to the first Rp2daq object:

In [2]: rp2daq.Rp2daq()                                                                                                                                       
2024-04-06 11:54:19,091 (MainThread) connected to rp2daq device with manufacturer ID = E66138935F6F8E28
Out[2]: <rp2daq.Rp2daq at 0x7f8dfee52c70>

In [3]: rp2daq.Rp2daq()                                                                                                                                       
2024-04-06 11:54:23,229 (Thread-273) Warning: Got callback that was not asked for
		Debug info: {'report_code': 0, '_data_count': 30, '_data_bitwidth': 8, 'data': b'rp2daq_231005_E66138935F6F8E28'}
2024-04-06 11:54:23,373 (MainThread)    port open, but device does not identify itself as rp2daq: b''
2024-04-06 11:54:23,374 (MainThread) Error: could not find any matching rp2daq device

So on line "[3]" we should have detected the device was not available, and reported an error instead.

Related to this: the port handler process must stop once we have sw-disconnect, then don't report hw disconnect and allow for clear reconnect.

In [4]: rp.quit()                                                                                                                                             
 In [5]: 2024-04-05 11:16:13,790 (MainThread) Device disconnected, check your cabling or possible short-circuits  

Also related: rp.quit() should release COM port on Win10, so that one can run it again.

And if there is a long command (like synchronous stepper_move) when the device gets physically disconnected, it hangs forever - because it ignores the disconnect event, and waits for the report over USB which never comes.

And when a device is still actively reporting some data (infinite ADC, gpio edge events etc.) and a script is re-started, it randomly won't initialize, because initialize immediately expects a specific short report. Maybe the device should be somehow reserved to stop any other reports (?), then port flushed, then called the init.

Calibrate frequency accurately

It appears that the clock frequency is few % off from nominal 250 MHz. This may be an issue with accurate timing applications! Will have to check this.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.