Custom firmware versionning and releases about plutoscripts HOT 8 OPEN

An option other than having buildroot do it is to cross-compile or compile on a pi3 and place the output files into a folder tree to mimic the sysroot of the firmware and specify this in buildroot as an overlay path. This is how I've always done it for items that are not natively available to be built in buildroot. Inside the buildroot dir, make xconfig, CTRL+F and search overlay to find the option easy. You can see an example overlay structure here: https://github.com/unixpunk/PlutoWeb/tree/master/plutoweb/overlay

from plutoscripts.

Hi Romain,

thank you for your work with the Pluto SDR. It's a great piece of hardware and you make it even more interesting. Sorry for creating a github issue to actually ask questions, I hope you're fine with it...

Thank you for your feedback !

After some reading, I realized I have a rev.B with 32MB flash only, so I will not be able to use your firmware (or at least the full version).
Question 1 : Is there an up to date "light" version that I can use, and did you list differences with full version somewhere (sorry I could not find this infomation).

Things were not very clear next days after discovering this issue.
Having a 32MB (which is - after all - the norm according to data specs) version implies you have a pre-requisite task to perform once (unlock flash) before reflashing. Once unlocked, you can flash up to 32MB size.

To make short, read carefully this thread : https://ez.analog.com/university-program/f/q-a/105941/adalm-pluto---firmware-dfu-flashing-fails-at-22-23mb/313815
Just remember the big "brick risk" is flashing file size over 32MB but this should never occur !
I'm using the "dfu-ram" trick described by Michael. It worked every time.
It seems that copying pluto.frm file on the USB drive (then eject) also works, but often failed for me.

In the same taste, would it be possible to share your work on fimware development using github ?
That would be much easier for finding documentation, managing versions/releases, and also allow collaborative work with 'pull requests' or 'forks'.

I try to update the plutoscripts repository with useful scripts I'm using on the pluto, mainly from /root directory, but not the whole filesystem.
Firmware image is not pushed to github, just because cloning may be a real pain if your repository contains many updates of big (firmware) files.
I'm also considering rewriting the too long doc as wiki and removing old versions.

Question 2 : I am not very good with linux, but my "wish" would be to develop a fairly light firmware mostly focused on dependencies (python, soapy, libiio, gnuplot + libpng, busybox, boost?), and put all cross-compiled binaries (and bootup script) on a USB stick.
In your opinion, is this something doable and valuable ?

Yes that's the best option.
Since flash size is limited, we have to deal with free space and make choices. Using external filesystem (network, or USB drive) provides a great comfort to work on pluto.
I'm running NFS filesystem since few months to store non-volatile files and extend pluto's filesystem.
Some files are copied at boot from NFS share (myscript.sh) to pluto allowing to customize my device. Other files (like rtl_433/dump1090 log files) are copied at regular interval to the NFS share, providing stats over a long time range even after a reboot.

Could you help to complete a list of dependencies that might be useful to integrate in the firmware (SDR focused and general utilities ) ?

Depends of your usage, if you are hamradio or not.
Let's say a pluto with network support (wifi and NFS), rx_tools, gnuplot and CSDR and luaradio is a very good basis.
Apart of this installing few linux utilities helps a lot : sox, socat, netcat, jq, ntp and others, timeout, crond...
Soapy is more or less mandatory ...
OpenwebRX as native on is very good for mobile use. You can use your pluto as SDr receiver on a computer without any driver installed.
Python + numpy can fit on the flash, very good to learn SDR basis, however it will take 60% of total space.
GNUradio is working very well when installed on NFS share, but to big to fit in flash (except by removing essentials modules like WiFi)
Im my configuration, python+numpy is running on the flash. GNUradio on NFS drive.

Question 3 : Would it be preferable to actually put the full filesystem on USB stick (and get a 'super super light' fw in flash) ?

Not really. You have to focus first on a pluto running as standalone. Add your applications, the apps you wan't to use every day.
Use the remote filesystem (network or USB) to add more features or extension to your pluto.
For example it's a non-sense to install openwebrx on the flash, and python on remote filesystem. Just because openwebrx needs python...

Last question(s):
I found this document: https://wiki.analog.com/university/tools/pluto/building_the_image

Is this what you started from ?

Yes, this doc is enough ! Big download first. Have to say I had skills on compiling firmware before starting on pluto.
If you plan to compile your own firmware, expect long hours, lot of tries but very good moments.

Please have also look to OSCIMP team work, it's really well documented and you will learn a lot.
If not tested yet, you can also try Plutoweb firmware from @unixpunk: it's a well-designed firmware, easy to use and update.

Did you maybe share instructions or how-to somewhere ?
Not really. To add applications, you can compile directly from buildroot, or sometimes have to tweak config files, or cross-compiling before adding files to the filesystem. Can be very tricky sometimes !
There is no universal recipe unfortunately, and it's not on my plan to provide support on pluto compilation. However I'm always happy to help anyone requesting help.
I'm just planning to share some customized packages config later on the summer.

FYI: I worked on a 0.30 version, unfinished, which was never released (but can provide a link as private)
Compiled yesterday version 0.31 for the first time. Will focus on GNUradio, SDRangel, scripts cleaning and interface to make use easier. Do not expect a release before one month !

Keeping this issue as open if you have more questions.
Regards,

from plutoscripts.

Wow, thank you very much for the comprehensive answer !

To make short, read carefully this thread : https://ez.analog.com/university-program/f/q-a/105941/adalm-pluto---firmware-dfu-flashing-fails-at-22-23mb/313815
Just remember the big "brick risk" is flashing file size over 32MB but this should never occur !
I'm using the "dfu-ram" trick described by Michael. It worked every time.

Yes I actually read that before posting yesterday.
However, after a long wait before buying my Pluto, I barely started using it so I'm probably not ready for the "brick risk" yet :)
I'll try to stick with the 22MB limit for now... Do you still have a light version somewhere that I could try ?

Firmware image is not pushed to github, just because cloning may be a real pain if your repository contains many updates of big (firmware) files. I'm also considering rewriting the too long doc as wiki and removing old versions.

That makes sense, I did not anticipate the cloning problem. The wiki doc is a great idea.
Thank you for the scripts BTW, that's a great piece of work.

I'm running NFS filesystem since few months to store non-volatile files and extend pluto's filesystem.
Some files are copied at boot from NFS share (myscript.sh) to pluto allowing to customize my device. Other files (like rtl_433/dump1090 log files) are copied at regular interval to the NFS share, providing stats over a long time range even after a reboot.

I like that, NFS is definitely in the list of things to include.

Depends of your usage

Correct. For a first try, I think about a very minimal firmware including essential dependencies and drivers.
Basically the stock firmware which allows using pluto connected to a computer, with few additions:

• some linux tools (as you suggested) for "everyday comfort"
• a list of essential libs/deps so we can use a large panel of tools (stored elsewhere)
• a boot script that searches what to download/do from a NFS share or usb stick (also containing the actual cross-compiled "standalone" tools)

If you plan to compile your own firmware, expect long hours, lot of tries but very good moments.

Yes, I wish I have enough time for that...
Before dreaming on the actual list of tools/dependencies, I'll first try to see if I can manage to build the stock firmware ;)

To add applications, you can compile directly from buildroot (...)

I guess there are some complex things involved here... First thing I'll try is to setup a compilation environment.
I was thinking about a virtual envirnoment that could be shared, did you maybe hear about a docker image with this environment setup ?
If not, should I start working on that ?

Thank you again, great to talk with you !

from plutoscripts.

Hi again,

However, after a long wait before buying my Pluto, I barely started using it so I'm probably not ready for the "brick risk" yet :)
I'll try to stick with the 22MB limit for now... Do you still have a light version somewhere that I could try ?

You have virtually no risk unless you flash later a firmware file size over 32MB (32 I repeat). This should never occur because you are a prudent guy, and you know 32MB is the maximum flash size :)

For a light version, follow this link, then go to "OLD" folder : https://mega.nz/#F!C8hgFASK!C9DfCjU7ou46mG-LNWQTrQ
However it's a little bit outdated (01/2019)

I like that, NFS is definitely in the list of things to include.

Oh yes, NFS is working well with a RPi on remote side! Unfortunately NFS support is not included on the light firmware version.

Before dreaming on the actual list of tools/dependencies, I'll first try to see if I can manage to build the stock firmware ;)

The first thing to do ! First default compilation is the longest, depending of your computer.
Once done simply try to add linux packages (gnuplot ...), and compile again.

I was thinking about a virtual envirnoment that could be shared, did you maybe hear about a docker image with this environment setup ?
If not, should I start working on that ?

Running on one of my computer, just on a separate directory, I don't use any VM or docker for this kind of task.
Expect 15 to 25 GB data size , you need 4GB RAM: I think using a VM or docker is not the best option.
You just have to go to plutosdr-fw/buildroot, then "source" the specific environment variable (as described in the doc) in your terminal window.
--> Remember to "source" again you CC variables if you open a new terminal window.

Keep us informed ! Regards.

from plutoscripts.

Awesome, thank you for the piece of advice.

I work on very diverse environments so Docker would be a great option regarding my "mobility".
I often dealt with multi-GB images so I'll give it a quick try. If it becomes hard to manage I'll just use the Dockerfile as recipe (or notes).

The first thing to do ! First default compilation is the longest, depending of your computer.
Once done simply try to add linux packages (gnuplot ...), and compile again.

Ok, seems to be a good strategy.

--> Remember to "source" again you CC variables if you open a new terminal window.

Sounds like you just saved me from future headhaches ;) I'll try to remember about that thank you.

I'll probably come back with more questions anyway.
Will keep you posted, thank you again for our support !

from plutoscripts.

Hello again, I had some time to play with docker and got my first firmware compilation done yesterday (stock firmware 0.31). Dockerfile and instructions here.

Now I'd like to customize it but I wonder about the strategy you typically use, so I come back with some questions again (sorry).

Question 1:
If I understand correctly, a lot of packages can be added using the buildroot menuconfig before compiling the firmware. I managed to add some tools this way (python, cifs, etc...).
Is it something that you use as first choice when the buildroot package is available ?

Question 2:
When comes the time to add stuff not available in native buildroot packages, what is you preferred strategy ?
From what I read, an interesting option is to put new programs as packages outside of the buildroot tree (doc here), so it gets compiled as a package with buildroot.
Have you tried that ?

Question 3:
Another track for further customization is to cross-compile manually (using Vivado SDK I guess). Do you have any recommendation or specific procedure for that ?

Thank you !

from plutoscripts.

Hi Romain,

Sorry for the delay, summer migration and poor connection during at least one week.

Hello again, I had some time to play with docker and got my first firmware compilation done yesterday (stock firmware 0.31). Dockerfile and instructions here.

Thank you for your contribution !!
I just had a quick look. Once back at home I will try your docker procedure.

Question 1:
If I understand correctly, a lot of packages can be added using the buildroot menuconfig before compiling the firmware. I managed to add some tools this way (python, cifs, etc...).
Is it something that you use as first choice when the buildroot package is available ?

As first choice ... hmmm seems you did it according your doc.
Regarding sox, you have to compile additional libs. (libssp and another one)
At this moment python3 is not the better choice if you plan to play with GNUradio.
I installed python2.7+numpy, that's enough for SDR processing.

Question 2:
When comes the time to add stuff not available in native buildroot packages, what is you preferred strategy ?
From what I read, an interesting option is to put new programs as packages outside of the buildroot tree (doc here), so it gets compiled as a package with buildroot.
Have you tried that ?

Yes !
Have a look here for package examples related to pluto/SDR :
https://github.com/seanstone/adrv-fw/tree/master/package

I make my own packages when needed, starting from sources, sometimes it works (depends of cmake or Makefile structure, can't predict), sometimes not.
You can then use BR2_EXTERNAL method to define your own-packages path (thus allowing to keep them on a separate location)
OSCIMP github : https://github.com/oscimp/PlutoSDR

Question 3:
Another track for further customization is to cross-compile manually (using Vivado SDK I guess). Do you have any recommendation or specific procedure for that ?

No, just RTFM, it works well.
https://wiki.analog.com/university/tools/pluto/devs/embedded_code
There is also a very good example on the FOSDEM2018 PDF from AnalogDevices (how to compile dump1090)
Sometimes when compiling through buildroot is generating errors, I compile manually.

Hope this can help. Sorry don't have my backups here !
Regards.

from plutoscripts.

Sorry for the delay, summer migration and poor connection during at least one week.

Don't you worry, I'm lucky to have your help already ! Thank you for the great piece of advice.

Have a look here for package examples related to pluto/SDR :
https://github.com/seanstone/adrv-fw/tree/master/package
OSCIMP github : https://github.com/oscimp/PlutoSDR

Wow ok, that's great ! I went through these repositories when looking for alternative firmwares but I did not realize these guys defined so many buildroot packages. Awesome !

You can then use BR2_EXTERNAL method to define your own-packages path (thus allowing to keep them on a separate location)

Yes that's what I'd like to do, still trying to figure out the best way to do it with git-submodules and/or git-subtree.

https://wiki.analog.com/university/tools/pluto/devs/embedded_code
There is also a very good example on the FOSDEM2018 PDF from AnalogDevices (how to compile dump1090)
Sometimes when compiling through buildroot is generating errors, I compile manually.

Ok I'll need some time for trying that too...

Thank you again, will keep you posted with progress.

from plutoscripts.