oresat / oresat-prucam-ar013x Goto Github PK

View Code? Open in Web Editor NEW

3.0 5.0 2.0 2.93 MB

Linux kernel module and PRU firmware for the Star Tracker's AR013x camera.

License: GNU General Public License v3.0

Makefile 2.11% C 84.80% Shell 0.88% Python 4.06% Batchfile 2.19% Assembly 5.97%

camera kernel-module linux am335x

oresat-prucam-ar013x's Introduction

oresat-linux-prucam

Kernel module, device tree overlay, and PRU firmware to interface to camera using the PRUs on the AM335x.

NOTE: This was tested on Debian with kernel 5.10. TI's PRU support libraries are constantly changing, so this likely does not work on older kernels anymore.

What is a PRU?

Programible Real-time Units. It a microcontroller that shares pins and other resource with the core processor allowing for custom interface to other hardware like cameras. See TI PRU-ICSS webpage for more details.

Supported cameras

Build and install prucam

NOTE: The kernel module requires both the device tree overlay and pru firmware binaries to be installed to work, so build and insert it last.

Device tree overlay

Install dependencies: $ sudo apt install device-tree-compiler
Compile dtbo: $ make -C src/device_tree_overlay
Install dtbo: $ sudo make -C src/device_tree_overlay install
Edit /boot/uEnv.txt
- Change the #dtb_overlay=<file8>.dtbo line to dtb_overlay=/lib/firmware/prudev-00A0.dtbo
- Make sure the #enable_uboot_cap_universal= line is commented out.
Reboot system to apply device tree overlay: $ sudo reboot

PRU firmware

Install dependencies: $ sudo apt install ti-pru-cgt-v2.3 ti-pru-software-v6.0
Compile binaries for both PRUs: $ make
Install binaries for both PRUs: $ sudo make install

Kernel module

Install the kernel headers: $ sudo apt-get install linux-headers-`uname -r`
Build kernel module: $ make -C src/kernel_module clean all
Insert kernel module: $ sudo insmod src/kernel_module/prucam.ko
Note: To remove kernel module: $ sudo rmmod prucam

Test prucam

$ cd testing/camera-test-c
Compile: $ make
Capture image:$ sudo ./test_camera
- This will produce capture_001.bmp

Debian package

prucam-dkms is the kernel module that provides the sysfs interfaces to the pru and ar013x camera settings. This package will also install the compiled pru firmware and the prucam dtbo. NOTE when install, it will auto load the module.

To build prucam-dkms:

Install build dependencies: $ sudo apt install --no-install-recommends --no-install-suggestions debhelper fakeroot dkms linux-headers-`uname -r` device-tree-compiler ti-pru-cgt-v2.3 ti-pru-software-v6.0
- The --no-install-* flags are due to dkms installing linux-headers package for the wrong kernel version
Build prucam Debian packages: $ ./makedeb.sh

oresat-prucam-ar013x's People

Contributors

Stargazers

Watchers

Forkers

aakarsh mfkiwl

oresat-prucam-ar013x's Issues

fix gnarly kernel panic when multi processes use prucam

if multiple processes try to use prucam, a gnarly kernel panic occurs that requires restart. Fix it! Since the camera can only be read by one process at a time, we should just disallow multiple processes from opening the char device

pin configs

Either the python library or kernel module needs to set the pins to the correct settings. Nothing is configuring the pins right know outside of deploy.sh in the PRU firmware source directory.

Pin configs not happy in boot logs

I caught this in the boot log. It is happening every time

[    1.017329] pinctrl-single 44e10800.pinmux: pin PIN109 already requested by ocp:cape-universal; cannot claim for ocp:P1_20_pinmux
[    1.029207] pinctrl-single 44e10800.pinmux: pin-109 (ocp:P1_20_pinmux) status -22
[    1.036794] pinctrl-single 44e10800.pinmux: could not request pin 109 (PIN109) from group pinmux_P1_20_default_pin  on device pinctrl-single
[    1.049513] bone-pinmux-helper ocp:P1_20_pinmux: Error applying setting, reverse things back
[    1.060482] pinctrl-single 44e10800.pinmux: pin PIN107 already requested by ocp:cape-universal; cannot claim for ocp:P1_29_pinmux
[    1.072292] pinctrl-single 44e10800.pinmux: pin-107 (ocp:P1_29_pinmux) status -22
[    1.079907] pinctrl-single 44e10800.pinmux: could not request pin 107 (PIN107) from group pinmux_P1_29_default_pin  on device pinctrl-single
[    1.092732] bone-pinmux-helper ocp:P1_29_pinmux: Error applying setting, reverse things back
[    1.103521] pinctrl-single 44e10800.pinmux: pin PIN104 already requested by ocp:cape-universal; cannot claim for ocp:P1_31_pinmux
[    1.115428] pinctrl-single 44e10800.pinmux: pin-104 (ocp:P1_31_pinmux) status -22
[    1.123030] pinctrl-single 44e10800.pinmux: could not request pin 104 (PIN104) from group pinmux_P1_31_default_pin  on device pinctrl-single
[    1.135761] bone-pinmux-helper ocp:P1_31_pinmux: Error applying setting, reverse things back
[    1.148037] pinctrl-single 44e10800.pinmux: pin PIN101 already requested by ocp:cape-universal; cannot claim for ocp:P1_33_pinmux
[    1.159952] pinctrl-single 44e10800.pinmux: pin-101 (ocp:P1_33_pinmux) status -22
[    1.170998] pinctrl-single 44e10800.pinmux: could not request pin 101 (PIN101) from group pinmux_P1_33_default_pin  on device pinctrl-single
[    1.184051] bone-pinmux-helper ocp:P1_33_pinmux: Error applying setting, reverse things back
[    1.199740] pinctrl-single 44e10800.pinmux: pin PIN100 already requested by ocp:cape-universal; cannot claim for ocp:P1_36_pinmux
[    1.212004] pinctrl-single 44e10800.pinmux: pin-100 (ocp:P1_36_pinmux) status -22
[    1.219652] pinctrl-single 44e10800.pinmux: could not request pin 100 (PIN100) from group pinmux_P1_36_default_pin  on device pinctrl-single
[    1.232353] bone-pinmux-helper ocp:P1_36_pinmux: Error applying setting, reverse things back
[    1.251980] pinctrl-single 44e10800.pinmux: pin PIN15 already requested by ocp:cape-universal; cannot claim for ocp:P2_18_pinmux
[    1.263700] pinctrl-single 44e10800.pinmux: pin-15 (ocp:P2_18_pinmux) status -22
[    1.271162] pinctrl-single 44e10800.pinmux: could not request pin 15 (PIN15) from group pinmux_P2_18_default_pin  on device pinctrl-single
[    1.283676] bone-pinmux-helper ocp:P2_18_pinmux: Error applying setting, reverse things back
[    1.295871] pinctrl-single 44e10800.pinmux: pin PIN14 already requested by ocp:cape-universal; cannot claim for ocp:P2_22_pinmux
[    1.307601] pinctrl-single 44e10800.pinmux: pin-14 (ocp:P2_22_pinmux) status -22
[    1.315207] pinctrl-single 44e10800.pinmux: could not request pin 14 (PIN14) from group pinmux_P2_22_default_pin  on device pinctrl-single
[    1.327738] bone-pinmux-helper ocp:P2_22_pinmux: Error applying setting, reverse things back
[    1.343909] pinctrl-single 44e10800.pinmux: pin PIN106 already requested by ocp:cape-universal; cannot claim for ocp:P2_28_pinmux
[    1.355717] pinctrl-single 44e10800.pinmux: pin-106 (ocp:P2_28_pinmux) status -22
[    1.363267] pinctrl-single 44e10800.pinmux: could not request pin 106 (PIN106) from group pinmux_P2_28_default_pin  on device pinctrl-single
[    1.375956] bone-pinmux-helper ocp:P2_28_pinmux: Error applying setting, reverse things back
[    1.389116] pinctrl-single 44e10800.pinmux: pin PIN103 already requested by ocp:cape-universal; cannot claim for ocp:P2_30_pinmux
[    1.400877] pinctrl-single 44e10800.pinmux: pin-103 (ocp:P2_30_pinmux) status -22
[    1.408423] pinctrl-single 44e10800.pinmux: could not request pin 103 (PIN103) from group pinmux_P2_30_default_pin  on device pinctrl-single
[    1.421110] bone-pinmux-helper ocp:P2_30_pinmux: Error applying setting, reverse things back
[    1.433880] pinctrl-single 44e10800.pinmux: pin PIN102 already requested by ocp:cape-universal; cannot claim for ocp:P2_32_pinmux
[    1.445660] pinctrl-single 44e10800.pinmux: pin-102 (ocp:P2_32_pinmux) status -22
[    1.453211] pinctrl-single 44e10800.pinmux: could not request pin 102 (PIN102) from group pinmux_P2_32_default_pin  on device pinctrl-single
[    1.465899] bone-pinmux-helper ocp:P2_32_pinmux: Error applying setting, reverse things back
[    1.478652] pinctrl-single 44e10800.pinmux: pin PIN105 already requested by ocp:cape-universal; cannot claim for ocp:P2_34_pinmux
[    1.490445] pinctrl-single 44e10800.pinmux: pin-105 (ocp:P2_34_pinmux) status -22
[    1.497994] pinctrl-single 44e10800.pinmux: could not request pin 105 (PIN105) from group pinmux_P2_34_default_pin  on device pinctrl-single
[    1.510682] bone-pinmux-helper ocp:P2_34_pinmux: Error applying setting, reverse things back

Add support for AR0134 sensor

we need to add AR0134 support to the driver, along with AR0130. Ideally, the driver would dynamically figure out which sensor it is being used.

Fix for rename of ti-pru-cgt-installer

The TI PRU C/C++ Compiler and Assembly Language Tools support development of applications for TI PRU (Programmable Real-Time Unit) coprocessors.

Library has been renamed from ti-pru-cgt-installer to ti-pru-cgt-v2.3, the control files need to be modified appropriately so that building the package does not fail.

create simple image capture local API server

AR013X autoexposure

Figure out how do auto exposure for the AR013X or get the data for it.

Major PRU firmware Improvements

This is not so much a single issue, as a high level placeholder for multiple individual (and complex) PRU FW related issues.

We should improve the maximum camera clock frequency which the PRUs can read in. This will likely be implemented with a scheme similar to what was done here.
https://theembeddedkitchen.net/beaglelogic-building-a-logic-analyzer-with-the-prus-part-1/449
It would be nice if the PRU FW supported a streaming mode. I think the best way to implement this would be for the kernel driver to allocate a ring buffer and put the PRU in "streaming" mode, and then the PRU constantly fills the ring buffer and the kernel driver can read from it.
The above 2 features should be "modes" in the PRU FW, along with the current mode, and the mode should be configurable from the kernel driver.

..more to come, I am sure

Make eMMC flashable via USB

We should make the eMMC flashable via USB so we don't need to use the SD card

Auto exposure by deault

The kernel module should enable auto exposure by default.

PRU firmware Instructions

Dear Oresat team,

I've tried following the steps as instructed in the README markdown to build and install prucam, namely the steps Device tree overlay, PRU firmware and Kernel module, however I'm having a bit of trouble at the second step.

After installing the dependencies ti-pru-cgt-v2.3 and ti-pru-software-v6.0, I tried compiling and installing through some makefile but I can't seem to find the correct one.
Here's my attempt at cgt-pru:

Do you have any tips on how to solve this?

Warm regards,
João

Fix a couple AE sysfs bugs

Remove auto exposure stats from image

The python lib should remove the auto exposure stats from images and save the stats to a protected variable that is accessible thru a property.

Fix camera i2c register write

looks like I forgot to flag the end of the camera init registers sequence, but maybe we were getting lucky in the past. However, recently something changes and then without flagging the end of the sequence with zeros, the register write would run past the end of the registers.

Also, it looks like we duplicated one register write in a recent change

Camera power control

We should add camera power control, so the kernel module can be install without having the camera on. Add a camera_status sysfs file to prucam that is similar to the pru's status sysfs file.

Overhaul unit tests

We need to overhaul the pytest unit tests in src/python_lib/tests

Add auto exposure sysfs to python lib

i2c reads dont work

I2C reads dont work. Upon investigation, it looks like I botched the endianess when I wrote this a year ago and I never used the function so we didn't until now

Fix image capture in Python library

Reading the image from /dev/prucam and writing it to a new file is currently broken. We should use the I/O technique from testing/camera_test.py instead.

(Also, _rows and _cols are swapped in __init__, leading to transposed images.)

video4linux2 driver

We've talked about adding video4linux2 support.

This will allow non root user access to the camera as long as they belong to the "video" group. Therefor the star tracker program could run as a non root process, as the current prucam is currently forcing the star tracker program to be run as root.

Also this would remove the need for the prucam python library as OpenCV can handle interfacing to video4linux2 devices and we can let the oresat-linux-manager can handle the PRU control.

This a probably a big overhaul, so it probably won't be ready for the OreSat0 handoff, but we can patch it in orbit.

ti-pru-cgt-installer

Thank for your work.
Can you provide me more information on where I can locate the "ti-pru-cgt-installer" package?
"Install dependencies: $ sudo apt install ti-pru-cgt-installer"
I use a newly installed Ubuntu 18.04 to study your code.

Best Regards
Jinsan Ko

Debian packaging bash script

Add a bash script that check the version in the prucam kernel module and python library and make a default change log with that version before calling dpkg-buildpackage -us -uc, therefor we can use the git history with tags for our change log.

Add the statistics to the image in order for debugging purposes.

It is at times nice to preserve the properties which the image was taken from. This can be done in the fits format but for now it is nice to be able to just use opencv putText to show these.

Add color image option to python-prucam

We can use opencv to do the bayer conversion.

Gnarly kernel bug when closing /dev/prucam

I got his dump when I closed the test program that uses /dev/pru:

[10161.105345] Unable to handle kernel paging request at virtual address bf1823b4
[10161.119439] pgd = fd78acde
[10161.122188] [bf1823b4] *pgd=ad966811, *pte=00000000, *ppte=00000000
[10161.139540] Internal error: Oops: 7 [#1] PREEMPT SMP ARM
[10161.144908] Modules linked in: prucam(O) usb_f_acm u_serial pru_rproc irq_pruss_intc pruss pm33xx wkup_m3_rproc wkup_m3_ipc remoteproc virtio virtio_ring pruss_soc_bus uio_pdrv_genirq uio usb_f_ecm g_ether usb_f_rndis u_ether libcomposite spidev [last unloaded: prucam]
[10161.168883] CPU: 0 PID: 2549 Comm: python3 Tainted: G        W  O      4.19.94-ti-r55 #1buster
[10161.177533] Hardware name: Generic AM33XX (Flattened Device Tree)
[10161.183669] PC is at filp_close+0x2c/0x8c
[10161.187707] LR is at put_files_struct+0xb8/0xfc
[10161.192257] pc : [<c03151f0>]    lr : [<c033b81c>]    psr: 20030113
[10161.198551] sp : ee97fda8  ip : ee97fdc8  fp : ee97fdc4
[10161.203798] r10: ffffe000  r9 : 0000001c  r8 : 00000000
[10161.209045] r7 : ece2efc0  r6 : ec82d300  r5 : ece74d80  r4 : ec82d300
[10161.215600] r3 : bf18237c  r2 : 00000000  r1 : ec82d300  r0 : ece74d80
[10161.222159] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[10161.229326] Control: 10c5387d  Table: aceb8019  DAC: 00000051
[10161.235103] Process python3 (pid: 2549, stack limit = 0x6a48fb33)
[10161.241224] Stack: (0xee97fda8 to 0xee980000)
[10161.245604] fda0:                   ec82d300 00000000 03ffffff ece2efc0 ee97fdec ee97fdc8
[10161.253822] fdc0: c033b81c c03151d0 eccf4780 eccf4ce4 ec82d300 ece8ba3c 00000564 00000000
[10161.262040] fde0: ee97fe0c ee97fdf0 c033b900 c033b770 eccf4780 ece8ba00 ee97e000 ece8ba3c
[10161.270257] fe00: ee97fe3c ee97fe10 c0142184 c033b8c0 eccf4800 20030193 ee97fe44 ee97fe28
[10161.278474] fe20: c0169bb4 bd6da72e ecd054c4 ee97fec4 ee97fe5c ee97fe40 c0142998 c0141d00
[10161.286692] fe40: ee97ffb0 ee97ffb0 c1506e08 ee97fec4 ee97feb4 ee97fe60 c014ed5c c0142954
[10161.294909] fe60: 00000000 418004fc ef6a9b00 ef6a9b00 ee97fed8 400004d8 ecd04fc0 ecdaac00
[10161.303126] fe80: ee97fec4 ecd054c4 ef6a9b00 ee97ffb0 ee97ffb0 c1506e08 ee97fec4 00000000
[10161.311345] fea0: ee97e000 00000004 ee97ff8c ee97feb8 c010d714 c014ec34 c0d44bcc c0169a40
[10161.319563] fec0: ee97ff14 ee97fed0 c0169880 c0d44ba0 ece0e000 c01624ac 00000009 00000002
[10161.327780] fee0: ee97ff14 ee989680 c01624ac ef6a9b00 eccf4780 eccf6580 ece8ba00 ece8bc00
[10161.335999] ff00: 00000060 c0e044e8 ee97ff74 ee97ff18 c0d3ed98 c01697fc c01c195c c0731654
[10161.344216] ff20: c14cd6c0 00000000 00000010 c0d3f514 c1506e08 c1507478 c14cd690 00000000
[10161.352433] ff40: 00000000 bd6da72e ee97ff74 ffffe000 ee97ffb0 00000000 10c5387d bd6da72e
[10161.360650] ff60: ee97e000 ffffe000 ee97ffb0 00000000 10c5387d 00000000 ee97e000 00000004
[10161.368867] ff80: ee97ffac ee97ff90 c010dd9c c010d644 000ac23e a0030030 ffffffff 10c5387d
[10161.377084] ffa0: 00000000 ee97ffb0 c010106c c010dcc0 b61dd0e0 0037f8b8 b61dd0e0 00303448
[10161.385301] ffc0: b619d550 00000005 0037f8b8 000ac23d 0000000f b619d544 00000004 b621a518
[10161.393518] ffe0: 00000000 be8393e0 001244cb 000ac23e a0030030 ffffffff 00000000 00000000
[10161.401748] [<c03151f0>] (filp_close) from [<c033b81c>] (put_files_struct+0xb8/0xfc)
[10161.409534] [<c033b81c>] (put_files_struct) from [<c033b900>] (exit_files+0x4c/0x50)
[10161.417327] [<c033b900>] (exit_files) from [<c0142184>] (do_exit+0x490/0xc08)
[10161.424500] [<c0142184>] (do_exit) from [<c0142998>] (do_group_exit+0x50/0xd0)
[10161.431766] [<c0142998>] (do_group_exit) from [<c014ed5c>] (get_signal+0x134/0x96c)
[10161.439465] [<c014ed5c>] (get_signal) from [<c010d714>] (do_signal+0xdc/0x54c)
[10161.446724] [<c010d714>] (do_signal) from [<c010dd9c>] (do_work_pending+0xe8/0x100)
[10161.454419] [<c010dd9c>] (do_work_pending) from [<c010106c>] (slow_work_pending+0xc/0x20)
[10161.462631] Exception stack(0xee97ffb0 to 0xee97fff8)
[10161.467707] ffa0:                                     b61dd0e0 0037f8b8 b61dd0e0 00303448
[10161.475924] ffc0: b619d550 00000005 0037f8b8 000ac23d 0000000f b619d544 00000004 b621a518
[10161.484139] ffe0: 00000000 be8393e0 001244cb 000ac23e a0030030 ffffffff
[10161.490790] Code: e1a06001 e3530000 0a000013 e5903014 (e5933038)
[10162.019530] ---[ end trace 838ef8d6031039a5 ]---
[10162.031359] Fixing recursive fault but reboot is needed!

base error is:
Unable to handle kernel paging request at virtual address bf1823b4
But the trace does not originate from our code.

We could probably reproduce this by using the python test server to take a bunch of images with /dev/prucam open

Add CFC image sensor support

fix kernel error when you rmmod the kernel module without using it

If you rmmod the kernel module without using the camera, it will dump this trace. Usually only happens on the first one after boot:

[  700.155691] remoteproc remoteproc1: powering up 4a334000.pru
[  700.156180] remoteproc remoteproc1: Booting fw image am335x-pru0-fw, size 64336
[  700.156232] remoteproc remoteproc1: remote processor 4a334000.pru is now up
[  709.221917] prucam: Freed 1228800 bytes
[  709.231937] ------------[ cut here ]------------
[  709.231985] WARNING: CPU: 0 PID: 2275 at lib/refcount.c:187 refcount_sub_and_test_checked+0xbc/0xc4
[  709.231990] refcount_t: underflow; use-after-free.
[  709.231996] Modules linked in: prucam(O-) pru_rproc irq_pruss_intc pruss pm33xx wkup_m3_ipc wkup_m3_rproc remoteproc virtio virtio_ring pruss_soc_bus usb_f_acm u_serial uio_pdrv_genirq usb_f_ncm uio usb_f_mass_storage usb_f_rndis u_ether libcomposite spidev
[  709.232061] CPU: 0 PID: 2275 Comm: rmmod Tainted: G           O      4.19.94-ti-r43 #1buster
[  709.232067] Hardware name: Generic AM33XX (Flattened Device Tree)
[  709.232107] [<c0113e18>] (unwind_backtrace) from [<c010e388>] (show_stack+0x20/0x24)
[  709.232132] [<c010e388>] (show_stack) from [<c0d20f5c>] (dump_stack+0x8c/0xa0)
[  709.232151] [<c0d20f5c>] (dump_stack) from [<c013c928>] (__warn.part.3+0xcc/0xe8)
[  709.232164] [<c013c928>] (__warn.part.3) from [<c013c9bc>] (warn_slowpath_fmt+0x78/0x94)
[  709.232178] [<c013c9bc>] (warn_slowpath_fmt) from [<c072c98c>] (refcount_sub_and_test_checked+0xbc/0xc4)
[  709.232193] [<c072c98c>] (refcount_sub_and_test_checked) from [<c072c9ac>] (refcount_dec_and_test_checked+0x18/0x1c)
[  709.232210] [<c072c9ac>] (refcount_dec_and_test_checked) from [<c0d25c98>] (kobject_put+0x2c/0x1e0)
[  709.232222] [<c0d25c98>] (kobject_put) from [<c0d25ed8>] (kset_unregister+0x28/0x2c)
[  709.232236] [<c0d25ed8>] (kset_unregister) from [<c08dec1c>] (class_unregister+0x34/0x54)
[  709.232249] [<c08dec1c>] (class_unregister) from [<c08dec64>] (class_destroy+0x28/0x2c)
[  709.232286] [<c08dec64>] (class_destroy) from [<bf1620a8>] (prucam_exit+0xe4/0x103c [prucam])
[  709.232332] [<bf1620a8>] (prucam_exit [prucam]) from [<c01ea418>] (sys_delete_module+0x1c4/0x280)
[  709.232347] [<c01ea418>] (sys_delete_module) from [<c0101000>] (ret_fast_syscall+0x0/0x54)
[  709.232355] Exception stack(0xecb7bfa8 to 0xecb7bff0)
[  709.232367] bfa0:                   00000000 0108e1a0 0108e1dc 00000800 79308600 79308600
[  709.232379] bfc0: 00000000 0108e1a0 0108e1a0 00000081 bed35590 bed3577c 00000001 00000002
[  709.232387] bfe0: 004faf5c bed3552c 004dfd97 b6d80d68
[  709.232395] ---[ end trace a76c20d6b3f459fc ]---
[  709.233547] prucam: module exit

I2C Slave addr should not be hard coded

I did this long ago. I apologize 😁