philkroos / tinkervision Goto Github PK

Vision library for Tinkerforge RedBrick developed at FH-Bielefeld/Campus Minden.

License: GNU General Public License v2.0

C++ 86.09% Python 4.06% Makefile 3.41% C 6.44%

tinkervision's Introduction

tinkervision

Vision library for Tinkerforge RedBrick developed at FH-Bielefeld/Campus Minden. http://www.tinkerforge.com/de/doc/Hardware/Bricks/RED_Brick.html

Tinkervision consists of

a core component that manages camera input, conversion of frames between several supported formats, execution of external computer vision algorithms, and gives access to their results.
Some basic example cv-algorithms provided as modules that are loadable by the core component.

The library is intended to be loaded as part of the Tinkerforge red-brick-apid, though it can be used independently as well. However, the functionality and interface was designed for the former usage.

Requirements

The library is being developed and tested on Ubuntu 15.10, which comes with g++-4.9.2. Earlier versions of g++ may fail, though 4.8 might be fine. It is also tested to build and run on the target platform, the Tinkerforge Red Brick, which operates a version of Linux Sunxi, also with g++-4.9.2

Building and installing

The repository comes with a script to install required libraries and several Makefiles to build the core component, the modules and tests.

Executing python install_required.py should install the missing libraries of the core component.
make or DEBUG=1 make builds the library.
make install installs necessary headers and the lib under /usr/include and /usr/lib.

Building the modules

There is a top-level makefile in src/modules which builds all modules, with the exception of the streamer module. This requires additional external libraries, which can be installed with a script found in this repository. Each module can also be build and installed individually with the corresponding makefile. To build and install all:

cd src/modules
make && sudo make install
cd streamer
sudo python install_required.py
make && sudo make install

Usage

The library provides a C-interface, which can be used directly, as demonstrated in the test files in src/test/. However, it is intended to be run as part of the Tinkerforge red-brick-apid on the Tinkerforge Red-Brick. Currently, the supported image is 1.7.

Usage on the Tinkerforge `Red-Brick`

To use the library, an extended version of the red-brick-apid is needed. On the Red Brick:

mkdir /home/tf/Software && cd /home/tf/Software
git clone https://github.com/philkroos/tinkervision
git clone https://github.com/philkroos/red-brick-apid

Build and install Tinkervision and the provided modules, as described above. To build the custom version of red-brick-apid:

cd /home/tf/Software/red-brick-apid
git checkout tinkervision && cd src
WITH_VISION=yes python build_pkg.py
sudo dpkg -i redapid-2.0.2+vision_armhf.deb

The next step is to test the library bindings. This repository comes with some scripts which call into the generated Python-API. In general, all languages supported by Tinkerforge should work, but only the Python bindings have been tested yet for Tinkervision. In any case, it should work both directly on the Red-Brick and on a PC which can access the device:

mkdir /home/tf/TinkervisionTest
cd /home/tf/TinkervisionTest
git clone https://github.com/philkroos/tinkervision
git clone https://github.com/philkroos/generators
cd generators && git checkout tinkervision
mkdir /home/tf/Tinkervision/Test/tinkervision/src/test/red-brick/scripts/tinkerforge
cd python && WITH_TINKERVISION=1 python generate_python_bindings.py
cp ip_connection bindings/brick_red.py /home/tf/TinkervisionTest/tinkervision/src/test/red-brick/scripts/tinkerforge
cd /home/tf/TinkervisionTest/tinkervision/src/test/red-brick/scripts
touch tinkerforge/__init__.py

Then you should be able to execute the scripts, but you need the UID of your Red-Brick, which can be found e.g. with the Tinkerforge brickv. main.py and rb_setup.py are utilities. The other files are tests which can be executed like:

./main.py <uid> colormotion

If execution fails there may be another instance of ip_connection or brick_red.py in the system, probably from an installation of brickv. The easiest solution would be to remove all system-wide installed tinkerforge packages then, as well as the egg from Pythons site-packages.

Note

The library searches for loadable modules in two paths:

/usr/lib/tinkervision is fixed
The subdirectory lib in a custom path, which is configured at compile time:
- USER_PREFIX=/home/me/vision/ make The default prefix is /home/<current-user>/tv/. The makefile tries to create a valid directory structure.

The provided modules will be installed to the system folder by default, but if PRE is set during make install, they'll be installed to the user folder: PRE=~/tv/lib make install installs the module to the pre-configured library path.

tinkervision's People

Contributors

Watchers

Forkers

loremipsum1988

tinkervision's Issues

Fix the Callback-handling in redapid::vision.c/h

Is leaking memory
Is recreating the pipe each time. Do that in some vision_init/vision_exit-method in main.

Embed the complete software into a fork of the redapid.

So the redapi functionality can be exploited, namely the File-API.
That would allow for more rich modules and the option to retrieve images from the brick automatically.

Allow setting a framerate.

Replace the set_execution_latency with sth. like set_execution_rate(rate).

Grab a frame only if at least 'rate' ms have passed since the last grabbing.

Fix motiondetection.

Used to work, now won't detect anything.

Fix V4L2Camera

See b1ae35a.
In test/stream, 'libv4l2: error dequeuing buf: Resource temporarily unavailable' is thrown after a few seconds. This also happened in other tests (e.g. gesture), so no relation to stream.

Add logging to all parts of the software

Let the Module class create images as required by their managed objects.

Seems to be the best solution to solve the image modification problems, see message to commit 37f2ffb.

Add getter for the results of modules.

Deliver the same data as provided by the callbacks.

Remove the need for sleep after calling certain api-functions.

This is e.g. due to the asynchronous initialization method which returns id's for modules not yet persisted in the execution loop.
Probably the handling of allocated/managed modules has to be reworked. How can the excessive locking be avoided? Synchronicity? Ringbuffer?

Add complete Demos.

E.g. reimplement the balltracking thing and check for the motiontracking/streaming app.
Also some smaller modules like snapshotting and filtering, all using multiple language bindings if possible.

Crash when calling quit.

Actually this happens when the application terminates after having called quit already. Commit dev:b8804f6, test colormatch.

Use user pointer in v4l2 cam

See here http://linuxtv.org/downloads/v4l-dvb-apis/capture-example.html for an example.
Could prevent unnecessary copies.

Add method to see all available modules.

System reflection: which libraries/modules are available?

Simplify module api or define a fassade-module to be able to implement a Pluginsystem.

The current approach with variadic templates and free interfaces is nice but probably not possible to be used as the API for a Pluginsystem.

Colormatch always outputs center.

tfv-colormatch in 6f7887e.

Break the current modules down into smaller modules.

Colortracking: Maybe split into color finding module and filter, limiting the resultset.
Motiontracking: Same
Streamer: Split into conversion and streaming? Possible?

In general, focus on smaller parts and let these be composed into larger modules in form of scenes.

stack smashing with new string size

Occuring in 7d239ea tfv-test-api str_callback.

Finish the scene approach.

Implement a Plugin-API.

The whole software currently leans to be extended to a Pluginsystem. This would limit the possible interfaces, i.e. some standard methods would have to be defined aka set_parameter(name : string, value : double). That should be doable and still be usable in the context of the TF-API-generators. It should also be possible to use a similar approach to allow modules aka plugins to be written in other languages - provided an according wrapperplugin is available. This could be generated by adopting the TF-generators software?

And so on.