openbase / bco.psc Goto Github PK

Use case:
A person interacts with a virtual agent multimodally and says "Turn the volume of that speaker up" and points at a speaker. He/she then moves over to a different room, says "Turn the color of that light to green" and points at a light.

The use case shows that fixed filters regarding object type or location given on the start of the application may be too restrictive. Instead the identification-component could provide an rsb server interface that allows to specify object filters during runtime such as:

• location
• object type
• services
• metaconfig flags
• ...

In the use case the filtering could restrict the objects to only objects in the room in which the person is located and another component could then select the most applicable speaker/light to do the described action on.

One could experiment using instead of the angular distance from the center of the bounding box the same distance compared to the maximal possible angular distance in the same direction.
This way the deviation for larger objects would not be punished as much as for smaller objects.
getMaximalPointOnBox() of AbstractDistanceMeasure already helps with the implementation.

The rst types in use should be extended such that selected objects as well as pointing rays can be identified to a certain tracked posture. This can be done by adding the current index of a the tracked posture to the corresponding pointing ray collections in skeleton-merging.

This way it becomes possible to also identify a person by adding face recognition to the tracked head position or by using other existing person tracking methods. It is highly relevant for multimodal interactions with the system involving multiple persons.