leapgim's Introduction

Leap Motion gesture input mapper

DEPRECATED: This project would require a legacy leap motion v2 sdk

Leapgim takes hand signs as input and outputs other types of emulated computer input.

Demo

http://www.youtube.com/watch?v=cck8rIonXgw

Installation

Prequisites: Robot.js dependencies.

Setup:

npm install leapgim

Usage

Run the service with:

npm start

Open a browser at "http://localhost:3000" for audio.

Background

Leap Motion is a sensor device for detecting detecting hand motions. A neat little piece of future technology. My first thought was that I would love to use the controller as a mouse replacement. Unfortunately the controller works with Leap-enabled software only. The goal of this project is to enable the Leap Motion controller to be used with other software also.

The basic premise of this project is that the Leap Motion controller can't replace both the mouse and the keyboard, since all the possible actions that a user could perform with a keyboard and a mouse can't feasibly be mapped into hand gestures. But it doesn't have to! A significant portion of desktop applications could be used with a small subset of possible mouse and keyboard input.

This project provides a way to define custom hand gestures and evoke actions based on those gestures. Supported actions include mouse and keyboard control, and shell scripts. Hand gesture customization allows timed, multi-step gestures to be defined, and includes as much as possible from the offical API.

Leapgim can be used as a prototyping platform to easily test a variety of hand gestures and their reliability.

Default Gesture Mapping

Mouse grab/release: Generic grabbing pose
Mouse 1: Pinch index finger
Mouse 2: Pinch ring finger
Mouse scrollwheel: Circle gesture while mouse is inactive

Leapgim takes hand signs as input and outputs other types of emulated computer input. Recipes are how signs are translated into actions. The term is borrowed from IFTTT (https://ifttt.com/).

signs -> recipe -> actions

Action Types

Mouse buttons:

Mouse button down
Mouse button up
Mouse button click
Mouse scroll up
Mouse scroll down

Mouse movement:

Move
Freeze
Unfreeze
Toggle freeze

Keyboard:

Key down
Key up
Key tap

Misc:

Compound action - execute a list of actions
Exec - run scripts
Load profile

Misc:

Compound (trigger a list of other actions)
System (run a script)
Reconfigure (load another profile / client config)

Signs

Signs are hand gestures in leapgim context. A sign can contain contain information about hand hand poses, native gestures, and time constraints.

We use the term 'sign' to provide a clear distinction from leap motion's native gestures circle, swipe, key tap and screen tap.

Application Architecture

leapgim's People

Contributors

Stargazers

Watchers

leapgim's Issues

installation failure

Hi Guys,

This project looks like it's already gotten some love.
Does this look anyway familiar? I guess some version incompatibility issue.

npm install -g leapgim
.
.
.

CXX(target) Release/obj.target/nodejavabridge_bindings/src/java.o
In file included from ../src/java.h:9:0,
                 from ../src/java.cpp:1:
../node_modules/nan/nan.h: In function ‘v8::Local<v8::Signature> NanNew(v8::Handle<v8::FunctionTemplate>, int, v8::Handle<v8::FunctionTemplate>*)’:
../node_modules/nan/nan.h:342:78: error: no matching function for call to ‘v8::Signature::New(v8::Isolate*, v8::Handle<v8::FunctionTemplate>&, int&, v8::Handle<v8::FunctionTemplate>*&)’
     return v8::Signature::New(v8::Isolate::GetCurrent(), receiver, argc, argv);
                                                                              ^
../node_modules/nan/nan.h:342:78: note: candidate is:
In file included from ../src/java.h:5:0,
                 from ../src/java.cpp:1:
.../.node-gyp/4.1.0/include/node/v8.h:4675:27: note: static v8::Local<v8::Signature> v8::Signature::New(v8::Isolate*, v8::Local<v8::FunctionTemplate>)
   static Local<Signature> New(
                           ^
.../.node-gyp/4.1.0/include/node/v8.h:4675:27: note:   candidate expects 2 arguments, 4 provided
In file included from ../src/java.h:9:0,
                 from ../src/java.cpp:1:
../node_modules/nan/nan.h: At global scope:
../node_modules/nan/nan.h:424:3: error: redefinition of ‘v8::Local<T> NanNew(P) [with T = v8::StringObject; P = v8::Local<v8::String>]’
   NanNew<v8::StringObject, v8::Handle<v8::String> >(
   ^
../node_modules/nan/nan.h:417:3: error: ‘v8::Local<T> NanNew(P) [with T = v8::StringObject; P = v8::Local<v8::String>]’ previously declared here
   NanNew<v8::StringObject, v8::Local<v8::String> >(
   ^
../node_modules/nan/nan.h:442:36: error: redefinition of ‘template<class T> v8::Local<v8::RegExp> NanNew(v8::Local<v8::String>, v8::RegExp::Flags)’
   NAN_INLINE v8::Local<v8::RegExp> NanNew(
                                    ^
../node_modules/nan/nan.h:436:36: error: ‘template<class T> v8::Local<v8::RegExp> NanNew(v8::Handle<v8::String>, v8::RegExp::Flags)’ previously declared here
   NAN_INLINE v8::Local<v8::RegExp> NanNew(
                                    ^
../node_modules/nan/nan.h:454:36: error: redefinition of ‘template<class T, class P> v8::Local<v8::RegExp> NanNew(v8::Local<v8::String>, v8::RegExp::Flags)’
   NAN_INLINE v8::Local<v8::RegExp> NanNew(
                                    ^
../node_modules/nan/nan.h:448:36: error: ‘template<class T, class P> v8::Local<v8::RegExp> NanNew(v8::Handle<v8::String>, v8::RegExp::Flags)’ previously declared here
   NAN_INLINE v8::Local<v8::RegExp> NanNew(
                                    ^
../node_modules/nan/nan.h:623:19: error: ‘NanNew’ declared as an ‘inline’ variable
       v8::String::ExternalAsciiStringResource *resource) {
                   ^
../node_modules/nan/nan.h:623:19: warning: ‘always_inline’ attribute ignored [-Wattributes]
../node_modules/nan/nan.h:623:19: error: ‘v8::Local<v8::String> NanNew’ redeclared as different kind of symbol
../node_modules/nan/nan.h:617:36: error: previous declaration of ‘v8::Local<v8::String> NanNew(v8::String::ExternalStringResource*)’
   NAN_INLINE v8::Local<v8::String> NanNew(
                                    ^
../node_modules/nan/nan.h:623:7: error: ‘ExternalAsciiStringResource’ is not a member of ‘v8::String’
       v8::String::ExternalAsciiStringResource *resource) {
       ^
../node_modules/nan/nan.h:623:48: error: ‘resource’ was not declared in this scope
       v8::String::ExternalAsciiStringResource *resource) {
                                                ^
../node_modules/nan/nan.h:637:27: error: redefinition of ‘template<class T> v8::Local<T> _NanEscapeScopeHelper(v8::Local<T>)’
   NAN_INLINE v8::Local<T> _NanEscapeScopeHelper(v8::Local<T> val) {
                           ^
../node_modules/nan/nan.h:632:27: error: ‘template<class T> v8::Local<T> _NanEscapeScopeHelper(v8::Handle<T>)’ previously declared here
   NAN_INLINE v8::Local<T> _NanEscapeScopeHelper(v8::Handle<T> val) {
                           ^

I don't mind installing directly from the repo, but not sure if the installation and running instructions are up-to-date, so maybe you can confirm, and/or say if you keep a certain branch stable.

gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04)

javac 1.7.0_80 but I also have java 8 if that's preferred.

Just installed the latest LeapMotion from https://developer.leapmotion.com/downloads/skeletal-beta?platform=linux&version=2.3.1.31549.

Thanks for having a look!

Windows Installation Issue

I am trying to install leapgim, its been two full days but not yet succeeded. Actually everything is new for me. I found your project very powerful solution in order to program leap as replacement for mouse and key board.
Can you please share some more details for installation the leapgim on windows 7 32/64 bit platform. I tried to install exact version specified in Troubleshooting Dependencies, but no success. Please, can you write complete installation details considering a newbie.

Appreciate and thanks.

Recommend Projects

zeukkari / leapgim Goto Github PK