Glassistant is a framework to write cognitive assistants that utilizes computer vision and runs on Google Glass. It is built on Gabriel infrastructure.
android
directory contains the starter code for the Android client.bin
contains the binaries to run the framework on the cloud.assistant
contains the code for the cognitive assistant.
For questions and feedback, please email [email protected]
Copyright (C) 2013-2014 Carnegie Mellon University This is a developing project and some features might not be stable yet.
All source code, documentation, and related artifacts associated with the Glassistant project are licensed under the Apache License, Version 2.0.
A copy of this license is reproduced in the LICENSE file.
Ensure the python
executable on your PATH is Python 2
with python --version
.
If you are using Python 3, setup and use a
[virtualenv][virtualenv] in an external directory
so that the python
on your PATH is Python 2.
- Initialize the virtual environment with
virtualenv -p python2.7 ~/.env-2.7
. - Use the environment with
source ~/.env-2.7/bin/activate
. - Stop using the environment with
deactivate
.
Replacing the symlink in a directory such as /usr/bin/python
is not recommended because this can potentially break
other Python 3 applications.
You will also need the following packages.
- parallel-ssh
- psutil >= 0.4.1
- JRE for UPnP
- six==1.1.0
- Flask==0.9
- Flask-RESTful==0.2.1
- OpenCV >2.3 (Run this script: https://github.com/jayrambhia/Install-OpenCV/blob/master/Ubuntu/2.4/opencv2_4_5.sh)
To install, you can either
-
run a installation script::
$ sudo apt-get install fabric openssh-server $ fab localhost install
-
install manually::
sudo apt-get install default-jre python-pip pssh python-psutil sudo pip install flask==0.9 flask-restful==0.2.1 six==1.1.0
If your default networking interface is not eth0
,
the current method to configuring other interfaces is
to replace eth0
occurrences in the following files.
<gabriel-repo>/gabriel/lib/gabriel_REST_server
<gabriel-repo>/bin/gabriel-ucomm
Described at README file of each application directory
We have tested at Ubuntu 12.04 LTS 64-bit but it should work well other version of Ubuntu with a current installation script. We expect this code also works other Linux distributions as long as you can install required package.
-
Run the
control server
from the binary directory.$ cd <gabriel-repo>/bin $ ./gabriel-control INFO Start RESTful API Server INFO Start UPnP Server INFO Start monitoring offload engines INFO * Mobile server(<class 'mobile_server.MobileVideoHandler'>) configuration INFO - Open TCP Server at ('0.0.0.0', 9098) INFO - Disable nagle (No TCP delay) : 1 INFO -------------------------------------------------- INFO * Mobile server(<class 'mobile_server.MobileAccHandler'>) configuration INFO - Open TCP Server at ('0.0.0.0', 9099) INFO - Disable nagle (No TCP delay) : 1 INFO -------------------------------------------------- INFO * Mobile server(<class 'mobile_server.MobileResultHandler'>) configuration INFO - Open TCP Server at ('0.0.0.0', 9101) INFO - Disable nagle (No TCP delay) : 1 INFO -------------------------------------------------- INFO * Application server(<class 'app_server.VideoSensorHandler'>) configuration INFO - Open TCP Server at ('0.0.0.0', 10101) INFO - Disable nagle (No TCP delay) : 1 INFO -------------------------------------------------- INFO * Application server(<class 'app_server.AccSensorHandler'>) configuration INFO - Open TCP Server at ('0.0.0.0', 10102) INFO - Disable nagle (No TCP delay) : 1 INFO -------------------------------------------------- INFO * UComm server configuration INFO - Open TCP Server at ('0.0.0.0', 9090) INFO - Disable nagle (No TCP delay) : 1 INFO --------------------------------------------------
-
Run
ucomm server
from the binary directory.$ cd <gabriel-repo>/bin $ ./gabriel-ucomm INFO execute : java -jar /home/krha/gabriel/src/control/lib/gabriel_upnp_client.jar ... INFO Gabriel Server : ... INFO connecting to x.x.x.x:9090 INFO * UCOMM server configuration INFO - Open TCP Server at ('0.0.0.0', 10120) INFO - Disable nagle (No TCP delay) : 1 INFO -------------------------------------------------- INFO Start forwarding data
If
ucomm server
is successfully connected tocontrol server
, you can see a log message "INFO User communication module is connected" atcontrol server
. -
Run the cognitive engine.
python gabriel-app-starter.py will run your code at
assistant
app -
Run a mobile client using source code at
<gabriel-repo>/android/
. Make sure to change IP address ofGABRIEL_IP
variable atsrc/edu/cmu/cs/gabriel/Const.java
.