nicolas-ivanov / rino Goto Github PK

A base for virtual personal assistant system

Java 27.93% Shell 62.11% Python 0.98% Ruby 4.88% PHP 0.41% Erlang 2.38% JavaScript 0.58% MATLAB 0.73%

rino's Introduction

Rino

Rino is a project of intelligent personal assistant for the Russian language. It's designed for smartphones with Android 4.0 and higher.

Currently there are 7 types of commands supported:

make a call
send sms to smb
send an email to smb
search smth on the web
launch some site in browser
set an alarm for particular time
check how much money you have on your account

See below the examples of users comands in Russian:

«позвони васе»
«отправь смс васе с текстом привет»
«напиши письмо васе с текстом как жизнь»
«найди в яндексе что значит дзен»
«открой википедию»
«поставь на завтра будильник на 7 30»
«покажи баланс

The entire project consists of three parts:

RinoRecognizer - represents an Android application for smartphone devices, utilises RinoLibrary
RinoServer - performs the training process of Rino's SVM classificators, utilises RinoLibrary
RinoLibrary - consists of classes, that should be identical for both RinoRecognizer and RinoServer

rino's People

Contributors

Stargazers

Watchers

Forkers

cmc-msu-ai esimpsontheartist

rino's Issues

Try/catch

Catch exceptions from Android. Make a pop-up window.

Simple patterns

Write simple pattern for recognizing phone numbers both long and short. Also write pattern for call request. Use group for number.

Hidden Markov Model

Apply HMM for command anlysing
https://code.google.com/p/jahmm/

Documentation for codes

Write a document (e.g. at google docs) to specify codes in patterns file. Special codes for operations should be specified. They should differ by a set of parameters, which are given names.

"PIck up a contact" feature

Realize a feature, that allows to pick up a contact proceeding from people names.

Subtasks:

detect names and aliases in the input sequence
get a person's contact from the contact book proceding from his name or alias
pass obtained contact to an application that requires it

Examples:

позвони маме
напиши письмо Коле
отправить Ире смску
набери Василия Ивановича

Dialog history

Save current dialog history to a data base and restore several last sentences from it on application launching.

Finite State Machine

Apply Finite State Machine for command analysing.

Installation manual

Please add INSTALL.md or "Install" section for README.md.

It would be cool if you also provided (or referred to) some info on how to get this on the phone.

Algorithms comparison

Compare these methods to find the best one for command analysing:

Regular Expressions
Finite State Machine
Hidden Markov Model

Category highlight

Highlight recognized words during the recognition process according to theirs categories to make it clear, which word is attributed to which category.

Word categories:

command
name
tel
url
email
time / date
text

See coloured examples in this doc.

Patterns in file

Put patterns in file and a way to download them on phone

Confirming operations

Make a confirmation of operations that may cost money. Pop-up window.

Interface design

Make application design, not similar to Siri

Continious recognition

Implement continious voice recognition feature as described here:
http://android-developers.blogspot.ru/2011/12/add-voice-typing-to-your-ime.html

Advantages

higher recognition speed (by the end of talking, some words of your utterance are already recognized)
better user experience (in my humble opinion)
more flexible adjustment is possible

Downsides

manual control of the recognition process:
- mark out a starting point of a phrase (the volume is higher than a predefined threshold)
- mark out a finishing point of a phrase (the silence lasts a predefined amount of time or a special key word is met, for example "всё", telling recognized to stop)
- dealing with a stream of incoming words (pass them to FSM or HMM)

I had problems with importing the project into Eclipse, so I suggest to discuss it. The main problem is in the file project.properties. In fact, I don't have the full version of all devices, because it takes a lot of space. So I always have to low down this version. I think we should either exclude this file from repository for manual parameters or set a lower version (10 or 14 there voice recognition is presented). The second question is in the Manifest. Should we really write things like this:

I had no problems with installing the app on my phone(though I have version 10), but is it ok to use such high target version?

History disappears

I met such a problem: after calling someone with no phone number, this command is in history for a while, but then is disappears after a little while. Seems like other commands have no problems with it.

Services vs Activities

Apparently,
classes SpeakButton, TextButton and CommandAnalyser should be implemented as services, not activities.

The difference:

An Activity is an application component that provides a screen with which users can interact in order to do something, such as dial the phone, take a photo, send an email, or view a map. Each activity is given a window in which to draw its user interface. (source)

A Service is an application component that can perform long-running operations in the background and does not provide a user interface.

Bound service:
A service is "bound" when an application component binds to it by calling bindService(). A bound service offers a client-server interface that allows components to interact with the service, send requests, get results, and even do so across processes with interprocess communication (IPC). A bound service runs only as long as another application component is bound to it. Multiple components can bind to the service at once, but when all of them unbind, the service is destroyed. (source)