Giter Club home page Giter Club logo

snips-platform-android-demo's Introduction

Snips Android Demo

A demo of the Snips Platform for Android

This shows basic integration of the snips platform in an Android application.

Compiling

A simple

$ ./gradlew installDebug

should install the demo on a connected device

If you see errors while connecting to nexus.snips.ai upgrade you java to a recent version (we're using Let's Encrypt for our SSL certificates and older version of java do not support it)

Running

Make sure you have an unzipped snips assitant in the folder snips_android_assistant at the root of the external storage on you Android. You can create and download assistants on the Snips Console

You can copy the files with adb :

unzip your_assistant.zip
adb push assistant/* /sdcard/snips_android_assistant/

Click the start button in the app to start the snips-platform.

Current implementation is completly dumb and show the detected hotword/intents. A simple TTS is said when an intent is detected.

Going further

You can check the installation and usage page in our documentation.

Licence

Licenced under the Apache License, Version 2.0 (LICENSE or http://www.apache.org/licenses/LICENSE-2.0)

snips-platform-android-demo's People

Contributors

coorfun avatar cpoisson avatar fredszaq avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

snips-platform-android-demo's Issues

installation instructions

The README doesn't indicate what has to be done in order to get a working demo.

I built and installed the APK, but logcat errors indicated that some files were missing. The directory in question (/storage/emulated/0/snips_android_assistant/) can be created manually in an adb shell session, and most of the files are inside the assistant_proj*.zip file that can be downloaded from your site, and I copied them over. However, even after placing these inside snips_android_assistant, there's a remaining detail-lacking error about missing files in /storage/emulated/0/snips_android_assistant/custom_asr

App crash on android

09-04 15:57:17.068 13247-13709/ai.snips.snipsdemo E/JavaBinder: !!! FAILED BINDER TRANSACTION !!! (parcel size = 2208)
09-04 15:57:17.108 13247-13709/ai.snips.snipsdemo E/SnipsPlatform: Could not send message to Snips platform: Transaction failed on small parcel; remote process probably died
android.os.DeadObjectException: Transaction failed on small parcel; remote process probably died
at android.os.BinderProxy.transactNative(Native Method)
at android.os.BinderProxy.transact(Binder.java:503)
at android.os.IMessenger$Stub$Proxy.send(IMessenger.java:89)
at android.os.Messenger.send(Messenger.java:57)
at ai.snips.platform.SnipsPlatformClient$RemoteServiceImpl.send(SnipsPlatformClient.kt:242)
at ai.snips.platform.SnipsPlatformClient.sendAudioBuffer(SnipsPlatformClient.kt:531)
at ai.snips.snipsdemo.MainActivity.runStreaming(MainActivity.java:288)
at ai.snips.snipsdemo.MainActivity.access$300(MainActivity.java:43)
at ai.snips.snipsdemo.MainActivity$11.run(MainActivity.java:271)
09-04 15:57:17.113 13247-13247/ai.snips.snipsdemo E/SnipsPlatform: Snips processing service disconnected, attempting a reconnect
09-04 15:57:21.318 13247-13247/ai.snips.snipsdemo E/SnipsPlatform: Snips processing service disconnected, attempting a reconnect

Querying without voice

Thank you for this neat platform and helpful demo.
I am wondering if it is possible to query the platform without using voice?
In otherwords, matching text input to intent.

I am aware of dialogflow and rasa and other similar solutions but I would really like to solve everything offline.

SnipsPlatformClient crashes on android without TTS package installed

Here is the relevant stacktrace
11:32:02.832 10831-10831/? E/AndroidRuntime: FATAL EXCEPTION: main
Process: pro.i_surveillance.comfort_life_poc, PID: 10831
java.lang.RuntimeException: Unable to start activity ComponentInfo{pro.i_surveillance.comfort_life_poc/pro.i_surveillance.comfort_life_poc.MainActivity}: java.lang.NullPointerException: Attempt to invoke virtual method 'int android.speech.tts.TextToSpeech.setOnUtteranceProgressListener(android.speech.tts.UtteranceProgressListener)' on a null object reference
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2946)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3046)
at android.app.ActivityThread.-wrap11(Unknown Source:0)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1688)
at android.os.Handler.dispatchMessage(Handler.java:105)
at android.os.Looper.loop(Looper.java:164)
at android.app.ActivityThread.main(ActivityThread.java:6809)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:240)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:767)
Caused by: java.lang.NullPointerException: Attempt to invoke virtual method 'int android.speech.tts.TextToSpeech.setOnUtteranceProgressListener(android.speech.tts.UtteranceProgressListener)' on a null object reference
at ai.snips.platform.TtsHandler.registerTtsUtteranceProgressListener(TtsHandler.kt:21)
at ai.snips.platform.TtsHandler.access$registerTtsUtteranceProgressListener(TtsHandler.kt:11)
at ai.snips.platform.TtsHandler$tts$1.onInit(TtsHandler.kt:14)
at android.speech.tts.TextToSpeech.dispatchOnInit(TextToSpeech.java:824)
at android.speech.tts.TextToSpeech.initTts(TextToSpeech.java:802)
at android.speech.tts.TextToSpeech.(TextToSpeech.java:733)
at android.speech.tts.TextToSpeech.(TextToSpeech.java:712)
at android.speech.tts.TextToSpeech.(TextToSpeech.java:696)
at ai.snips.platform.TtsHandler.(TtsHandler.kt:13)
at ai.snips.platform.SnipsPlatformClient.connect(SnipsPlatformClient.kt:173)

Got error "Failed Binder transaction" when running it on android

Got the following logs when running:

05-22 00:03:26.454 9862-9901/? I/HwSystemManager: SpaceCleanReceiver:package added: ai.snips.snipsdemo
05-22 00:03:26.455 1388-11563/? I/HwSystemManager: AppCleanUpService:begin to insertAppWithPkgName, pkg:ai.snips.snipsdemo
05-22 00:03:26.479 11546-11546/ai.snips.snipsdemo:snipsProcessingService I/System.out: Found library 'snips_megazord' at libsnips_megazord.so
05-22 00:03:26.504 11546-11567/ai.snips.snipsdemo:snipsProcessingService I/snips_megazord: Starting components
05-22 00:03:26.505 11546-11567/ai.snips.snipsdemo:snipsProcessingService I/snips_megazord: Starting manual Audio-Server
Starting Hotword
05-22 00:03:26.518 11546-11574/ai.snips.snipsdemo:snipsProcessingService I/snips_hotword_lib::fe..: Audio thread for default started
05-22 00:03:26.520 11546-11567/ai.snips.snipsdemo:snipsProcessingService I/Process: Sending signal. PID: 11546 SIG: 9
05-22 00:03:26.540 2614-11578/? I/ChromeSync: [Sync,SyncIntentOperation] Handling the intent: Intent { act=android.intent.action.PACKAGE_ADDED dat=package:ai.snips.snipsdemo flg=0x4000010 cmp=com.google.android.gms/.chimera.GmsIntentOperationService (has extras) }.
05-22 00:03:26.565 11505-11505/ai.snips.snipsdemo E/SnipsPlatform: Snips processing service disconnected, attempting a reconnect
05-22 00:03:26.759 2614-11579/? I/FontsPackageChangeOp: Package ai.snips.snipsdemo has no metadata
05-22 00:03:26.814 11584-11584/ai.snips.snipsdemo:snipsProcessingService I/SnipsDemoApp: in the snips process
05-22 00:03:26.833 11505-11505/ai.snips.snipsdemo I/SnipsPlatform: Snips processing service connected, starting the platform
05-22 00:03:26.864 11584-11584/ai.snips.snipsdemo:snipsProcessingService I/System.out: Looking for library 'snips_megazord'
05-22 00:03:26.865 11584-11584/ai.snips.snipsdemo:snipsProcessingService I/System.out: Adding paths from jna.library.path: null
Trying libsnips_megazord.so
05-22 00:03:26.867 11584-11584/ai.snips.snipsdemo:snipsProcessingService W/linker: /data/app/ai.snips.snipsdemo-1/lib/arm64/libsnips_kaldi.so: unused DT entry: type 0x7ffffffd arg 0x1ced29
05-22 00:03:27.057 11584-11584/ai.snips.snipsdemo:snipsProcessingService I/System.out: Found library 'snips_megazord' at libsnips_megazord.so
05-22 00:03:27.081 11584-11606/ai.snips.snipsdemo:snipsProcessingService I/snips_megazord: Starting components
Starting manual Audio-Server
Starting Hotword
05-22 00:03:27.083 11584-11608/ai.snips.snipsdemo:snipsProcessingService I/snips_hotword_lib::fe..: Audio thread for default started
05-22 00:03:27.089 11584-11608/ai.snips.snipsdemo:snipsProcessingService I/snips_hotword_lib::fe..: Audio thread for default ended
05-22 00:03:27.091 11584-11584/ai.snips.snipsdemo:snipsProcessingService I/Process: Sending signal. PID: 11584 SIG: 9
05-22 00:03:27.135 11505-11540/ai.snips.snipsdemo E/JavaBinder: !!! FAILED BINDER TRANSACTION !!! (parcel size = 2856)
05-22 00:03:27.173 307-10528/? E/BufferQueueProducer: [ai.snips.snipsdemo/ai.snips.snipsdemo.MainActivity] query: BufferQueue has been abandoned
05-22 00:03:27.174 307-337/? E/BufferQueueProducer: [ai.snips.snipsdemo/ai.snips.snipsdemo.MainActivity] dequeueBuffer: BufferQueue has been abandoned
mali gles error info : 1
05-22 00:03:27.175 11505-11521/ai.snips.snipsdemo E/DISPLAY  GPU: mali gles error info : 1 at (vendor/hisi/thirdparty/gpu/arm/r8p0-01dev0/gles/src/state/mali_gles_state_error.c:54 - gles_state_set_error_internal)

Configure MQTT broker

Hi!
How can we configure the IP address and the port of the MQTT broker?
I would like to have this Android prototype publish actions in a dedicated broker I have subscribed on in another computer.
How can we do that?

Megazord crashes when initiating Dialog

I have an application in Google Play (beta release) that seems to be crashing when starting the dialog. I didn't see this error when testing the application locally on different devices.

The same error occurred on multiple devices, running Android 8.0 , Android 9.0, and Android 6.0. The application running Snips v0.58.3. Out of 13 installs, I have a 4 reported devices crashing.

Since the error reported is "caused by: data provided contains a nul byte" with a misspelled "nul" word, I assume this is an internal reported error. The error is also identical across all devices.

java.lang.RuntimeException: Dialogue failed to start
-> caused by: could not open file "\u{0}\u{5}\u{0}\u{0}\xDD\u{3}\u{0}\u{0}\u{16})\xA1?L\u{8}\u{0}\u{0}\u{7}\u{6}\u{0}\u{0}\x8C\u{5}\u{0}\u{0}\xB3\xBA\xA8?O\u{4}\u{0}\u{0}r\t\u{0}\u{0}\u{0}\u{0}\u{0}\u{0}\u{16}7\xC3>\xCC\u{2}\u{0}\u{0}L\n\u{0}\u{0}N\n\u{0}\u{0}\u{0}\u{0}`@/assistant.json"
-> caused by: data provided contains a nul byte
at ai.snips.platform.Megazord$Companion.parseError(Megazord.kt
  at ai.snips.platform.Megazord$Companion.access$parseError (Megazord.kt:36)
  at ai.snips.platform.Megazord.start (Megazord.kt:73)
  at ai.snips.platform.SnipsProcessingService$registerClient$1.invoke (SnipsProcessingService.kt:108)
  at ai.snips.platform.SnipsProcessingService$registerClient$1.invoke (SnipsProcessingService.kt:27)
  at kotlin.concurrent.ThreadsKt$thread$thread$1.run (Thread.kt:30)

I am not sure how to reproduce the error in development, so any insight as to the issue would be great. My concern is that the Snips APK has stability issues and its not production ready, so I would be able to fully publish an Android application built with the Snips SDK.

The application can be side-loaded using the release section of project page https://github.com/thanksmister/voice-panel-android.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.