botmill / fb-botmill Goto Github PK

It would be nice to have a collection of examples and tutiorials for facebot. Currently there are some classes under src/test/java/demo but I think it would be better to have the tutorial under a separate project. For this purpose I've created this repository: https://github.com/Aurasphere/facebot-examples.

We should implement a way to use conversation flows.

Some features of FaceBot are not yet documented on the wiki. These includes the ReplyFactory object and the templates. We should add these pages.

It would be nice to add the possibility of adding more AutoReply in response to one event. This can be achieved at the moment by adding more action frames with the same event and setting the FaceBotPolicy to PROCESS_ALL. This works fine but there should be a method which takes as parameters one event and more AutoReply which does this automatically for you. Also, this may be applied the other way around with a method which takes only one AutoReply and binds it to multiple events.

This should be very easy as well. We need to implement the new buttons on the platform. Here's the reference:

Buy: https://developers.facebook.com/docs/messenger-platform/send-api-reference/buy-button
Share: https://developers.facebook.com/docs/messenger-platform/send-api-reference/share-button
Log in: https://developers.facebook.com/docs/messenger-platform/account-linking/link-account
Log Out: https://developers.facebook.com/docs/messenger-platform/account-linking/unlink-account

We need to support the quick reply of location and as well as it's respective payload handling.

When the user sends his/her location, an incoming payload (from Income message attachment) will include a latitude and a longitude. We need to introduce a handler for this.

Adding support for the Facebook Receipt Template with fluent builders.

I was thinking of introducing a new feature that will improve the network handler. Basicallly I will add a registerNetworkMonitor(FbBotMillNetworkMonitor) on the FbBotMillContext. FbBotMillNetworkMonitor is an interface with two methods: onResponse(FbMessengerCallback) and onError(FacebookErrorMessage). Each of this methods will be used as a callback when a message is received from the Messenger platform, according to the kind of message. This will make the developers access data that are otherwise unavailable and may be useful in some situations. This will also be really good for unit testing.

We need to limit/control the logs being generated by the bot.

Here's Facebook documentation on this: https://developers.facebook.com/docs/messenger-platform/user-profile. It's not difficult. We should be able to implement this.

Some classes are missing Javadoc. Since this is needed in order to publish FaceBot on Maven repository we should address this issue as well.

We need to incorporate PROGRAM-AB to support AIML as input.

List Template was introduced recently and we need to support this feature.

We need to introduce a SprintBoot context to handle the get and post callbacks. This will be an alternative for developers that doesn't want to use web.xml/xml or servlet configuration.

The bot seems to be rendering it perfectly but an exception is thrown every time it does it

2017-01-01T01:11:44.084430+00:00 app[web.1]: 01:11:44.084 [http-nio-43734-exec-6] ERROR c.a.botmill.fb.FbBotMillServlet - Error during MessengerCallback parsing:
2017-01-01T01:11:44.084432+00:00 app[web.1]: java.lang.NullPointerException: null
2017-01-01T01:11:44.084433+00:00 app[web.1]: at com.google.gson.internal.$Gson$Preconditions.checkNotNull($Gson$Preconditions.java:39) ~[gson-2.7.jar:na]
2017-01-01T01:11:44.084433+00:00 app[web.1]: at com.google.gson.reflect.TypeToken.(TypeToken.java:72) ~[gson-2.7.jar:na]
2017-01-01T01:11:44.084434+00:00 app[web.1]: at com.google.gson.reflect.TypeToken.get(TypeToken.java:296) ~[gson-2.7.jar:na]
2017-01-01T01:11:44.084435+00:00 app[web.1]: at com.google.gson.Gson.fromJson(Gson.java:885) ~[gson-2.7.jar:na]
2017-01-01T01:11:44.084436+00:00 app[web.1]: at com.google.gson.Gson.fromJson(Gson.java:952) ~[gson-2.7.jar:na]
2017-01-01T01:11:44.084436+00:00 app[web.1]: at com.google.gson.internal.bind.TreeTypeAdapter$GsonContextImpl.deserialize(TreeTypeAdapter.java:162) ~[gson-2.7.jar:na]

We should add the possibility of include a botmill.properties file in the classpath with the tokens instead of calling the setup method.

I'm planning a datatype review after the first FaceBot official review.

This will include:

• incoming datatypes converted to enum and Date (or Calendar, not sure) when possible.
• outcoming decimal datatypes converted to java.math.BigDecimal and BigInteger when possible.

We should check if this is the case.

FaceBotNetworkAwareBean is the base class of the elements on FaceBot that requires to do network operations. These are: AutoReply (and children), FaceBotMockMediator and FaceBotThreadSettingsConfiguration. As a consequence, whenever you extend one of this class (this is mostly valid for AutoReplies since they are likely to be extended), all the methods for do network operations are inherited. and thus visible to the developer. Should we keep it like this? Or would be better if we hide this methods somehow?

I'm trying to automatically reply to the first event sent by facebook :

JSON input: {"object":"page","entry":[{"id":"683477205158863","time":1485793125243,"messaging":[{"recipient":{"id":"683477205158863"},"timestamp":1485793125243,"sender":{"id":"1108634062596634"},"postback":{"payload":"FACEBOOK_WELCOME"}}]}]}

But I've not found code snippet to manage this case. Do you have suggestions?

We need to define a new feature/enhancement to implement a state management functionality. The state management will take care of storing user input.

We should implement a fluent builder for the BuyButton since it's a very complex object and the ButtonFactory alone can't manage it.

Create a Wiki Guide with Screenshot of each ActionFrame response.

Currently the reply factory object will allow only to attach files using URLS (audio, video, images and generic files attachments). Facebook API's also allows to upload a file directly. It would be nice to add this feature to the framework. Here's an instance of how to do it with cURL: https://developers.facebook.com/docs/messenger-platform/send-api-reference/audio-attachment.
In order to do this with FaceBot, we should add a method that makes a POST with multipart/data.
I've made some tests with Postman and the API works even if you don't specify the extension of file you are attaching, so we don't need to add that.

https://developers.facebook.com/docs/messenger-platform/webhook-reference/pre-checkout

The echo message is missing from the received message. Instead it gets parsed as a ReceivedMessage (the same classes used for the text messages). Fixing this will need to introduce an interface and a custom JSON deserializer that switch the implemetation to use according to the field is_echo_message.

We are missing some thread settings. These are really easy to implement. The references are here:

Payment: https://developers.facebook.com/docs/messenger-platform/thread-settings/payment
Account Linking: https://developers.facebook.com/docs/messenger-platform/thread-settings/account-linking
Domain Whitelisting: https://developers.facebook.com/docs/messenger-platform/thread-settings/domain-whitelisting

Whenever I try to create a get started button via FbBotMillThreadSettingsConfiguration.setGetStartedButton("Get Started Button Payload"); it gives me the following exception:

java.lang.NullPointerException: null
	at co.aurasphere.botmill.fb.internal.util.json.ButtonSerializer.getButtonClass(ButtonSerializer.java:92) ~[classes/:na]
	at co.aurasphere.botmill.fb.internal.util.json.ButtonSerializer.serialize(ButtonSerializer.java:63) ~[classes/:na]
	at co.aurasphere.botmill.fb.internal.util.json.ButtonSerializer.serialize(ButtonSerializer.java:1) ~[classes/:na]
	at com.google.gson.internal.bind.TreeTypeAdapter.write(TreeTypeAdapter.java:81) ~[gson-2.7.jar:na]
	at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:69) ~[gson-2.7.jar:na]
	at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:97) ~[gson-2.7.jar:na]
	at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:61) ~[gson-2.7.jar:na]
	at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:69) ~[gson-2.7.jar:na]
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:125) ~[gson-2.7.jar:na]
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:243) ~[gson-2.7.jar:na]
	at com.google.gson.Gson.toJson(Gson.java:669) ~[gson-2.7.jar:na]
	at com.google.gson.Gson.toJson(Gson.java:648) ~[gson-2.7.jar:na]
	at com.google.gson.Gson.toJson(Gson.java:603) ~[gson-2.7.jar:na]
	at com.google.gson.Gson.toJson(Gson.java:583) ~[gson-2.7.jar:na]
	at co.aurasphere.botmill.fb.internal.util.json.JsonUtils.toJson(JsonUtils.java:116) ~[classes/:na]
	at co.aurasphere.botmill.fb.internal.util.network.NetworkUtils.toStringEntity(NetworkUtils.java:358) [classes/:na]
	at co.aurasphere.botmill.fb.internal.util.network.NetworkUtils.postThreadSetting(NetworkUtils.java:162) [classes/:na]
	at co.aurasphere.botmill.fb.threadsettings.FbBotMillThreadSettingsConfiguration.setGetStartedButton(FbBotMillThreadSettingsConfiguration.java:132) [classes/:na]

We need to introduce a SparkJava context to handle the get and post callbacks. This will be an alternative for developers that doesn't want to use web.xml/xml or servlet configuration.

I would like to have my bot embedded on multiple Facebook pages. Is there a way to create FbBotMillContext based on PageToken or better way to handle this scenario?

We need to create a plugin to integrate messages from api.ai. Make sure that our framework can utilize the open endpoint services that api.ai has. This will be a crucial feature for our framework since we can take advantage of api.ai offerings.

This is a separate plugin project.

I'd like to add different methods to configure the payments thread settings in order to simplify the configuration tasks. the methods are: addTesters(List), removeTesters(List), addPublicKey(String), addPrivacyUrl(String). This will let configure the settings without knowledge of the underlying model.

These two enums are not correctly deserialized in incoming callbacks.

Thank you for this awesome work.

I want to contribute to an empty response type. Namely, an EmptyAutoReply class. For this type of response, nothing will be sent to facebook but there will be some changes applied on server. I needed it to achieve state management with a better software design. And currently for this purpose, I am using new ActionAutoReply(TypingAction.TYPING_OFF).

If you want, I can make pull requests by the following usage:

addActionFrame(event, new MessageAutoReply("just message"), new EmptyReply(){

        @Override
        public void createEmptyResponse(MessageEnvelope envelope) {
                // do something here
        }
    }
});

Also, I want to make pull request for AWS Lambda context(which I already have done) and its setup guidelines.

So, what is your opinion?

At the moment the ListTemplateElement is both model and a fluent builder. We want to split these behaviors like we did for other components.

It seems that all Event is having issues passing the template type.

02:29:15.738 ERROR [main][NetworkUtils] HTTP connection failed with error code 400.
02:29:15.739 ERROR [main][NetworkUtils] Error message from Facebook. Message: [(#100) Missing template_type for attachment's payload], Code: [100], Type: [OAuthException], FbTraceID: [DtlNZ2Ig9GY].
Sent!

Tested this with:

• List Template
• Generic Template
• Button Template

Please take a look.

Facebook introduced this new callbacks:

Referral: https://developers.facebook.com/docs/messenger-platform/webhook-reference/referral
Payment: https://developers.facebook.com/docs/messenger-platform/webhook-reference/payment
Checkout: https://developers.facebook.com/docs/messenger-platform/webhook-reference/checkout-update

These should not be difficult to implement because it's all about modelling.

I configured the server instance and seems to work fine.
I got the message but the deserialization goes in error:

What I'm doing wrong ?

My code:

`
public class MyFbBotDefinitionClass extends AbstractFbBot {
public void defineBehavior() {

    // Setting my tokens from Facebook (page token and validation token for webhook).
    FbBotMillContext.getInstance().setup("real_token", "real_validation_token");
    // Defining a bot which will reply with "Hello World!" as soon as I write "Hi"
    addActionFrame(new MessageEvent("Hi"),new MessageAutoReply("Hello World of Fluttr!"));
    
}

}
`

Stacktrace

2017-01-30T12:58:48.827982+00:00 app[web.1]: 12:58:48.827 [http-nio-17732-exec-5] DEBUG c.a.botmill.fb.FbBotMillServlet - JSON input: {"object":"page","entry":[{"id":"683477205158863","time":1485781128667,"messaging":[{"sender":{"id":"1108634062596634"},"recipient":{"id":"683477205158863"},"timestamp":1485781128631,"message":{"mid":"mid.1485781128631:03b7a3eb69","seq":32271,"text":"ciao"}}]}]}
2017-01-30T12:58:48.830507+00:00 app[web.1]: 12:58:48.830 [http-nio-17732-exec-5] ERROR c.a.botmill.fb.FbBotMillServlet - Error during MessengerCallback parsing:
2017-01-30T12:58:48.830509+00:00 app[web.1]: java.lang.NullPointerException: null
2017-01-30T12:58:48.830510+00:00 app[web.1]: at co.aurasphere.botmill.fb.internal.util.json.IncomingMessageDeserializer.deserialize(IncomingMessageDeserializer.java:56) ~[fb-botmill-1.1.0.jar:na]
2017-01-30T12:58:48.830511+00:00 app[web.1]: at co.aurasphere.botmill.fb.internal.util.json.IncomingMessageDeserializer.deserialize(IncomingMessageDeserializer.java:44) ~[fb-botmill-1.1.0.jar:na]
2017-01-30T12:58:48.830512+00:00 app[web.1]: at com.google.gson.internal.bind.TreeTypeAdapter.read(TreeTypeAdapter.java:69) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830512+00:00 app[web.1]: at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:129) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830513+00:00 app[web.1]: at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:220) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830514+00:00 app[web.1]: at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:41) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830515+00:00 app[web.1]: at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:82) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830515+00:00 app[web.1]: at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:61) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830516+00:00 app[web.1]: at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:129) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830517+00:00 app[web.1]: at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:220) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830518+00:00 app[web.1]: at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:41) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830518+00:00 app[web.1]: at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:82) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830519+00:00 app[web.1]: at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:61) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830520+00:00 app[web.1]: at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:129) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830521+00:00 app[web.1]: at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:220) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830522+00:00 app[web.1]: at com.google.gson.Gson.fromJson(Gson.java:887) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830522+00:00 app[web.1]: at com.google.gson.Gson.fromJson(Gson.java:852) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830523+00:00 app[web.1]: at com.google.gson.Gson.fromJson(Gson.java:801) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830526+00:00 app[web.1]: at com.google.gson.Gson.fromJson(Gson.java:773) ~[gson-2.7.jar:na]
2017-01-30T12:58:48.830526+00:00 app[web.1]: at co.aurasphere.botmill.fb.internal.util.json.JsonUtils.fromJson(JsonUtils.java:103) ~[fb-botmill-1.1.0.jar:na]
2017-01-30T12:58:48.830527+00:00 app[web.1]: at co.aurasphere.botmill.fb.FbBotMillServlet.doPost(FbBotMillServlet.java:201) ~[fb-botmill-1.1.0.jar:na]
2017-01-30T12:58:48.830528+00:00 app[web.1]: at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830528+00:00 app[web.1]: at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830529+00:00 app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830530+00:00 app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830531+00:00 app[web.1]: at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:217) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830531+00:00 app[web.1]: at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830532+00:00 app[web.1]: at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830533+00:00 app[web.1]: at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830533+00:00 app[web.1]: at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830534+00:00 app[web.1]: at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830535+00:00 app[web.1]: at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830535+00:00 app[web.1]: at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1091) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830540+00:00 app[web.1]: at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:673) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830552+00:00 app[web.1]: at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830555+00:00 app[web.1]: at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830556+00:00 app[web.1]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_102-cedar14]
2017-01-30T12:58:48.830556+00:00 app[web.1]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_102-cedar14]
2017-01-30T12:58:48.830557+00:00 app[web.1]: at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.0.28.jar:8.0.28]
2017-01-30T12:58:48.830558+00:00 app[web.1]: at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102-cedar14]

The problem seems to be :

boolean isEcho = json.getAsJsonObject().get("is_echo").getAsBoolean();

in co.aurasphere.botmill.fb.internal.util.json.IncomingMessageDeserializer.deserialize(..) method

We should add support for web view. Here's Facebook documentation on this: https://developers.facebook.com/docs/messenger-platform/messenger-extension. I haven't looked into details yet.

We need to create a plugin to integrate messages from wit.ai. This should be a separate project.

Incoming location is currently being passed through from the payload of the attachment. Since payload object is an interface, we need to find a way to get the payload string and map that to a coordinate (lat/lang) object.

I'm investigating and will issue a fix.

If reply contains for example Scandinavian or Spanish "special" characters, they are shown incorrectly. Example:

String test = "For example Scandinavian characters: työssä, Spanish: Habitación.";
return ReplyFactory.addTextMessageOnly(test).build(envelope);

Result in Messenger: For example Scandinavian characters: ty?ss?, Spanish: Habitaci?n.
Expected result in Messenger: For example Scandinavian characters: työssä, Spanish: Habitación.
Potential fix: In NetworkUtils::toStringEntity replace input = new StringEntity(json); with input = new StringEntity(json, "UTF-8");
Other things that may affect to the result in Messenger: User seeing the problem has locale fi_FI. Not tested with other user locales.
Other: Request seems to handle these "special" characters correctly.

https://developers.facebook.com/docs/messenger-platform/webhook-reference#validation

https://developers.facebook.com/docs/messenger-platform/webhook-reference#subscribe