Secure Offline Synchronization using IBM Mobile Foundation

The increasing focus on Digital Transformation has led to more and more use cases where organizations want their enterprise apps to be usable even when the device is offline, and later synchronize the data with the enterprise when the device comes online again. In addition, organizations want to leverage the benefits of:

hybrid mobile apps where a single code base, developed using standard web technologies, works across platforms - Android, iOS and Windows phones, thereby enabling organizations to more easily embrace the policy of bring your own device (BYOD).
cloud hosted mobile backend servers for robust handling of security challenges posed by above scenarios, backend integration, app life cycle management and app analytics.

In this IBM code pattern, we will show you how to combine the following technologies to securely implement the mobile offline synchronization use case.

IBM Mobile Foundation - the enterprise grade mobile backend server available as a service on IBM Cloud,
Ionic framework - an open-source SDK for hybrid mobile app development built on top of Apache Cordova and Angular,
JSONStore - the encrypted on device storage & automated data sync for better app performance & offline access,
IBM Cloudant - a fully managed NoSQL JSON database service available on IBM Cloud, and
IBM Cloud Object Storage - A highly scalable cloud storage service, designed for high durability, resiliency and security.

When you have completed this code pattern, you will understand:

How to achieve offline user authentication in mobile apps using JSONStore.
How to store data securely on the device using encrypted JSONStore.
How to achieve downstream and upstream synchronization of data between CouchDB/Cloudant database and the device using JSONStore's automated data synchronization feature.
How to achieve downstream and upstream synchronization of images between Cloud Object Storage and the device using imgCache.js and Cordova File API.

Flow

Online scenario

When there is network connectivity, user installs and launches the mobile app, enters his/her credentials on the login screen and clicks Login.
Mobile app sends the user credentials to MFP server for validation.
MFP server invokes the security adapter logic to validate user credentials and returns an appropriate response to the mobile app. For the sake of this demo, we will use a simple security adapter that returns success when password equals username.
If user authentication succeeds, mobile app initializes JSONStore collection with the current username and password.
Mobile app initiates data synchronization with Cloudant database by making a call to MFP sync adapter.
MFP sync adapter makes REST calls to Cloudant database and returns synchronization data to the mobile app. The data fetched from Cloudant database will have references to the images stored on Cloud Object Storage.
In parallel to step 5 above, mobile app makes a call to MFP adapter to get the Authorization token for interacting with Cloud Object Storage service.
MFP adapter makes a call to Cloud Object Storage service's token manager endpoint to get the Authorization token and returns it to the mobile app.
Mobile app initializes image-caching plugin and asks it to use an HTTP header of Authorization=<value returned from MFP adapter> while fetching images.
Once JSONStore synchronization is complete and Cloud Object Storage Authorization token is fetched, mobile app displays the synchronized data from JSONStore as a list of items on the Home page. The image caching plugin running on the mobile app downloads and caches images from Cloud Object Storage.
User clicks on one of the list item to see more details. A detail page is shown consisting of image and geo-location marked inside Google Maps.
Back in the home page, user clicks on + button to report a new civic problem. A new page is shown where user can enter a description for the new civic problem as well as capture image and geo-location of the problem spot. User clicks on Submit button.
Mobile app stores the new data into JSONStore collection, which automatically initiates synchronization of the new data with Cloudant database by making a call to MFP sync adapter.
MFP sync adapter POSTs the new data to Cloudant database.
In parallel to step 14 above, mobile app creates a thumbnail image by resizing the captured image and uploads both the captured image and thumbnail to Cloud Object Storage.
Other users who click on refresh button on the home page (and those who log in afresh) are shown the updated list of problem reports.

Offline scenario

User launches the mobile app when the device is offline, enters his/her credentials on the login screen and clicks Login.
Mobile app tries to initialize the JSONStore collection with the username and password entered by user. JSONStore init succeeds only if the correct password is entered. (Recollect that the JSONStore password was set when the device was last online and user authentication had succeeded after invocation of the MFP security adapter).
If user authentication succeeds (through successful JSONStore init), mobile app reads data from the (previously synchronised) JSONStore collection, and shows the list of civic problems on the Home page.
User can click on one of the problems to see more details. In case the problem detail was previously seen when the device was online, then the problem's image would have been cached by imgcache.js, and the Cordova plugin for Google Maps would make sure that the map view works even in offline mode.
Back in the home page, user clicks on + button to report a new civic problem. A new page is shown where user can enter a description for the new civic problem as well as capture image and geo-location of the problem spot. User clicks on Submit button.
Mobile app stores the new data in JSONStore collection, and the image and its thumbnail on local file storage. Back on the Home page, user can see the new problem listed.
At at later time, when the device comes online, the mobile app automatically initiates the synchronization of JSONStore collection with Cloudant database by making a call to MFP sync adapter.
MFP sync adapter POSTs new data to Cloudant database.
In parallel to step 7 above, mobile app fetches Authorization token for interacting with Cloud Object Storage service by making a call to MFP adapter, and then uploads the new images to Cloud Object Storage.
Other users who click on refresh button on the home page (and those who log in afresh) can see the newly reported civic problem and its details.

Included Components

Cloudant NoSQL DB: A fully managed data layer designed for modern web and mobile applications that leverages a flexible JSON schema.
Cloud Object Storage: A highly scalable cloud storage service, designed for high durability, resiliency and security.
Mobile Foundation: A scalable mobile access gateway powered by the market-leading IBM Mobile Foundation Technology. The service offers a comprehensive set of mobile backend capabilities such as, App life cycle, Push, Analytics, Feature Toggle, Security and Authentication and offline synch.

Featured Technologies

Mobile: Systems of engagement are increasingly using mobile technology as the platform for delivery.

Watch the Video

Steps

1. Setup Ionic and MFP CLI
2. Create Cloudant database and populate it with sample data
3. Create IBM Cloud Object Storage service and populate it with sample data
4. Create Mobile Foundation service and configure MFP CLI
5. Download source repo and customize
6. Deploy the MFP Adapters and Test them
7. Run application on Android phone
8. Test the app functionality in offline mode
- 8.1 Test app in online mode
- 8.2 Test app in offline mode

Prerequisite steps

This project builds on top of https://github.com/IBM/Ionic-MFP-App. Run following steps from that base project to provision the needed mobile backend services from IBM Cloud and populate them with sample data, as well as to setup Ionic and MFP CLI on your development machine.

Step 5. Download source repo and customize

5.1 Clone repo

$ git clone https://github.com/IBM/MFP-JSONStore-OfflineSync.git
$ cd MFP-JSONStore-OfflineSync

5.2 Update App ID, Name and Description

Update IonicMobileApp/config.xml as below. Change id, name, description and author details appropriately.


<?xml version='1.0' encoding='utf-8'?>
<widget id="org.mycity.myward" version="2.0.0" xmlns="http://www.w3.org/ns/widgets" xmlns:cdv="http://cordova.apache.org/ns/1.0" xmlns:mfp="http://www.ibm.com/mobilefirst/cordova-plugin-mfp">
    <name>MyWard</name>
    <description>Get your civic issues resolved by posting through this app.</description>
    <author email="[email protected]" href="https://developer.ibm.com/code/author/shivahr/">Shiva Kumar H R</author>
...

5.3 Specify Cloudant credentials in MFP adapter

Open MobileFoundationAdapters/JSONStoreCloudantSync/src/main/adapter-resources/adapter.xml and update the following properties to point to the Cloudant database created in Step 2.

Update username and password with the Cloudant API key as generated in Step 2.2.
For property host, specify the Cloudant Dashboard URL portion after (and excluding) https:// and upto (and including) -bluemix.cloudant.com as shown in the snapshot of Step 2.2.
For property protocol, leave the default value of https as-is.
For property port, leave the default value of 443 as-is.
For property dbname, leave the default value of myward as-is.
For property createnewdbifnotexist, leave the default value of false as-is.


<mfp:adapter name="JSONStoreCloudantSync" ...>
  <property name="username" displayName="Cloudant Username" defaultValue=""/>
  <property name="password" displayName="Cloudant Password" defaultValue=""/>
  <property name="host" displayName="Cloudant Host" defaultValue=""/>
  <property name="protocol" displayName="DB protocol" defaultValue="https" />
  <property name="port" displayName="Db port" defaultValue="443" />
  <property name="dbname" displayName="Cloudant Database Name" defaultValue="myward"/>
  <property name="createnewdbifnotexist" displayName="Create database if it does not exist?" defaultValue="false" />
  ...
</mfp:adapter>

5.4 Specify Cloud Object Storage credentials in MFP Adapter

Open MobileFoundationAdapters/MyWardData/src/main/adapter-resources/adapter.xml and update the following properties to point to the Cloud Object Storage created in Step 3.

Specify value for bucketName as created in Step 3.1.
Specify serviceId and apiKey created in Step 3.2.
While creating the bucket in Step 3.1, if you selected a different Location/Resiliency, then update the endpointURL as per the specification in https://console.bluemix.net/docs/services/cloud-object-storage/basics/endpoints.html#select-regions-and-endpoints.


<mfp:adapter name="MyWardData" ...>
  ...
  <property name="endpointURL" displayName="Cloud Object Storage Endpoint Public URL" defaultValue="https://s3-api.us-geo.objectstorage.softlayer.net"/>
  <property name="bucketName" displayName="Cloud Object Storage Bucket Name" defaultValue=""/>
  <property name="serviceId" displayName="Cloud Object Storage Service ID" defaultValue=""  />
  <property name="apiKey" displayName="Cloud Object Storage API Key" defaultValue=""/>
</mfp:adapter>

Step 6. Deploy the MFP Adapters and Test them

6.1 Build and Deploy the MFP adapters

Build and deploy UserLogin Adapter as below.

$ cd MobileFoundationAdapters/

$ cd UserLogin
$ mfpdev adapter build
$ mfpdev adapter deploy

Note: In Step 4, if you specified No to Make this server the default?, then you need to specify the name of your server profile (Cloud-MFP in our case) at the end of mfpdev adapter deploy command as shown below.

$ mfpdev adapter deploy Cloud-MFP

Build and deploy MyWardData adapter as below.

$ cd ../MyWardData/
$ mfpdev adapter build
$ mfpdev adapter deploy

Build and deploy JSONStoreCloudantSync adapter as below.

$ cd ../JSONStoreCloudantSync/
$ mfpdev adapter build
$ mfpdev adapter deploy

6.2 Launch MFP dashboard and verify adapter configurations

Launch MFP Dashboard as below:

In the IBM Cloud dashboard, under Cloud Foundry Services, click on the Mobile Foundation service you created in Step 4. The service overview page that gets shown, will have the MFP dashboard embedded within it. You can also open the MFP dashboard in a separate browser tab by appending /mfpconsole to the url mentioned in Step 4.
Inside the MFP dashboard, in the list on the left, you will see the JSONStoreCloudantSync, UserLogin and MyWardData adapters listed.

Verify MFP Adapter configuration as below:

Inside the MFP dashboard, click on the JSONStoreCloudantSync adapter. Under Configurations tab, you should see the various properties we specified in Step 5.3 for accessing Cloudant database as shown below. As an alternative to specifying those property values in MobileFoundationAdapters/JSONStoreCloudantSync/src/main/adapter-resources/adapter.xml as previously shown in Step 5.3, you can deploy the adapters with empty defaultValue, and once the adapter is deployed, change the values on this page.

Click on Resources tab. You should see the various REST APIs exposed by JSONStoreCloudantSync adapter as shown below. The Security column should show the protecting scope UserLogin against each REST method.

6.3 Test the JSONStoreCloudantSync adapter

Create temporary credentials to test adapter REST API as below:

Inside the MFP dashboard, click on Runtime Settings. Click on Confidential Clients. Then click on New.
In the form that pops up, specify values for ID and Secret as shown in snapshot below. For Allowed Scope enter ** and click on Add. Finally click on Save.

Test adapter REST API as below:

Inside the MFP dashboard, click on the JSONStoreCloudantSync adapter. Click on Resources and then click on View Swagger Docs. The Swagger UI for adapter REST APIs will get shown in a new window/tab.
Inside the Swagger UI, click on Expand Operations.
To test the POST /modifications API, first click on OFF toggle button to enable authentication. Select UserLogin and click on Authorize as shown below. Enter the ID and Secret created above against Username and Password. Click OK. If authentication is successful, the toggle button will switch to ON position.

Specify value for parametrer db_name as myward, for parameter seqid as 0, and following JSON object under POST body:

{
  "_id": "string",
  "_rev": "string",
  "json": "[]"
}

Finally click on Try it out button to run the POST /modifications API. The API response should get shown in the Response Body as shown in snapshot below.

Delete the temporary credentials after testing adapter REST API as below:

Inside the MFP dashboard, click on Runtime Settings. Click on Confidential Clients.
Delete the Client ID created previously.

Step 7. Run application on Android phone

Follow the instructions in Step 7 of base project to run the application on Android phone.

Step 8. Test the app functionality in offline mode

How the app works in offline mode

A note on how the offline mode is supported for each of the pages^* in MyWard app: (^*Ionic page is the equivalent of iOS View or Android Activity)

Login page:
- The first time the app is launched after installation, the user login will work only in online mode. This is to make sure that the user credentials are validated by making a call to MFP security adapter. Once the user authentication succeeds, subsequent login attempts by the same user are supported in offline mode with the help of encrypted JSONStore.
Home page: (downstream sync)
- JSONStore for storing/syncing data with Cloudant.
- imgcache.js for caching the image thumbnails loaded from Cloud Object Storage.
Problem Detail page: (downstream sync)

Grievances for which the details are already seen in online mode:
- JSONStore for storing/syncing data with Cloudant.
- imgcache.js for caching the image loaded from Cloud Object Storage.
- Cordova plugin for Google Maps to make sure that the map view works even in offline mode.
Report New Problem page: (upstream sync)
- Data to be uploaded to Cloudant is stored in JSONStore which later syncs it with Cloudant when device comes online.
- Image and its thumbnail are stored on local storage, and are later uploaded to Cloud Object Storage when devices comes online.

Note: In the current implementation, images and its thumbnails are stored as-is on the local file storage at cordova.file.dataDirectory which is private to the application. If you have a more stringent compliance requirement of having to encrypt the images stored locally on the phone, then follow the recommendations on this page.

8.1 Test app in online mode

Build and run the application on your phone as per instructions in Step 7.6.
Login with username say Test and password Test. (Note: We have used a simple MFP security adapter that returns success when password equals username.)
Make sure that the Home page displays list of grievances along with image thumbnails.
Click on a few of the grievances to see their details. On the Problem Detail page, make sure that the image and Google Maps location are displayed.
Back on the Home page, click on the + icon to report a new grievance. Add description, take photo, grab geolocation and finally submit. Make sure that the new grievance is successfully submitted to server.
On a different device, whoever launches the app (or clicks on refresh button) should see your newly reported grievance.

8.2 Test app in offline mode

Have the device go offline by turning off Mobile data and Wi-Fi.
Launch the MyWard app.
Login using the same username and password as before. Make sure login succeeds even in offline mode.
Make sure that the Home page displays list of grievances along with image thumbnails.
Click on the grievances for which you had seen the details before. Make sure that the image and Google Maps location are displayed even in offline mode.
Back on the Home page, click on the + icon to report a new grievance. Add description, take photo, grab gelocation and finally submit. Make sure that the new grievance report is successfully accepted even in offline mode.
Back on the Home page, the newly reported problem should get listed at the end along with its thumbnail. Upon clicking it, the Problem Detail page should show image.
Get the device online by turning on either Mobile data or Wi-Fi.
In a while the newly reported grievance would get uploaded to server and page would get refreshed to show new list of grievances as seen on server. Make sure that the grievance that you just reported is listed as well.
On a different device, whoever launches the app (or clicks on refresh button) should see your newly reported grievance.

Troubleshooting

Please see instructions for debugging Android hybrid app using Chrome Developer Tools or troubleshooting guide for solutions to some commonly occuring problems.

Clearing JSONStore data on the phone

In case the JSONStore data on the phone gets messed up during app development/testing, then you can clear it as per the instructions below.

Install Google Chrome.
Open Google Chrome. Open URL chrome://inspect/#devices
Under Devices, click on Inspect below your connected device.
Click on Console view.
In the command prompt at the end, type following command and press enter. This would delete all the JSONStore collections on the device.

WL.JSONStore.destroy();

References

License

Apache 2.0

amolsinkar / mfp-jsonstore-offlinesync Goto Github PK

mfp-jsonstore-offlinesync's Introduction