sabre1041 / iot-ocp Goto Github PK

Internet of Things (IoT) OpenShift Demo Project

License: Apache License 2.0

Shell 45.03% Java 54.97%

iot-ocp's Introduction

IoT OpenShift Example Project

This example showcases an Intelligent Internet-of-Things (IoT) Gateway on Red Hat’s OpenShift Container Platform. The Gateway enables real-time decision making at the edge, and demonstrates the ease with which you can deploy a secure and scalable microservices application on the enterprise-grade OpenShift application platform.

Components

The application is deployed as a set of microservices inside containers on OpenShift:

JBoss Fuse Integration Services (FIS): Tools that facilitate integration, routing, and transformation of messages and protocols, by leveraging Apache Camel technology.
JBoss A-MQ: Apache ActiveMQ-based message broker container image for receiving and storing telemetry from the Sensor via MQTT
JBoss BRMS Decision Server: Real-time execution of business rules for processing telemetry
Apache Zeppelin: Interactive data visualization and analytics
PostgreSQL: Provides persistence for sensor data
Simulated Software Sensor: A simple Spring Boot application that simulates temperature, vibration, and GPS sensor data

Prerequisites

Git
OpenShift command-line tools for your platform
Access to an OpenShift environment
Two (2) available Persistent Volumes

Setup

Step 1: Login to your OpenShift environment using the command-line tools

Note: If you don’t have an environment handy, the Red Hat Container Development Kit provides a fully-functional preconfigured OpenShift environment, including Red Hat Enterprise Linux (RHEL) and JBoss, delivered as a Virtual Machine image. The VM should be enabled with minimum of 4 CPU and 6144MB memory (i.e. Vagrant file for CDK should be updated with this VM configuration)

Step 2: Download the iot-ocp project in the OpenShift instance:

$ git clone https://github.com/sabre1041/iot-ocp.git

Step 3: Execute the init script

$ cd iot-ocp
$ ./init.sh

The script automatically builds, deploys, and wires the necessary application components together. Once the process is complete, you can see a graphical visualization of the application architecture using the OpenShift Web Console.

Step 4: Login to the OpenShift web console using a web browser

Username: admin

Password: admin

Project: iot-ocp

Configure the Zeppelin visualization tool

The data visualization is stored in Notebooks. The interpreters allow Zeppelin to connect to PostgreSQL.

Step 5: Locate the Zeppelin service on the overview page and click on its url to launch the Zeppelin web console

Step 6: Under notebook, select Import note to import the pre-configured iot-ocp notebook

Step 7: Execute all visualizations by hitting the "Play" button on the top lefthand corner of the page next to the name of the note

Step 8: Periodically refresh the note data through cron scheduler

Starting the Gateway

To bring the project back up, the components should be started in the following order:

PostgreSQL
JBoss A-MQ
JBoss BRMS Decision Sever
JBoss Fuse Integration Services (FIS)
Simulated Software Sensor
Apache Zeppelin

Copyright and License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this package except in compliance with the License (see the LICENSE file included in this distribution). You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

iot-ocp's People

Contributors

Stargazers

Watchers

iot-ocp's Issues

OSE System Setup

I needed to do the following in order to build the sensor:

$ sudo subscription-manager register --username=rhnuser --password=rhnpasswd
$ sudo subscription-manager list --available Find pool ID for RHEL subscription
$ sudo subscription-manager attach --pool=pool_id

$ sudo yum --enablerepo=rhel-7-server-optional-rpms install maven

$ yum install java-1.8.0-openjdk-devel

Add functionality to resume init script

Add functionality to resume init script at certain points.

Initial ideas

After initial project resources (new project, imagestreams)
After postgresql
After AMQ
After KIE
After FIS
After Software Sensor

Build fails for rhel-zeppelin at the End

Hello,

Thx for the great hard work, the build fails on OpenShift Origin, does this work only with OpenShift Enterprise?

In the logs I can see:

This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. There are no enabled repos.

[root@master iot-ocp]# ./init.sh --restart-from build-zeppelin
Restarting at Step "build-zeppelin"
No resources found
No resources found

Deploying Visualization Tool...

Error from server: services "rhel-zeppelin" already exists
Error from server: routes "rhel-zeppelin" already exists
Error from server: imagestreams "rhel-zeppelin" already exists
Error from server: persistentvolumeclaims "rhel-zeppelin" already exists
Error from server: buildconfigs "rhel-zeppelin" already exists
Error from server: deploymentconfigs "rhel-zeppelin" already exists

Waiting for a running build..

Build has failed

Provisioning Failed. Execute "./init.sh --restart-from build-zeppelin" to Restart From The Failed Stage

Configure Persistent Storage of Zeppelin Components

To enable resiliency in a containerized environment, Zeppelin should be backed with persistent storage

connect external MQTT client

How does an external MQTT client connect? I was originally trying to connect via an Adafruit MQTT library (compat w/ MQTT v3) but switched to mqtt-spy to diagnose and am seeing connection failure. It looks like the broker is exposed as a route to the service that runs TCP and SSL, so what would the complete URL be? tcp://:443? It looks like TLS is required so I downloaded the cert used by Firefox to connect to the route but that cert isn't allowing mqtt-spy to connect. Exception thrown is below:

2017-06-28 19:02:46,719 INFO  [mqtt-spy [ConnectionRunnable for mqtt-iot-ocp-ssl]] [MqttAsyncConnectionRunnable   ]  - Connecting client ID [mqtt-spy] to server [[ss
l://broker-amq-mqtt-iot-ocp.192.168.42.252.nip.io:443]]; options = 
============== Connection options ==============
CleanSession                :  true
SocketFactory               :  sun.security.ssl.SSLSocketFactoryImpl@334460d9
MqttVersion                 :  0
KeepAliveInterval           :  60
ConTimeout                  :  30
UserName                    :  iotuser
SSLProperties               :  null
WillDestination             :  null
==========================================

2017-06-28 19:02:46,920 WARN  [MQTT Con: mqtt-spy            ] [MqttConnectionResultHandler   ]  - Connecting to mqtt-iot-ocp-ssl failed
MqttException (0) - javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:38)
        at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:664)
        at java.lang.Thread.run(Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1514)
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:961)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
        at org.eclipse.paho.client.mqttv3.internal.SSLNetworkModule.start(SSLNetworkModule.java:93)
        at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:650)
        ... 1 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387)
        at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
        at sun.security.validator.Validator.validate(Validator.java:260)
        at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)

Update project to reflect iot-ocp naming

Review project contents and update all references to iot-ose -> iot-ocp

when to support openshift 3.9.0

@sabre1041 dear sir, when i used your code under openshift 3.9.0 there are many image are outdated. So when do you to update project to support openshift 3.9.0? thanks!

fis-app build error

I got below error during fis-app build on cdk2.3 and still reproduce it.

=========================================================
Deploying FIS Application...

service "fis-app" created
imagestream "fis-app" created
buildconfig "fis-app" created
deploymentconfig "fis-app" created

Waiting for a running build..

Error from server: builds "fis-app-1" not found
Pulling image "registry.access.redhat.com/jboss-fuse-6/fis-karaf-openshift@sha256:48e7f7854878becec1a450e40da63b0a27da76ad63068b407ef03f3ac01cd0d1" ...
Pulling image "registry.access.redhat.com/jboss-fuse-6/fis-karaf-openshift@sha256:48e7f7854878becec1a450e40da63b0a27da76ad63068b407ef03f3ac01cd0d1" ...
Cloning "https://github.com/sabre1041/iot-ose.git" ...
Commit: 41998b9 (Merge pull request #19 from jawnsy/topics/license)
Author: Andrew Block [email protected]
Date: Fri Dec 23 18:57:07 2016 -0500

==================================================================
Starting S2I Karaf Build .....
Maven build detected
Using custom maven settings from /tmp/src/configuration/settings.xml
Found pom.xml ...
Running 'mvn -Dmaven.repo.local=/tmp/artifacts/m2 -s /tmp/src/configuration/settings.xml clean install -DskipTests '
Picked up JAVA_TOOL_OPTIONS: -Duser.home=/home/jboss -Duser.name=jboss
Apache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06; 2015-04-29T02:52:58-04:00)
Maven home: /usr/share/java/maven-3
Java version: 1.8.0_111, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.111-1.b15.el7_2.x86_64/jre
Default locale: en_US, platform encoding: ANSI_X3.4-1968
OS name: "linux", version: "3.10.0-514.el7.x86_64", arch: "amd64", family: "unix"
Picked up JAVA_TOOL_OPTIONS: -Duser.home=/home/jboss -Duser.name=jboss
[INFO] Scanning for projects...
Downloading: https://repo.fusesource.com/nexus/content/groups/public/org/apache/karaf/tooling/karaf-maven-plugin/4.0.2.redhat-621079/karaf-maven-plugin-4.0.2.redhat-621079.pom
Downloading: https://repo.fusesource.com/nexus/content/groups/ea/org/apache/karaf/tooling/karaf-maven-plugin/4.0.2.redhat-621079/karaf-maven-plugin-4.0.2.redhat-621079.pom
Downloading: https://maven.repository.redhat.com/ga/org/apache/karaf/tooling/karaf-maven-plugin/4.0.2.redhat-621079/karaf-maven-plugin-4.0.2.redhat-621079.pom
Downloading: https://repo.maven.apache.org/maven2/org/apache/karaf/tooling/karaf-maven-plugin/4.0.2.redhat-621079/karaf-maven-plugin-4.0.2.redhat-621079.pom
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[ERROR] Unresolveable build extension: Plugin org.apache.karaf.tooling:karaf-maven-plugin:4.0.2.redhat-621079 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.karaf.tooling:karaf-maven-plugin:jar:4.0.2.redhat-621079 @
[ERROR] Unknown packaging: bundle @ line 6, column 14
@
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR] The project com.redhat.examples.iot:iot-ose-routing-service:0.0.1-SNAPSHOT (/tmp/src/pom.xml) has 2 errors
[ERROR] Unresolveable build extension: Plugin org.apache.karaf.tooling:karaf-maven-plugin:4.0.2.redhat-621079 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.karaf.tooling:karaf-maven-plugin:jar:4.0.2.redhat-621079: Could not transfer artifact org.apache.karaf.tooling:karaf-maven-plugin:pom:4.0.2.redhat-621079 from/to fusesource.m2 (https://repo.fusesource.com/nexus/content/groups/public): Remote host closed connection during handshake: SSL peer shut down incorrectly -> [Help 2]
[ERROR] Unknown packaging: bundle @ line 6, column 14
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/PluginManagerException
Aborting due to error code 1 from mvn command

error: build error: non-zero (13) exit code from registry.access.redhat.com/jboss-fuse-6/fis-karaf-openshift@sha256:48e7f7854878becec1a450e40da63b0a27da76ad63068b407ef03f3ac01cd0d1

fis-app build did not complete successfully. Status is: Failed

Allow use on platforms other than RHEL (such as CentOS, Fedora)

Currently, this example project utilizes RHEL as base image, and therefore fails when not using RHEL to perform the build. See #24.

If would be preferred to allow the use CentOS as a base image by for instance a configuration option or variable.

troubleshooting Zepplin Notebook

How do I troubleshoot the Zepplin graphs? Pods are running in Openshift. My imported Notebook displays no graphs and instead displays an "ERROR" in the far right column. There are 2 rows: Counter Readings, and Number of Readings. It's almost like the Notebook import via JSON isn't working (I've tried importing a couple times)

I can send you a screenshot directly.

./init.sh --skip-steps=configure-zeppelin