fiware / tutorials.big-data-flink Goto Github PK

Today I was following this tutorial and run into an issue when following the steps on https://fiware-tutorials.readthedocs.io/en/latest/big-data-analysis/index.html rather than the ones on this github page.

1. orion.flink.connector-1.2.1.jar was upgraded to orion.flink.connector-1.2.3.jar
2. There are a couple of typos on the Logger subscription to context changes. Missing " and } at the end of the payload.
   curl -iX POST
   'http://localhost:1026/v2/subscriptions'
   -H 'Content-Type: application/json'
   -H 'fiware-service: openiot'
   -H 'fiware-servicepath: /'
   -d '{
   "description": "Notify Flink of all context changes",
   "subject": {
   "entities": [
   {
   "idPattern": ".*"
   }
   ]
   },
   "notification": {
   "http": {
   "url": "http://jobmanager:9001" <------
   }
   } <-------
   }'

Using GUI for uploading the jar may be cumbersome, as it didn't take the path option but showed the file browser.

Instead this command is very handy, will be nice to include as an alternate option to GUI:
curl -X POST -H "Expect:" -F "jarfile=@/cosmos-examples-1.0.jar" http://localhost:8081/jars/upload

Replace above with the path to the jar file

Also, will be nice to mention in the GUI about the 'Entry Class' option display on selecting the check box next to the jar in the GUI. It takes some time to figure this out.

Copying the request as is gives error as the closing quote after http://jobmanager:9001 is not there.

I was unable to successfully subscribe using 9001 port, I have instead used the web port 8081 with the following command:

curl -iX POST
'http://localhost:1026/v2/subscriptions/'
-H 'Content-Type: application/json'
-H 'fiware-service: openiot'
-H 'fiware-servicepath: /'
-d '{
"description": "Notify Flink of all context changes",
"subject": {
"entities": [
{
"idPattern": ".*"
}
]
},
"notification": {
"http": {
"url": "http://jobmanager:8081"
}
}
}'

I'm trying Big-Data-Flink.
However, there were two problems.

1. When I execute "./services create", the following message appears and I cannot execute it.
   docker:'compose' is not a docker command.

Looking at the services file, the variables are set as follows in the 9th to 12th lines.

dockerCmd="docker compose"
if (( $# == 2 )); then
    dockerCmd="docker-compose"
fi

So I was able to solve it by changing the 9th line of the services file as below, and execute "./services create" and "./services start".

#dockerCmd="docker compose"
dockerCmd="docker-compose"
if (( $# == 2 )); then
    dockerCmd="docker-compose"
fi

2."Logger --Reading Context Data Streams" GET request does not work.
GET request of "Logger --Subscribing to context changes"

The response when

curl -X GET \
'http://localhost:1026/v2/subscriptions/' \
-H 'fiware-service: openiot' \
-H 'fiware-servicepath: /'

is executed is described as

[
    {
        "id": "5d76059d14eda92b0686f255",
        "description": "Notify Flink of all context changes",
        "status": "active",
        "subject": {
            "entities": [
                {
                    "idPattern": ".*"
                }
            ],
            "condition": {
                "attrs": []
            }
        },
        "notification": {
            "timesSent": 362,
            "lastNotification": "2019-09-09T09:36:33.00Z",
            "attrs": [],
            "attrsFormat": "normalized",
            "http": {
                "url": "http://jobmanager:9001"
            },
            "lastSuccess": "2019-09-09T09:36:33.00Z",
            "lastSuccessCode": 200
        }
    }
]

in tutorial, but it is actually

[
    {
        "id": "5d76059d14eda92b0686f255",
        "description": "Notify Flink of all context changes",
        "status": "failed",
        "subject": {
            "entities": [
                {
                    "idPattern": ".*"
                }
            ],
            "condition": {
                "attrs": []
            }
        },
        "notification": {
            "timesSent": 362,
            "lastNotification": "2019-09-09T09:36:33.00Z",
            "attrs": [],
            "attrsFormat": "normalized",
            "http": {
                "url": "http://jobmanager:9001"
            },
            "lastFailure": "2019-09-09T09:36:33.00Z",
            "lastFailureReason": "Couldn't connect to server"
        }
    }
]

status is "failed" and lastFailureReason is "Couldn't connect to server".

What is the point I should review?

I was getting errors while running mongo-db, and for fixing this, I added the following under 'mongo-db' service:

expose:
  - "${MONGO_DB_PORT}"

Under jobmanager and taskmanager service, adding the following helped resolve the hostname:

networks:
    default: ~