Python2.7 is not supported any more:
Python 2.7.18 will be last release of python 2.7

https://www.python.org/dev/peps/pep-0373/#maintenance-releases

Some places where python2.7 is wired:
https://github.com/telefonicaid/orchestrator/blob/master/Dockerfile#L56
https://github.com/telefonicaid/orchestrator/blob/master/orchestrator.spec#L23
https://github.com/telefonicaid/orchestrator/blob/master/orchestrator.spec#L24

PR #152

By default, if no provided by conf, will be /notify, but by conf could be another different path (as /noticies)

dev.py file could be replated or a more significant file:

One way to do it could be adda import to settings/common.py

try:
    from settings_dev import *
except (ImportError, NameError):
    pass

https://github.com/telefonicaid/orchestrator/blob/master/src/orchestrator/core/mongo.py#L52

Include new roles per component and policies.

For each component (orion, sth, perseo, keypass) include a new roles (customer, admin, subServiceAdmin, subServiceCustomer) with policies just for that component.

Must be backward compatible

Hello I am testing the new orchestrator version and I have found an error. the error appear in api and script.

The postman API:
URL: http:/XXXXXX:XXXX/v1.0/service
The postman body:
{
"DOMAIN_NAME":"admin_domain",
"DOMAIN_ADMIN_USER":"cloud_admin",
"DOMAIN_ADMIN_PASSWORD": "somepass",
"DOMAIN_ADMIN_TOKEN": "somepass",
"NEW_SERVICE_NAME":"test",
"NEW_SERVICE_DESCRIPTION":" description",
"NEW_SERVICE_ADMIN_USER":"admin_test",
"NEW_SERVICE_ADMIN_PASSWORD":"somepass"
}

The response:
"(500, '')"

The script
./createNewService.py http keystone XXXX admin_domain cloud_admin somepass test dominio_test admin_test somepass http keypass XXXX

The response:
time=2018-11-13T10:26:23.747Z | lvl=INFO | corr=0b50825b-c838-4fcf-b811-69d5ec4d60d5 | trans=0b50825b-c838-4fcf-b811-69d5ec4d60d5 | srv=None | subsrv=/ | comp=Orchestrator | op=orchestrator_core:createNewService() | msg=removing uncomplete created domain 950878f9830d4df48c132a180a50a9b5
{'code': 500, 'error': "(500, '')"}

If an error happens when create new service, errors disabling and deleting service are not convenient handled and so many redundants logs are printed.

Hello everyone.

I am following the installation guide. When I execute the command below:
pip install -r requirement.txt
I have got the following error:
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DHAVE_SASL -DHAVE_TLS -DHAVE_LIBLDAP_R -DHAVE_LIBLDAP_R -DLDAPMODULE_VERSION=3.0.0 -DLDAPMODULE_AUTHOR=python-ldap project -DLDAPMODULE_LICENSE=Python style -IModules -I/usr/include/python2.7 -c Modules/LDAPObject.c -o build/temp.linux-x86_64-2.7/Modules/LDAPObject.o
In file included from Modules/LDAPObject.c:8:0:
Modules/constants.h:7:18: fatal error: lber.h: No existe el archivo o el directorio
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------

Command "/usr/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-install-WrJf1x/python-ldap/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-record-QkOxPe/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-WrJf1x/python-ldap/

Do someone knows to solve the error?
Thank you very much.
Best regard.

new container option should be added to inform that cygnus was in multiagent mode
Entrypoint should update configuration about CYGNUS ant IOT_MODULES

Two links, listed in README.md are broken.

https://pdihub.hi.inet/fiware/iotp-portal
https://pdihub.hi.inet/ep/fiware-components/wiki/Keystone-scenario-test

1 django vulnerability found in requirements.txt 14 hours ago
Remediation

Upgrade django to version 1.11.19 or later. For example:

django>=1.11.19

Always verify the validity and compatibility of suggestions with your codebase.
Details
CVE-2019-6975 More information
moderate severity
Vulnerable versions: < 1.11.19
Patched version: 1.11.19

Django 1.11.x before 1.11.19, 2.0.x before 2.0.11, and 2.1.x before 2.1.6 allows Uncontrolled Memory Consumption via a malicious attacker-supplied value to the django.utils.numberformat.format() function.

Currently orchestrator when interacts with Orion Context Broker is still using NGSIv1
https://github.com/telefonicaid/orchestrator/blob/master/src/orchestrator/core/orion.py

To be upgrade:

• deleteAllSubscriptions
• subscribeContext
• getListSubscriptions
• get_subscription_callback_endpoint
• unsubscribeContext
• extract_modules_from_subscriptions

To be deleted:

• registerContext

• getContextEntity

• updateContext

• getContextTypes

• Tested

Orchestrator uses 1.6.10 version, stable on 2015 when orchestrator was started, but now is deprecated with no security updates:

https://www.djangoproject.com/download/

Orchestrator should try to use 1.8 version of django, but upper version of django implies upper version of python, python27 at least.
Python27 in Centos implies Centos7.

Orchestrator RPM is ready to use python27.

https://docs.google.com/document/d/1RzoYyAON_6VFKCrpmh-ATUCfFabkrGN9i9AnMpnVfdg/edit#heading=h.zay0ixgoh60l

This issue is about implementing the basic API metrics operations, along with the first batch of counters (the common ones for all the IoTP components).

Operations to implement:

GET <host:port>/admin/metrics
DELETE <host:port>/admin/metrics
GET <host:port>/admin/metrics?reset=true (maybe, yet under discussion)

Syntax:

{
    "service": {
        "service1": {
            "subservs": {
                "subserv1": {},
                "subserv2": {},
                "subserv3": {}
            },
            "sum": {}
        },
        "service2": {
            "subservs": {
                "subserv1": {},
                "subserv2": {},
                "subserv3": {}
            },
            "sum": {}
        },
        "service3": {
            "subservs": {
                "subserv1": {},
                "subserv2": {},
                "subserv3": {}
            },
            "sum": {}
        }
    },
    "sum": {
        "subservs": {
            "subserv1": {},
            "subserv2": {},
            "subserv3": {}
        },
        "sum": {}
    }
}

Metrics to implement:

• incomingTransactions: number of requests consumed by the component.
• incomingTransactionRequestSize: total size (bytes) in requests associated to incoming transactions (“in” from the point of view of the component).
• incomingTransactionResponseSize: total size (bytes) in responses associated to incoming transactions (“out” from the point of view of the component).
• incomingTransacionError: number of incoming transactions resulting in error.
• serviceTime: average time to serve a transaction.
• outgoingTransactions: number of requests sent by the component.
• outgoingTransactionRequestSize: total size (bytes) in requests associated to outgoing transactions (“out” from the point of view of the component).
• outgoingTransactionResponseSize: total size (bytes) in responses associated to outgoing transactions (“in” from the point of view of the component).
• outgoingTransacionError: number of outgoing transactions resulting in error.

Note: all incoming/outgoing channels count for the above metrics.

This log is obtained:

No handlers could be found for logger "orchestrator_api"

Orchestrator is trying to delete in perseo/orion with insufficient token privileges, so these errors are false errors.

iot-orchestrator            | time=2017-08-18T10:58:41.395Z | lvl=ERROR | corr=8f0e1307-670a-47ab-98dd-33f34c39040e | trans=8f0e1307-670a-47ab-98dd-33f34c39040e | srv=smartavila | subsrv=/murallas | comp=Orchestrator | op=orchestrator_core:deleteAllSubscriptions() | msg=(401, u'The provided token does not belong to the provided service.') trying getListSubscriptions from CB: smartavila/

• Include service and subservice in earch query

https://github.com/telefonicaid/iotagent-manager

• Subscription API: https://github.com/telefonicaid/iotagent-manager/tree/develop#subscriptionapi

Hello

I have a container with orchestrator in version 2.1.1 and when I want to execute a python script happen below.
cd orchestrator/commands
python ./createNewService.py http keystone 5001 admin_domain cloud_admin password test "servicio de test" admin_test password http localhost 7070

The python script returns me this error:

Traceback (most recent call last):
File "./createNewService.py", line 36, in
from settings.common import LOGGING
ImportError: No module named settings.common

Hi,
I wanted to play with the tool and launched the docker compose. But I can't find a proper login.
The only one working is:

• user: nagios
• pwd: 4pass1w0rd
• domain: Default

once logged in i get several errors:
Internal server error. Please try again later
reference: Fiware-Correlator 471511f6-1322-4209-862e-6233da508baf

So i suppose it's not this the correct login.

Also, is there any way to access the code repository for the ui.

Cheers,
Federico

bug in how getPolicyFromRole return results
API affected: RolePolicy

time=2019-06-27T08:13:06.274Z | lvl=WARNING | corr=ec33af3f-c0f7-46d0-9994-4229ac845025 | trans=ec33af3f-c0f7-46d0-9994-4229ac845025 | srv=None | subsrv=/ | comp=Orchestrator | op=orchestrator_core:createOrionIndexes() | msg=createIndex database orion-s026pep2cbv2 exception: not enough arguments for format string
time=2019-06-27T08:13:06.278Z | lvl=WARNING | corr=ec33af3f-c0f7-46d0-9994-4229ac845025 | trans=ec33af3f-c0f7-46d0-9994-4229ac845025 | srv=None | subsrv=/ | comp=Orchestrator | op=orchestrator_core:createSTHIndexes() | msg=createIndex database sth_s026pep2cbv2 exception: not enough arguments for format string

An error trace is reported when metrics are trying to calculate errorresponse size

This is a minor bug, just a trace error was reported.

This fix solve these kind of traces:

iot-orchestrator            | time=2017-08-18T10:58:41.294Z | lvl=ERROR | corr=8f0e1307-670a-47ab-98dd-33f34c39040e | trans=8f0e1307-670a-47ab-98dd-33f34c39040e | srv=smartavila | subsrv=/murallas | comp=Orchestrator | op=orchestrator_core:collectOutgoingMetrics() | msg=ERROR collecting outgoing metrics 'URLError' object has no attribute 'headers'

orchestrator_1 | Traceback (most recent call last):
orchestrator_1 | File "/usr/lib64/python2.6/logging/init.py", line 784, in emit
orchestrator_1 | msg = self.format(record)
orchestrator_1 | File "/usr/lib64/python2.6/logging/init.py", line 662, in format
orchestrator_1 | return fmt.format(record)
orchestrator_1 | File "/usr/lib64/python2.6/logging/init.py", line 447, in format
orchestrator_1 | s = self._fmt % record.dict
orchestrator_1 | KeyError: 'correlator'

• IoTModules (default are CYGNUS, PERSEO)
• LogLevel (default is INFO)

We need to adjust correctly the orchestrator spec file for RPM generate and use.

Includes:

• remove old comments
• fix indent
• no dereference links
• assure remove link
• remove repeated directories
• not be verbose
• indent and write correctly
• remove bad create links
• move files section to the end file
• manage correctly user add orchestrator

We need to include at master and at least the release/2.10.0 (last in Platform) and release/3.0.0.

Like https://orchestrator2.docs.apiary.io/#reference/orchestrator/ldap-user

• Create Group
• Delete Group

When new Orion API will be released orchestrator should use these new API to enable pagination to retrieve subscription list
Current options:

 url='/v2/subscriptions?offset=0&limit=1000'

https://github.com/telefonicaid/orchestrator/blob/develop/src/orchestrator/core/orion.py#L288

Some options to use:
i.e.: limit=15&offset=0&options=count

Hello

I have deployment every docker like it is described in Manual.
I have executed the following command to get a token:
curl localhost:5001/v3/auth/tokens -s -S --header 'Content-Type: application/json' -d @- <<EOF
{ "auth": {
"identity": {
"methods": ["password"],
"password": {
"user":{
"name": "cloud_admin",
"domain": { "name": "admin_domain" },
"password": "4pass1w0rd"
}
}
}
}
}

I check in Mysql database that this user is created:
{
"token": {
"methods": [
"password"
],
"expires_at": "2018-08-30T11:52:40.044334Z",
"extras": {
"password_creation_time": "2018-08-30T10:42:53Z",
"last_login_attempt_time": "2018-08-30T10:48:27.000000",
"pwd_user_in_blacklist": true,
"password_expiration_time": "2019-08-30T10:42:53Z"
},
"user": {
"domain": {
"id": "ab458c371df547e486f25e29dea8705e",
"name": "admin_domain"
},
"id": "3538ae754cb44599bc7fac1e93750cd2",
"name": "cloud_admin"
},
"audit_ids": [
"1e7R-69ESdCemFuIV2B9Fg"
],
"issued_at": "2018-08-30T10:52:40.044357Z"
}
}
Keysone does not return the token.

Whenever a new service is created, ORCH has to create the indexes defined at https://fiware-orion.readthedocs.io/en/master/admin/perf_tuning/index.html#database-indexes in the entities collection the DB collection corresponding to the service (typically orion-servicename).

This would be done accessing directly to MongoDB. A MongoDB endopint should be enabled at ORCH (configuration, etc.). This MongoDB access could be used in the future for additional task.

Hello

I am running provision_smartcity.sh to create a test environment. I got the following error:
[root@6b564bd3e2b2 scenarios]# ./provision_smartcity.sh
Traceback (most recent call last):
File "./createNewService.py", line 36, in
from settings.common import LOGGING
ImportError: No module named settings.common
Error found while Creating service. Code: 1. Aborting

Does anyone know what's going on?
Best regard.

pep perseo conf is overwrited by pep perseo conf in docker

wrong pwd of iotagent and pep in docker

#9

orion subscriptions are not deleted if orion is not in the same host than orchestrator when a service or a subservice is deleted using orchestrator

Hello,

I was trying to execute the tests in order to understand better the repository and I found a little difficult to run and verify the system is properly working. It would be really helpful if the tests' documentation is updated.

Thank you very much in advance.
PS: Also the unit tests are highly coupled to external dependencies. It could be nice to decouple them.

Things detected:

• Alarms section missing (go to a troubleshooting section)
• Alarms definition
• Missing new parameters
• General review text and links

Hello,
first, thank you for the component, I'm found it really helpful to interoperate with security components.
In spite of this, it's really hard to understand what the component is really doing when a request is made. I mean, when you create a new service, requests to Keystone IDM and Keypass PAP are done creating roles and policies and no documentation is provided about those roles nor policies.
Could the documentation be extended to cover those aspects for each request?
Maybe I'm wrong and there is some guide explaining this. In that case, I'd really appreciate if you can point me out to this guide.
Thank you in advance