Executing install workflow was failed in /neutron_plugin/floatingip.py.

My Environments

• CentOS 7.4
  • Cloudify Manager 4.3
• cloudify-openstack-plugin 2.7.1

Error Message

[__main__] ERROR: Task neutron_plugin.floatingip.create[6359de12-4d2b-44aa-ad97-e7a09f6f6538] raised:

Traceback (most recent call last):

  File "/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify/dispatch.py", line 641, in main

    payload = handler.handle()

  File "/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify/dispatch.py", line 397, in handle

    result = self.func(*self.args, **kwargs)

  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-2.7.1/lib/python2.7/site-packages/openstack_plugin_common/__init__.py", line 850, in wrapper

    return f(*args, **kw)

  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-2.7.1/lib/python2.7/site-packages/neutron_plugin/floatingip.py", line 67, in create

    network_from_rel = connected_network.runtime_properties[

AttributeError: 'NodeContext' object has no attribute 'runtime_properties'

Blueprints

  floating_ip:
    type: cloudify.openstack.nodes.FloatingIP
    properties:
      openstack_config: { get_input: openstack_configuration }
      floatingip:
        floating_network_name: { get_input: external_network_name }
    relationships:
      - type: cloudify.relationships.depends_on
        target: external_network_subnet
      - type: cloudify.relationships.depends_on
        target: external_network

Source Code

Error Message means connected_network is 'NodeContext' object wchich has no runtime_properties.

The definition of connected_network is get_single_connected_node_by_openstack_type() which return 'NodeContext' object.

• floatingip.py
  https://github.com/cloudify-cosmo/cloudify-openstack-plugin/blob/master/neutron_plugin/floatingip.py#L67

#62    connected_network = \
#63        get_single_connected_node_by_openstack_type(
#64         ctx, NETWORK_OPENSTACK_TYPE, True)
#65
#66    if connected_network:
#67        network_from_rel = connected_network.runtime_properties[
#68                             OPENSTACK_ID_PROPERTY]

• get_single_connected_node_by_openstack_type()
  https://github.com/cloudify-cosmo/cloudify-openstack-plugin/blob/master/openstack_plugin_common/__init__.py#L263

#263   def get_single_connected_node_by_openstack_type(
#264       ctx, type_name, if_exists=False):
#265    nodes = get_connected_nodes_by_openstack_type(ctx, type_name)
#266   check = len(nodes) > 1 if if_exists else len(nodes) != 1
#267    if check:
#268       raise NonRecoverableError(
#269            'Expected {0} one {1} node. got {2}'.format(
#270                'at most' if if_exists else 'exactly', type_name, len(nodes)))
#271    return nodes[0] if nodes else None

Test Code

get_single_connected_node_by_openstack_type was changed MockNodeInstanceContext() in Test Code.
I think connected_network was expected to be 'NodeInstanceContext' object.

• mock.patch( 'get_single_connected_node_by_openstack_type', MockNodeInstanceContext() )
  https://github.com/cloudify-cosmo/cloudify-openstack-plugin/blob/master/neutron_plugin/tests/test_floating_ip.py#L35
  https://github.com/cloudify-cosmo/cloudify-openstack-plugin/blob/master/neutron_plugin/tests/test_floating_ip.py#L49

#18    def test_network_rel(self, cfy_local, *_):
#19       def _mock_rel(*_):
#20            return MockNodeInstanceContext(runtime_properties={
#21               OPENSTACK_ID_PROPERTY: 'my-id'
#22            })
#23
#24        def _mock_create(_, fip):
#25            self.assertEqual(fip[FLOATINGIP_OPENSTACK_TYPE][
#26                                 'floating_network_id'], 'my-id')
#27            return {FLOATINGIP_OPENSTACK_TYPE: {
#28                'id': '1234',
#29                'floating_ip_address': '1.2.3.4'
#30            }}
#31
#32       with mock.patch('neutronclient.v2_0.client.Client.create_floatingip',
#33                        new=_mock_create):
#34            with mock.patch(
#35                    'neutron_plugin.floatingip.get_single_connected_node_by_'
#36                    'openstack_type', new=_mock_rel):
#37                            cfy_local.execute('install')

A request for a new node type to be able to create an openstack volume type which is apparently not (yet) supported in the mentioned OpenStack plugin.

Command required:
openstack volume type create --description "for vIMS cfx cbam scripts to work" --private --project imvilb2 tripleo-ceph

anyone has awesome blog about it，seeking for help

Cloudify composer can not import openstack types .. complaining that plugin can not be parsed.

Tried 2.0.1 and b3.1 without success

Currently, username, password and auth_url are required.
However, some customers use openstack-config.json as the provider of certain (or all) OpenStack connection properties.
Therefore, from a DSL point of view, it is OK to have the aforementioned keys empty.

Environment:

• Cloudify Manager 3.4.1.
• OpenStack Plugin 2.7.4.

I use the "helloworld" blueprint. It creates a port with a security group:

  port:
    type: cloudify.openstack.nodes.Port
    properties:
      fixed_ip: 10.0.0.3
    relationships:
      - type: cloudify.relationships.contained_in
        target: private_network
      - type: cloudify.relationships.depends_on
        target: private_subnet
      - type: cloudify.openstack.port_connected_to_floating_ip
        target: ip
      - type: cloudify.openstack.port_connected_to_security_group
        target: security_group

  security_group:
    type: cloudify.openstack.nodes.SecurityGroup
    properties:
      security_group:
        description: generic security group
      rules:
        - remote_ip_prefix: 0.0.0.0/0
          port:
            get_input: webserver_port

The security group is created successfully. But the port fails with error "Port must be in one SecurityGroup" even we already defined the relationship between the port and the security group.

2018-05-08 12:17:33,828 [keystoneauth.session] DEBUG: REQ: curl -g -i -X POST https://.../v2.0/ports.json -H "User-Agent: python-neutronclient" -H "Content-Type
: application/json" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}..." -d '{"port": {"network_id": "f4b0b5c1-...", 
"fixed_ips": [{"subnet_id": "d2bb51da-...", "ip_address": "10.0.0.3"}], "name": "port_helloworld_port_axo7fz", "security_groups": []}}'

RESP BODY: {"NeutronError":{"message":"Port must be in one SecurityGroup.","type":"InvalidInput","detail":""}}

• Remove Deprecated Functions
• Research to find what other functions were deprecated.
• Deprecate Old relationships (Server connected to floating IP --> Should use Port connected to floating IP)
• Upgrade all fucntions to use openstacksdk library.

In the image.py plugin the _validate_image function does not allow a bypass of the _check_url.

This could be applicable in the scenario where the OpenStack enviornment has access to the repository but Cloudify/NFVO does not.

Therefore there should be a flag to bypass the _check_url when the network does not support it.

If you execute create operation on a resource and the resource fails either during the create or after for another reason, we should cleanup the resource's runtime properties.

Similarly, we need to make delete operations infer when a resource has been cleaned up from a failed create.

In NfV realm many of the VMs will be routers, hence the arp filtering must be disabled.

How can I get a nonfiltered port in Cloudify to give to my VMs ??

Currently, when using a floating IP node with use_external_resource, the plugin calls OpenStack to associate the floating IP with the instance.

If the floating IP is already in use (that is, already in ACTIVE state), OpenStack just steals it from the other VM.

While this is a possibly legitimate use case, it presents a problem in production environments as stealing IP addresses like this is rarely expected and acceptable.

We should add a property such as allow_reallocation, defaulting to False, which controls whether reallocation is allowed.

It is impossible to specify a CA cert to use for authenticating OpenStack's certificate.

Unless OpenStack's cert is trusted by a CA that is included in the requests library itself (note that the requests library does NOT consult the CA certs bundle of the OS), the user must specify insecure: True in order to connect, but this is not secure.

The reason is:

https://github.com/cloudify-cosmo/cloudify-openstack-plugin/blob/2.5.0/openstack_plugin_common/__init__.py#L588

This function will always end up passing either True or False to the verify kwarg of session.Session. We should also allow a string to be passed on. The string would be the path to the cert.

I'm trying to boot an openstack instance using an existing openstack volume resource. While running the blueprint I get an error similar to
KeyError: 'availability_zone'.

This is because of like az = boot_volume.runtime_properties[OPENSTACK_AZ_PROPERTY] at

Looking at runtime properties of volume, I see availability_zone isn't at top level. Rather it is stored under key external_resource.

For now in my openstack plugin source code, when I change the above line to
az = boot_volume.runtime_properties['external_resource'][OPENSTACK_AZ_PROPERTY] I can get this working.

Can you please suggest what changes needs to be done in blueprint to make sure I don't have to patch cloudify-openstack-plugin source.

My sample blueprint is:

  host:
    type:  cloudify.openstack.nodes.Server
    properties:
      openstack_config: *openstack_config
      resource_id:  XXX
      flavor: XXX
      management_network_name: XXX
    relationships:
      - type: cloudify.openstack.volume_attached_to_server
        target: volume

  volume:
    type: cloudify.openstack.nodes.Volume
    properties:
      openstack_config: *openstack_config
      boot: true
      resource_id: XXX
      use_external_resource: true

While running an uninstall workflow:

2015-02-05T19:02:51 CFY <testrhino> [rhino_db_host_61ad7] Deleting node
2015-02-05T19:02:51 CFY <testrhino> [rhino_db_host_61ad7] Deleting node
2015-02-05T19:02:52 CFY <testrhino> [rhino_db_host_61ad7.delete] Sending task 'nova_plugin.server.delete'
2015-02-05T19:02:52 CFY <testrhino> [rhino_db_host_61ad7.delete] Task started 'nova_plugin.server.delete'
2015-02-05T19:02:57 CFY <testrhino> [rhino_db_host_61ad7.delete] Task failed 'nova_plugin.server.delete' -> RecoverableError("ConnectionError: ('Connection aborted.', gaierror(-3, 'Temporary failure in name resolution'))",) [attempt 1]

The failure is ignored and not retried, and now that server has been leaked.

Summary

I want to boot a VM by using a boot volume, which is created by providing the openstack name for an image (not the openstack id).

Symptoms / repro

Here's the relevant node_templates:

  vm:
    type: cloudify.openstack.nodes.Server
    properties:
      flavor: { get_input: vm_flavor }
      management_network_name: {
        "name": { get_input: management_network }
      }
      openstack_config: {
        "username": { get_secret: openstack_user },
        "tenant_name": { get_secret: openstack_project },
        "region": "RegionOne",
        "auth_url": { get_secret: openstack_auth_url },
        "password": { get_secret: openstack_password }
      }
    relationships:
    - type: cloudify.openstack.server_connected_to_keypair
      target: keypair
    - type: cloudify.relationships.connected_to
      target: mvs_node_boot_volume


  boot_volume:
    type: cloudify.openstack.nodes.Volume
    properties:
      openstack_config: {
        "username": { get_secret: openstack_user },
        "tenant_name": { get_secret: openstack_project },
        "region": "RegionOne",
        "auth_url": { get_secret: openstack_auth_url },
        "password": { get_secret: openstack_password }
      }
      boot: true
    interfaces:
      cloudify.interfaces.lifecycle:
        create:
          inputs:
            args:
              size: 35
              imageRef: { get_input: vm_image_name }

The result of this is:

• The boot volume is created successfully, and is properly created using the image at vm_image_name.
• The image fails to be created, with error:

  Task failed | my_blueprint | my_deployment | install | cloudify.interfaces.lifecycle.create | vm | vm_gs6yit 
  | Task failed 'nova_plugin.server.create' -> Invalid input for field/attribute imageRef. Value: my-image-
  name. u'my-image-name' is not valid under any of the given schemas [status_code=400]
  

• The workflow fails.

I find it odd that the image was fine to be created with an image name (as opposed to a UUID), but the server (which doesn't need an image to be created!) cannot be created with an image name (as opposed to a UUID).

Environment

Cloudify manager version: v4.1.1
Openstack plugin version: v2.2.0

Suggestion

I see that there's some existing code which goes and gets the ID from a name-or-ID. This is used for the Server node-type. I imagine this could be used elsewhere to solve this issue.

Today we are creating absurdly long hostnames in servers because we have an absurd default naming policy:

    return "{0}_{1}_{2}".format(type_name, ctx.deployment.id, ctx.instance.id)

Change this to something else. Maybe do what we do in Azure, which is generate a UUID.

This code tests that the insecure variable is True or False. This is assuming that the user is providing a YAML value. However, if this value is set via an intrinsic function, the value will be evaluated as string, because all intrinsic function values are strings.

Basically, want to change something from fig 1 to something essentially the same as fig2 :

fig 1:

verify = not (cfg[AUTH_PARAM_INSECURE] is True)

fig 2:

cfg_insecure = cfg[AUTH_PARAM_INSECURE]
if isinstance(cfg_insecure, basestring) and cfg_insecure.Capitalize() == 'True':
    cfg[AUTH_PARAM_INSECURE] = True
verify = not (cfg[AUTH_PARAM_INSECURE] is True)

I have been searching in the code how to boot with config-drive on ( a necessity for some NfVs) and could not find anything.

Can we have a boolean parameter that allows using config drive when calling nova boot API ???

https://github.com/cloudify-cosmo/cloudify-openstack-plugin/blob/2.11.0/nova_plugin/server.py#L406

The start_retry_interval and private_key_path inputs have defaults in plugin.yaml, but don't have defaults in the function's signature.

If someone wants to subtype openstack.Server to apply a default to private_key_path, they should be able to do this:

start:
  implementation: openstack.nova_plugin.server.start
  inputs:
    private_key_path:
      default: { get_secret: my_key }

But they can't. That's because Cloudify's DSL replaces the entire start block, so only private_key_path would be sent to the operation. That would break the plugin because there are no defaults in the function's signature.

Hi!

Since the release of Openstack Havana the preferred version of Identity to use is v3. Some cloud providers have now migrated to only supporting v3 for token generation to get support for domains.

Currently the Cloudify Openstack plugin does not work with providers that require support for Identity v3.

As far as I can see, you're already using keystoneclient.v3.client, so it should basically amount to adding the correct arguments.

Would it be possible to add support for Identity v3 in this project as well?

inputs_cfg = {
'username': 'Admin',
'password': 'password',
'auth_url': 'http://192.168.84.88:5000/v2.0',
'region': 'RegionOne',
'tenant_name': 'demo'
}

import openstack_plugin_common as common

nova_client, neut_client, cind_client, keys_client = common.NovaClient().get(config=inputs_cfg),
common.NeutronClient().get(config=inputs_cfg),
common.CinderClient().get(config=inputs_cfg),
common.KeystoneClient().get(config=inputs_cfg),

output:

common.get_resource_by_name_or_id('default', 'security_group', nova_client)

{'name': 'default'} 95fb874d-6d23-49b6-9aac-81125584c234

output:

common.get_resource_by_name_or_id('default', 'security_group', neut_client)

Traceback (most recent call last):
File "/Users/Wayne/WorkSpace/mycodebase/mypython/openstack-plugin/system_tests/test_client.py", line 55, in
common.get_resource_by_name_or_id('default', 'security_group', neut_client)
File "/Users/Wayne/WorkSpace/mycodebase/mypython/openstack-plugin/openstack_plugin_common/init.py", line 197, in get_resource_by_name_or_id
*_search_param)
File "/Users/Wayne/WorkSpace/mycodebase/mypython/openstack-plugin/openstack_plugin_common/init.py", line 616, in cosmo_get_if_exists
return self._cosmo_get(obj_type_single, True, *_kw)
File "/Users/Wayne/WorkSpace/mycodebase/mypython/openstack-plugin/openstack_plugin_common/init.py", line 626, in _cosmo_get
obj_type_single, kw, len(ls)))
cloudify.exceptions.NonRecoverableError: Expected at most one object of type security_group with match {'name': 'default'} but there are 5

Process finished with exit code 1

Currently, we accept resource_id as a property. In a local profile, however there is no way to set resource_id at runtime. So we need to accept this value also as an input.

When OpenStack fails to create a new server, we get the following error in the logs:

NonRecoverableError: Unexpected server state ERROR:None

However, this doesn't actually give the operator any idea as to what the problem is. Instead, an operator must log onto their OpenStack rig and look at the server in question. In this case, the fault message in OpenStack was 'No valid host was found. There are not enough hosts available.'

It would be useful for Cloudify to expose the fault message so that operators don't always need to go digging in their OpenStack rig.

Logs are available by link https://cloudify-cs.atlassian.net/wiki/download/attachments/597688326/cloudify-manager-logs_20181113T102551_localhost.tar.gz?api=v2 . 'oliver23' is the deployment name.

This project insists on keystoneauth1<3.

This conflicts with the requirements of os-client-config and osc-lib (via python-neutronclient), and also openstacksdk (via osc-lib) - which variously require keystoneauth1>=3.7.0 or keystoneauth1>=3.8.0.

pipenv, at least, is unable to find a set of packages that satisfies everyone's requirements.

Can the requirement on keystoneauth1<3 be relaxed? (Does there exist some solution to all of these constraints that pipenv - and I! - have failed to find?)

Environment:

• Cloudify Manager 3.4.1
• OpenStack Plugin 2.7.4.

In the Openstack Plugin document [1], you say we can specify a key-value configuration in the "properties" for the Server as in the OpenStack Rest API [2]. However, it does not work. For example, I specify a "networks" the following blueprint:

host:
  type: cloudify.openstack.nodes.Server
  properties:
    server:
      networks:
        - uuid: 'f4b0b5c1-9e41-4df9-bb29-beb873572f53'
          fixed_ip: '10.0.0.3'

The REST request (to create the server) has no information about the network as I expected:

DEBUG: REQ: curl -g -i -X POST https://...com/v2/ab2c6f48c344402db70a8041847b0076/servers -H "User-Agent: python-nov
aclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}..." -d '{"server": {"name": "server_helloworld_
host_a9k0ha", "imageRef": "...", "flavorRef": "computev1-1", "user_data": "...", "max_count": 1, "min_count": 1, "metadata": {"blueprint_version": "v1"}}}'
2018-05-08 09:58:47,810 [ad9ec1c0-9ee0-4e2b-8349-75692bb6b4ea]

RESP BODY: {"badRequest":{"message":"Multiple possible networks found, use a Network ID to be more specific.","code":"400"}}

[1] https://docs.cloudify.co/4.3.0/developer/official_plugins/openstack/
[2] https://developer.openstack.org/api-ref/compute/#create-server

Currently, there's only one way (except for the deprecated provider context method) to tell a floating IP where to get the IP from: by specifying floating_network_id or floating_network_name.

Need to provide a way to connect a floating IP to the external cloudify.openstack.nodes.Network.

Hi everybody,

i want to link my security groups with remote_group_id

 - direction: ingress
    ethertype: IPv4
    port_range_min: 1
    port_range_max: 65535
    protocol: tcp
    remote_group_id: { get_attribute: [origin_security_group, id ] }

but it does not work just throws internal error

i tried to use remote_group_node: origin_security_group but it does not work too.

So how to use any group in security group rule?

Apologies for vague question.
I just started with Cloudify and stuck while creating a guest VM on my ESX server.

While trying to create a guest VM on VCenter getting following error:

(cloudify)vagrant@cloudify:~/cloudify$ cfy bootstrap -p ag_blueprint2.yaml executing bootstrap validation
mapping error: server_plugin.server has no attribute 'shutdown_guest' [node=test_vm, type=operations]

BLUEPRINT:

tosca_definitions_version: cloudify_dsl_1_1

imports:

• http://www.getcloudify.org/spec/cloudify/3.2/types.yaml
• http://getcloudify.org.s3.amazonaws.com/spec/vsphere-plugin/1.2/plugin.yaml

node_templates:
test_vm:
type: cloudify.vsphere.nodes.Server
properties:
networking:
dns_servers: ['10.10.10.10']
connected_networks:
- name: devops-vm-cfy
management: true
switch_distributed: false
use_dhcp: true
server:
name: cfy_deleteit
template: test_template1

  connection_config:
    url: 192.168.20.30
    username: [email protected]
    password: myPass
    datacenter_name: alok_esx
    resource_pool_name: /Resources
    auto_placement: false

http://docs.getcloudify.org/latest/plugins/openstack/ doesn't exist.

http://docs.getcloudify.org/3.4.1/plugins/openstack/ does exist, but I'm not sure if you want this link to be versioned.

If the nova_plugin.server.create operation is run twice on the same node for any reason (e.g. you rerun a partially-failed install workflow) then a new server is created, and the existing server is leaked - it is no longer known to Cloudify and will not be cleaned up.

Running the operation twice should either be idempotent, or fail. (idempotency would be nicer!)

Please add these runtime properties:

• ipv6_address Resolves to primary IPv6 Address (like ip_address resolves to private IPv4 address.
• ipv4_addresses list of IPv4 addresses when there are more than one IPv4 Address.
• ipv6_addresses list of IPv6 addresses when there are more than one IPv6 Address.

This commit contains the v3 support.

Please create a patch to backport these changes to the v1.2.1 base. It needs to apply on a 4.3 Manager that has been restored from 3.4.2 snapshot, that was restored from 3.2.1 Snapshot.

I declared a port range in my blueprint...

node_templates:
  # SECURITY
  node_security_group:
    type: cloudify.openstack.nodes.SecurityGroup
    properties:
      security_group:
        name: websphere_security_group
      rules:
        - remote_ip_prefix: 0.0.0.0/0
          port: 19000 19999

but the 'neutron_plugin.security_group.create' task failed with a 'Invalid value for port 19000 19999'... is there a specific way to set the port range for this plugin?

I know in openstack's nova client it is possible to declare a range.
http://docs.openstack.org/openstack-ops/content/security_groups.html

$ nova secgroup-add-rule <secgroup> <ip-proto> <from-port> <to-port> <cidr>

+-------------+-----------+---------+-----------+--------------+
| IP Protocol | From Port | To Port | IP Range  | Source Group |
+-------------+-----------+---------+-----------+--------------+
| tcp         | 80        | 80      | 0.0.0.0/0 |              |
+-------------+-----------+---------+-----------+--------------+

cfy command execution and error output...

(env)[root@ryansaltdev blueprints]# cfy executions start -w deploy -d websphere_test_deploy -l
Executing workflow 'deploy' on deployment 'websphere_test_deploy' at management server 10.155.255.181 [timeout=900 seconds]
2014-12-04T09:18:22 CFY <websphere_test_deploy> Starting 'deploy' workflow execution
2014-12-04T09:18:23 CFY <websphere_test_deploy> [node_security_group_5b414] Creating node
2014-12-04T09:18:23 CFY <websphere_test_deploy> [floatingip_22f57] Creating node
2014-12-04T09:18:23 CFY <websphere_test_deploy> [node_security_group_5b414.create] Sending task 'neutron_plugin.security_group.create'
2014-12-04T09:18:23 CFY <websphere_test_deploy> [floatingip_22f57.create] Sending task 'neutron_plugin.floatingip.create'
2014-12-04T09:18:23 CFY <websphere_test_deploy> [node_security_group_5b414.create] Task started 'neutron_plugin.security_group.create'
2014-12-04T09:18:23 CFY <websphere_test_deploy> [floatingip_22f57.create] Task started 'neutron_plugin.floatingip.create'
2014-12-04T09:18:23 LOG <websphere_test_deploy> [node_security_group_5b414.create] INFO: deleting security_group
2014-12-04T09:18:23 CFY <websphere_test_deploy> [floatingip_22f57.create] Task succeeded 'neutron_plugin.floatingip.create'
2014-12-04T09:18:23 LOG <websphere_test_deploy> [node_security_group_5b414.create] ERROR: Exception raised on operation [neutron_plugin.security_group.create] invocation
Traceback (most recent call last):
  File "/home/cloudify/cloudify.websphere_test_deploy/env/local/lib/python2.7/site-packages/cloudify/decorators.py", line 114, in wrapper
    result = func(*args, **kwargs)
  File "/home/cloudify/cloudify.websphere_test_deploy/env/local/lib/python2.7/site-packages/openstack_plugin_common/__init__.py", line 458, in wrapper
    _re_raise(e, recoverable=False, status_code=e.status_code)
  File "/home/cloudify/cloudify.websphere_test_deploy/env/local/lib/python2.7/site-packages/openstack_plugin_common/__init__.py", line 455, in wrapper
    return f(*args, **kw)
  File "/home/cloudify/cloudify.websphere_test_deploy/env/local/lib/python2.7/site-packages/neutron_plugin/security_group.py", line 118, in create
    {'security_group_rule': sgr})
  File "/home/cloudify/cloudify.websphere_test_deploy/env/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 98, in with_params
    ret = self.function(instance, *args, **kwargs)
  File "/home/cloudify/cloudify.websphere_test_deploy/env/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 496, in create_security_group_rule
    return self.post(self.security_group_rules_path, body=body)
  File "/home/cloudify/cloudify.websphere_test_deploy/env/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1325, in post
    headers=headers, params=params)
  File "/home/cloudify/cloudify.websphere_test_deploy/env/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1251, in do_request
    self._handle_fault_response(status_code, replybody)
  File "/home/cloudify/cloudify.websphere_test_deploy/env/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1216, in _handle_fault_response
    exception_handler_v20(status_code, des_error_body)
  File "/home/cloudify/cloudify.websphere_test_deploy/env/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 66, in exception_handler_v20
    status_code=status_code)
NonRecoverableError: Invalid value for port 19000 19999 [status_code=400]
2014-12-04T09:18:23 CFY <websphere_test_deploy> [node_security_group_5b414.create] Task failed 'neutron_plugin.security_group.create' -> NonRecoverableError('Invalid value for port 19000 19999 [status_code=400]',) [attempt 1]
2014-12-04T09:18:23 CFY <websphere_test_deploy> 'deploy' workflow execution failed: RuntimeError: Workflow failed: Task failed 'neutron_plugin.security_group.create' -> NonRecoverableError('Invalid value for port 19000 19999 [status_code=400]',)
Execution of workflow 'deploy' for deployment 'websphere_test_deploy' failed. [error=Traceback (most recent call last):
  File "/home/cloudify/cloudify.websphere_test_deploy_workflows/env/local/lib/python2.7/site-packages/cloudify/decorators.py", line 200, in child_wrapper
    ctx, func, args, kwargs)
  File "/home/cloudify/cloudify.websphere_test_deploy_workflows/env/local/lib/python2.7/site-packages/cloudify/decorators.py", line 303, in _execute_workflow_function
    result = func(*args, **kwargs)
  File "/home/cloudify/cloudify.websphere_test_deploy_workflows/env/local/lib/python2.7/site-packages/td_workflow_plugin/workflows.py", line 464, in install
    InstallationTasksGraphFinisher
  File "/home/cloudify/cloudify.websphere_test_deploy_workflows/env/local/lib/python2.7/site-packages/td_workflow_plugin/workflows.py", line 176, in _install_node_instances
    return graph.execute()
  File "/home/cloudify/cloudify.websphere_test_deploy_workflows/env/local/lib/python2.7/site-packages/cloudify/workflows/tasks_graph.py", line 120, in execute
    self._handle_terminated_task(task)
  File "/home/cloudify/cloudify.websphere_test_deploy_workflows/env/local/lib/python2.7/site-packages/cloudify/workflows/tasks_graph.py", line 188, in _handle_terminated_task
    task.error))
RuntimeError: Workflow failed: Task failed 'neutron_plugin.security_group.create' -> NonRecoverableError('Invalid value for port 19000 19999 [status_code=400]',)
]

When trying to deploy a blueprint locally, I was receiving errors stating that the parameters I had supplied (in the blueprint) were not among the list of allowed parameter combinations.
It then listed the exact combination of parameters I had supplied among the acceptable options.

The actual cause was running with correctly supplied parameters in the blueprint, as well as having a sources openstackrc file with other parameters.

It would be nice if there were a way to prevent merging, e.g. an openstack_config flag in the blueprint that could force only specific configuration sources to be used (e.g. merging would occur as normal with the default value of:
openstack_config:
config_sources: [env_vars, file, blueprint]
but if only one (or two) sources were used then they would be the only ones used, e.g.
config_sources: [env_vars, blueprint]
or
config_sources: [blueprint]
)

Alternatively, for simplicity, just a way to specify only one config source would be helpful.

Some time ago 'list' operations has been introduced for openstack resources node types.
In general these operations are working fine, but for standard "openstack_config" (containing information about tenant e.g. with paramters: auth_url, password, tenant_name, username)
some methods are returning list of items used by given tenant only and some methods are returning list of items used by whole openstack system (sum for all tenants).

Methods which by default are returning list of items used by single tenant:

• openstack.nova_plugin.server.list_servers
• openstack.nova_plugin.keypair.list_keypairs
• openstack.cinder_plugin.volume.list_volumes
• openstack.glance_plugin.image.list_images
• openstack.nova_plugin.host_aggregate.list_host_aggregates
• openstack.nova_plugin.server_group.list_servergroups

•   openstack.nova_plugin.flavor.list_flavors
  

Methods which by default are returning list of items used by all tenants (even if tenant is set in openstack_config):

• openstack.neutron_plugin.subnet.list_subnets
• openstack.neutron_plugin.port.list_ports
• openstack.neutron_plugin.network.list_networks
• openstack.neutron_plugin.floatingip.list_floatingips
• openstack.neutron_plugin.security_group.list_security_groups
• openstack.neutron_plugin.router.list_routers

Reason of this bahaviour is fact that 'list' methods in neutron_client are returing data for all tenants.
Proper behaviour can be enforced by workaround in blueprint:

        list:
          implementation: openstack.neutron_plugin.subnet.list_subnets
          inputs:
            args:
		tenant_id: '<tenant_id>' # tenant name doesn't work !

But issue is that current inconsistent behaviour is misleading for end user of plugin.
All 'list' methods should return list of tenant resources when tenant ID or name is present in openstack_config.

See also:
https://bugs.launchpad.net/python-neutronclient/+bug/1250766

When running HEAL workflow on instance which have floating IP's, plugin is detaching them at "port detach" but not reassigning them on "attach". As a result, instance after HEAL workflow is loosing FIP

this is my blueprint yaml:
-------------------begin-------------------------
inputs:
..image:
....description: >
......Image to be used when launching agent VM's
..flavor:
....description: >
......Flavor of the agent VM's
..agent_user:
....description: >
......User for connecting to agent VM's
..userdata:
node_types:
..nodecellar.nodes.MonitoredServer:
....derived_from: cloudify.openstack.nodes.Server
....properties:
......cloudify_agent:
........default:
..........user: { get_input: agent_user }
......server:
........default:
..........image: { get_input: image }
..........flavor: { get_input: flavor }
..........userdata:
............type: http
............url: { get_input: userdata }
-----------------end---------------------
this is my input file:
----------------begin-------------------
{
...."image": "71d939f6-2690-4c13-8324-942950392d2f",
...."flavor": "3",
...."agent_user": "root",
...."userdata": "http://10.107.248.232:8181/script/config-agent.sh"
}
----------------end--------------------
after the VM created,"config-agent.sh" doesn't work on it.and there are no error message in the cli console.I use cloudify 3.1.

Customers would like the option to see the actual OpenStack requests/responses being generated, for troubleshooting purposes.

For example:

  cloudify.openstack.nodes.Server:
    derived_from: cloudify.nodes.Compute
    properties:
      server:
        default: {}
        description: >
          The keypair object as described in OpenStack compute API (create server method):
          https://developer.openstack.org/api-ref/compute/#servers-servers

Should be changed to:

  cloudify.openstack.nodes.Server:
    derived_from: cloudify.nodes.Compute
    properties:
      resource_config:
        default: {}
        description: >
          The keypair object as described in OpenStack compute API (create server method):
          https://developer.openstack.org/api-ref/compute/#servers-servers

Update all node-type naming convention from cloudify.openstack.nodes.Type to cloudify.nodes.openstack.Type.

For example, cloudify.openstack.nodes.Server should now be cloudify.nodes.openstack.Server and cloudify.openstack.port_connected_to_security_group should be cloudify.relationships.openstack.port_connected_to_security_group.

Hello,

I am trying to get cloudify to talk to openstack. I have tried a number of ways but I always get an error saying the "request requires authentication".

I have provided credentials every time I have tried.

Here is my most recent attempt at a blueprint. All of the secrets in the inputs have been defined.
I get the same error when I go through the GUI for creating deployments and also when I hard coded the values into the blueprint.

tosca_definitions_version: cloudify_dsl_1_3

imports:
  - http://www.getcloudify.org/spec/cloudify/5.0.0/types.yaml
  - plugin:cloudify-openstack-plugin

inputs:
  auth_url:
    type: string
    default: { get_secret: keystone_url }
  username:
    type: string
    default: { get_secret: keystone_username }
  password:
    type: string
    default: { get_secret: keystone_password }
  project_name:
    type: string
    default: { get_secret: keystone_tenant_name }
  region_name:
    type: string
    default: { get_secret: keystone_region }
  external_network_id:
    type: string
    default: nfis
    description: A network that provisions floating IPs.
  user_domain_name:
    type: string
    default: { get_secret: user_domain_name }
  project_domain_name:
    type: string
    default: { get_secret: project_domain_name }


node_templates:
  example-network:
    type: cloudify.nodes.openstack.Network
    properties:
      client_config:
        auth_url: { get_input: auth_url }
        username: { get_input: username }
        password: { get_input: password }
        insecure: true
        project_name: { get_input: project_name }
        region_name: { get_input:  region_name }
        project_domain_name: { get_input:  project_domain_name }
        user_domain_name: { get_input:  user_domain_name }
      resource_config:
        name: example-network

Here is the stack trace:

2019-12-12 18:59:13.797  CFY <deployment1> [example-network_x5c4gv.create] Task failed 'openstack_plugin.resources.network.network.create' -> The request you have made requires authentication. (HTTP 401) (Request-ID: req-54f88489-9f9d-4e9e-87f1-ff4661678497)
Traceback (most recent call last):
  File "/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify/dispatch.py", line 813, in main
    payload = handler.handle()
  File "/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify/dispatch.py", line 461, in handle
    result = self._run_operation_func(ctx, kwargs)
  File "/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify/dispatch.py", line 519, in _run_operation_func
    return self.func(*self.args, **kwargs)
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/openstack_plugin/decorators.py", line 147, in wrapper
    func(**kwargs_config)
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/openstack_plugin/decorators.py", line 104, in wrapper_inner
    func(**kwargs)
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/openstack_plugin/resources/network/network.py", line 55, in create
    created_resource = openstack_resource.create()
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/openstack_sdk/resources/networks.py", line 69, in create
    network = self.connection.network.create_network(**self.config)
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/openstack/service_description.py", line 87, in __get__
    proxy = self._make_proxy(instance)
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/openstack/service_description.py", line 235, in _make_proxy
    **version_kwargs
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/openstack/config/cloud_region.py", line 710, in get_session_client
    service_type, constructor)
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/openstack/config/cloud_region.py", line 656, in _get_hardcoded_endpoint
    endpoint = adapter.get_endpoint()
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 282, in get_endpoint
    return self.session.get_endpoint(auth or self.auth, **kwargs)
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/keystoneauth1/session.py", line 1225, in get_endpoint
    return auth.get_endpoint(self, **kwargs)
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 380, in get_endpoint
    allow_version_hack=allow_version_hack, **kwargs)
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 271, in get_endpoint_data
    service_catalog = self.get_access(session).service_catalog
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 134, in get_access
    self.auth_ref = self.get_auth_ref(session)
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref
    return self._plugin.get_auth_ref(session, **kwargs)
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/keystoneauth1/identity/v3/base.py", line 184, in get_auth_ref
    authenticated=False, log=False, **rkwargs)
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/keystoneauth1/session.py", line 1131, in post
    return self.request(url, 'POST', **kwargs)
  File "/opt/mgmtworker/env/plugins/default_tenant/cloudify-openstack-plugin-3.2.9/lib/python2.7/site-packages/keystoneauth1/session.py", line 968, in request
    raise exceptions.from_response(resp, method, url)
Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-54f88489-9f9d-4e9e-87f1-ff4661678497)

I have been working on this for a while and would greatly appreciate any help, i'm sure its something simple that I am missing.

I would be happy to provide any more information you require.

Not possible to add static routes to existing router

I tried to create a server with the following node_template:

  firewall_host:
    type: cloudify.openstack.nodes.Server
    properties:
      openstack_config: *openstack_config
      agent_config:
        key: { get_property: [ keypair, private_key_path ] }
      management_network_name: private
      image:  { get_input: image }
      flavor: { get_input: flavor }
      cloudify_agent:
          user: { get_input: agent_user }
    relationships:
      - type: cloudify.openstack.server_connected_to_keypair
        target: keypair
      - type: cloudify.openstack.server_connected_to_port
        target: fw_port_test

An when specifying the management_network_name by its name, it raises the following error:
Expected exactly one object of type network with match {'name': u'private'} but there are 7
If I try to use the network ID, I got the following error:
Expected exactly one object of type network with match {'name': u'1b94ecd1-abd7-44e4-bcbf-70b9a371f5c0'} but there are 0
I also tried to create the network (or port) node_template and create a relationship to the server but then, it raises another error:
Workflow failed: Task failed 'nova_plugin.server.create' -> Nova server with NICs requires 'management_network_name' in properties or id from provider context, which was not supplied
It comes from the fact that the user that is used to create the objects in OpenStack has admin rights (needed to create a port on a shared network) and can then see all the networks. In those networks, there are several that have the name private.
Is there any way to create a server and have its management network correctly configured?
I see in the errors and in the documentation that I could define it in the provider context. Unfortunately, in the documentation, it is also said that: The exact details of the structure of the OpenStack Provider Context are not documented because this feature is being deprecated and will be replaced with a more advanced one.
Thanks for the help!

Keypair creation is failing with below error:

Task failed 'nova_plugin.keypair.create' -> 'str' object has no attribute 'id'

This is due to incorrect order of parameter passing to the method set_openstack_runtime_properties. (See below line...)
https://github.com/cloudify-cosmo/cloudify-openstack-plugin/blob/master/nova_plugin/keypair.py#L72

set_openstack_runtime_properties method definition:
https://github.com/cloudify-cosmo/cloudify-openstack-plugin/blob/master/openstack_plugin_common/__init__.py#L550

node template definition:
node_templates:
keypair:
type: cloudify.openstack.nodes.KeyPair
properties:
private_key_path: { get_input: private_key_path }
use_external_resource: true
create_if_missing: true
resource_id: { get_input: keypair_name }
openstack_config: *openstack_config

Complete execution log:
[root@cloudify-manager43 f5-usecase-blueprints]# cfy execution start install -d keypair_create_test
Executing workflow install on deployment keypair_create_test [timeout=900 seconds]
2018-03-20 17:19:24.787 CFY <keypair_create_test> Starting 'install' workflow execution
2018-03-20 17:19:25.911 CFY <keypair_create_test> [keypair_at4bsp] Creating node
2018-03-20 17:19:25.911 CFY <keypair_create_test> [keypair_at4bsp.create] Sending task 'nova_plugin.keypair.create'
2018-03-20 17:19:25.911 CFY <keypair_create_test> [keypair_at4bsp.create] Task started 'nova_plugin.keypair.create'
2018-03-20 17:19:28.341 CFY <keypair_create_test> [keypair_at4bsp.create] Task failed 'nova_plugin.keypair.create' -> 'str' object has no attribute 'id'
2018-03-20 17:19:43.453 CFY <keypair_create_test> [keypair_at4bsp.create] Sending task 'nova_plugin.keypair.create' [retry 1/60]
2018-03-20 17:19:43.920 CFY <keypair_create_test> [keypair_at4bsp.create] Task started 'nova_plugin.keypair.create' [retry 1/60]
2018-03-20 17:19:46.053 LOG <keypair_create_test> [keypair_at4bsp.create] INFO: Using external resource keypair: keypair_for_monitoring1
2018-03-20 17:19:45.990 CFY <keypair_create_test> [keypair_at4bsp.create] Task failed 'nova_plugin.keypair.create' -> Failed to use external keypair (node keypair): the public key keypair_for_monitoring1 is available on Openstack, but the private key could not be found at /tmp/keypair_for_monitoring.pem [retry 1/60]
Execution ended, waiting 3 seconds for additional log messages
2018-03-20 17:19:46.924 CFY <keypair_create_test> 'install' workflow execution failed: RuntimeError: Workflow failed: Task failed 'nova_plugin.keypair.create' -> Failed to use external keypair (node keypair): the public key keypair_for_monitoring1 is available on Openstack, but the private key could not be found at /tmp/keypair_for_monitoring.pem
Execution of workflow install for deployment keypair_create_test failed. [error=Traceback (most recent call last):
File "/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify/dispatch.py", line 519, in _remote_workflow_child_thread
workflow_result = self._execute_workflow_function()
File "/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify/dispatch.py", line 550, in _execute_workflow_function
result = self.func(*self.args, **self.kwargs)
File "/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify/plugins/workflows.py", line 27, in install
node_instances=set(ctx.node_instances))
File "/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify/plugins/lifecycle.py", line 28, in install_node_instances
processor.install()
File "/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify/plugins/lifecycle.py", line 93, in install
graph_finisher_func=self._finish_install)
File "/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify/plugins/lifecycle.py", line 114, in _process_node_instances
self.graph.execute()
File "/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify/workflows/tasks_graph.py", line 133, in execute
self._handle_terminated_task(task)
File "/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify/workflows/tasks_graph.py", line 207, in _handle_terminated_task
raise RuntimeError(message)
RuntimeError: Workflow failed: Task failed 'nova_plugin.keypair.create' -> Failed to use external keypair (node keypair): the public key keypair_for_monitoring1 is available on Openstack, but the private key could not be found at /tmp/keypair_for_monitoring.pem
]

• Run 'cfy events list -e 78f16ce2-108e-4d78-8bbe-ef7da7a451f5' to retrieve the execution's events/logs

Currently, there appears to be no way to automate keystone domains, only projects. This would make for a great enhancement as V3 is in use almost everywhere now and more and more groups are using multi-tenancy features.

We want a cloudify.openstack.nodes.Stack type that can be used to deploy Openstack Stacks using Heat templates.

Based on these API operations:

• https://developer.openstack.org/api-ref/orchestration/v1/index.html#stacks

Based on this client:

• https://github.com/openstack/python-heatclient.

The latest version of cmd2 requires Python3.4 or above. this cause installation failure.

Collecting cmd2>=0.6.7 (from cliff!=1.16.0,!=1.17.0,>=1.15.0->python-neutronclient==6.0.0->cloudify-openstack-plugin==2.9.0)
Using cached https://files.pythonhosted.org/packages/07/4a/f8c16175fa806b9c81f60688411b09a84a23decf774f2cdd5657e43dfe9b/cmd2-0.9.0.post1.tar.gz
cmd2 requires Python '>=3.4' but the running Python is 2.7.14