Giter Club home page Giter Club logo

python-3parclient's Introduction

Latest Version Downloads

HPE Alletra 9000 and HPE Primera and HPE 3PAR REST Client

This is a Client library that can talk to the HPE Alletra 9000 and Primera and 3PAR Storage array. The HPE Alletra 9000 and Primera and 3PAR storage array has a REST web service interface and a command line interface. This client library implements a simple interface for talking with either interface, as needed. The python Requests library is used to communicate with the REST interface. The python paramiko library is used to communicate with the command line interface over an SSH connection.

This is the new location for the rebranded HP 3PAR Rest Client and will be where all future releases are made. It was previously located on PyPi at: https://pypi.python.org/pypi/hp3parclient

The GitHub repository for the old HP 3PAR Rest Client is located at: https://github.com/hpe-storage/python-3parclient/tree/3.x

The HP 3PAR Rest Client (hp3parclient) is now considered deprecated.

Requirements

This branch requires 3.1.3 version MU1 or later of the HPE 3PAR firmware. This branch requires 4.3.1 version of the HPE Primera firmware. This branch requires 9.3.0 version of the HPE Alletra 9000 firmware. File Persona capabilities require HPE 3PAR firmware 3.2.1 Build 46 or later.

Capabilities

  • Create Volume
  • Delete Volume
  • Get all Volumes
  • Get a Volume
  • Modify a Volume
  • Copy a Volume
  • Create a Volume Snapshot
  • Create CPG
  • Delete CPG
  • Get all CPGs
  • Get a CPG
  • Get a CPG's Available Space
  • Create a VLUN
  • Delete a VLUN
  • Get all VLUNs
  • Get a VLUN
  • Create a Host
  • Delete a Host
  • Get all Hosts
  • Get a Host
  • Get VLUNs for a Host
  • Find a Host
  • Find a Host Set for a Host
  • Get all Host Sets
  • Get a Host Set
  • Create a Host Set
  • Delete a Host Set
  • Modify a Host Set
  • Get all Ports
  • Get iSCSI Ports
  • Get FC Ports
  • Get IP Ports
  • Set Volume Metadata
  • Get Volume Metadata
  • Get All Volume Metadata
  • Find Volume Metadata
  • Remove Volume Metadata
  • Create a Volume Set
  • Delete a Volume Set
  • Modify a Volume Set
  • Get a Volume Set
  • Get all Volume Sets
  • Find one Volume Set containing a specified Volume
  • Find all Volume Sets containing a specified Volume
  • Create a QOS Rule
  • Modify a QOS Rule
  • Delete a QOS Rule
  • Set a QOS Rule
  • Query a QOS Rule
  • Query all QOS Rules
  • Get a Task
  • Get all Tasks
  • Get a Patch
  • Get all Patches
  • Get WSAPI Version
  • Get WSAPI Configuration Info
  • Get Storage System Info
  • Get Overall System Capacity
  • Stop Online Physical Copy
  • Query Online Physical Copy Status
  • Stop Offline Physical Copy
  • Resync Physical Copy
  • Query Remote Copy Info
  • Query a Remote Copy Group
  • Query all Remote Copy Groups
  • Create a Remote Copy Group
  • Delete a Remote Copy Group
  • Modify a Remote Copy Group
  • Add a Volume to a Remote Copy Group
  • Remove a Volume from a Remote Copy Group
  • Start Remote Copy on a Remote Copy Group
  • Stop Remote Copy on a Remote Copy Group
  • Synchronize a Remote Copy Group
  • Recover a Remote Copy Group from a Disaster
  • Enable/Disable Config Mirroring on a Remote Copy Target
  • Get Remote Copy Group Volumes
  • Get Remote Copy Group Volume
  • Admit Remote Copy Link
  • Dismiss Remote Copy Link
  • Start Remote Copy
  • Remote Copy Service Exists Check
  • Get Remote Copy Link
  • Remote Copy Link Exists Check
  • Admit Remote Copy Target
  • Dismiss Remote Copy Target
  • Target In Remote Copy Group Exists Check
  • Remote Copy Group Status Check
  • Remote Copy Group Status Started Check
  • Remote Copy Group Status Stopped Check
  • Create Schedule
  • Delete Schedule
  • Get Schedule
  • Modify Schedule
  • Suspend Schedule
  • Resume Schedule
  • Get Schedule Status
  • Promote Virtual Copy
  • Get a Flash Cache
  • Create a Flash Cache
  • Delete a Flash Cache

File Persona Capabilities

  • Get File Services Info
  • Create a File Provisioning Group
  • Grow a File Provisioning Group
  • Get File Provisioning Group Info
  • Modify a File Provisioning Group
  • Remove a File Provisioning Group
  • Create a Virtual File Server
  • Get Virtual File Server Info
  • Modify a Virtual File Server
  • Remove a Virtual File Server
  • Assign an IP Address to a Virtual File Server
  • Get the Network Config of a Virtual File Server
  • Modify the Network Config of a Virtual File Server
  • Remove the Network Config of a Virtual File Server
  • Create a File Services User Group
  • Modify a File Services User Group
  • Remove a File Services User Group
  • Create a File Services User
  • Modify a File Services User
  • Remove a File Services User
  • Create a File Store
  • Get File Store Info
  • Modify a File Store
  • Remove a File Store
  • Create a File Share
  • Get File Share Info
  • Modify a File Share
  • Remove a File Share
  • Create a File Store Snapshot
  • Get File Store Snapshot Info
  • Remove a File Store Snapshot
  • Reclaim Space from Deleted File Store Snapshots
  • Get File Store Snapshot Reclamation Info
  • Stop or Pause a File Store Snapshot Reclamation Task
  • Set File Services Quotas
  • Get Files Services Quota Info

Installation

To install from source:

$ sudo pip install .

To install from http://pypi.org:

$ sudo pip install python-3parclient

Unit Tests

To run all unit tests:

$ tox -e py27

To run a specific test:

$ tox -e py27 -- test/file.py:class_name.test_method_name

To run all unit tests with code coverage:

$ tox -e cover

The output of the coverage tests will be placed into the coverage dir.

Folders

  • docs -- contains the documentation.
  • hpe3parclient -- the actual client.py library
  • test -- unit tests
  • samples -- some sample uses

Documentation

To build the documentation:

$ tox -e docs

To view the built documentation point your browser to:

docs/html/index.html

Running Simulators

The unit tests should automatically start/stop the simulators. To start them manually use the following commands. To stop them, use 'kill'. Starting them manually before running unit tests also allows you to watch the debug output.

  • WSAPI:

    $ python test/HPE3ParMockServer_flask.py -port 5001 -user <USERNAME> -password <PASSWORD> -debug
    
  • SSH:

    $ python test/HPE3ParMockServer_ssh.py [port]
    

Building wheel dist

This client now supports building via the new python WHEELS standard. Take a look at http://pythonwheels.com

  • building:

    $ python setup.py bdist_wheel
    
  • building and uploading:

    $ python setup.py sdist bdist_wheel upload
    

python-3parclient's People

Contributors

ansarars avatar aorourkehp avatar bhagyashree-sarawate avatar dockerciuser avatar farhan7500 avatar garthmbooth avatar hemna avatar jbranen avatar jyotsnalothe avatar kumartin avatar ladaniv avatar leeantho avatar mohammadsayyed avatar rahulvermahp avatar ravichandrann avatar sneharai4 avatar traghavendra avatar viraj-hp avatar wdurairaj avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

python-3parclient's Issues

TypeError: 'NoneType' object is not iterable

Hello, when we try to create a new volume in openstack, everything is alright, but when we try to create an instance with cinder from image, the mesage below appears.

   |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin

_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2017-12-19T18:53:55Z,del
eted=False,deleted_at=None,display_description='',display_name='',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='hostgroup@hpe3par#REDHAT',id=df3842f6-
e422-40e7-b392-7aa80177aac5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='653e53dca79a45beb6f6d9a1a9afe78d',provider_auth=None,provid
er_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status='disabled',scheduled_at=2017-12-19T18:53:55Z,size=1
,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2017-12-19T18:53:55Z,user_id='deb50ca683d84c6ab17f1683ecee6d7d',volume_attachment=,volum
e_type=,volume_type_id=None), 'context': }, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zo ne='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2017-12-19T18:53:55Z,deleted=False,deleted_at=None,display_description='',dis play_name='',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='hostgroup@hpe3par#REDHAT',id=df3842f6-e422-40e7-b392-7aa80177aac5,launched_at=None,metadata ={},migration_status=None,multiattach=False,previous_status=None,project_id='653e53dca79a45beb6f6d9a1a9afe78d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=N one,replication_driver_data=None,replication_extended_status=None,replication_status='disabled',scheduled_at=2017-12-19T18:53:55Z,size=1,snapshot_id=None,snapshots=,source_volid=None,sta
tus='creating',terminated_at=None,updated_at=2017-12-19T18:53:55Z,user_id='deb50ca683d84c6ab17f1683ecee6d7d',volume_attachment=,volume_type=,volume_type_id=None)}
|__Flow 'volume_create_manager'
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager Traceback (most recent call last):
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager result = task.execute(**arguments)
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 873, in execute
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager **volume_spec)
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 831, in _create_from_image
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager image_service)
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 285, in wrapped
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager return f(*a, **k)
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 740, in _create_from_image_cache_or_download
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager image_service, context, image_id) as tmp_image:
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager File "/usr/lib64/python2.7/contextlib.py", line 17, in enter
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager return self.gen.next()
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/image/image_utils.py", line 594, in fetch
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager fetch_verify_image(context, image_service, image_id, tmp)
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/image/image_utils.py", line 244, in fetch_verify_image
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager None, None)
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/image/image_utils.py", line 190, in fetch
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager image_service.download(context, image_id, image_file)
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 340, in download
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager for chunk in image_chunks:
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager TypeError: 'NoneType' object is not iterable
2017-12-19 21:53:56.754 18953 ERROR cinder.volume.manager
2017-12-19 21:53:56.760 18953 WARNING cinder.volume.manager [req-e0599bff-687a-4671-8901-f05dcede5b1f deb50ca683d84c6ab17f1683ecee6d7d 653e53dca79a45beb6f6d9a1a9afe78d - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4d1dfb0e-811e-418a-82b5-1ee58bda0084) transitioned into state 'REVERTED' from state 'REVERTING'
2017-12-19 21:53:56.762 18953 WARNING cinder.volume.manager [req-e0599bff-687a-4671-8901-f05dcede5b1f deb50ca683d84c6ab17f1683ecee6d7d 653e53dca79a45beb6f6d9a1a9afe78d - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (636a8db6-d320-43f0-bdde-10f02d097bac) transitioned into state 'REVERTED' from state 'REVERTING'

ssh session never renewed if lost

in ssh.py, _run_ssh.py will fail if the ssh session has been closed, which can happen on a heavily loaded environment. In our patched version of the 3parclient, I moved my retries to 2 and reinitialize ssh if the session is dead, like this: (see next entrry)

Driver initialize connection failed error: "'hostname'"

The following exception occurs when attempting to attach a volume to a 3PAR that has VLUNs with no hostnames. This could happen if volume or volume set were exported to direct ports. The client's getHostVluns needs to take this into account.

2015-05-12 13:22:12.859 ERROR cinder.volume.manager [req-ae836f09-b688-4803-b710-bbc2b46cef20 None None](u'Driver initialize connection failed %28error: %%28err%29s%29.', {'err': u"'hostname'"})
2015-05-12 13:22:12.861 ERROR oslo_messaging.rpc.dispatcher [req-ae836f09-b688-4803-b710-bbc2b46cef20 None None] Exception during message handling: Bad or unexpected response from the storage volume backend API: (u'Driver initialize connection failed (error: %(err)s).', {'err': u"'hostname'"})
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatc
her.py", line 142, in _dispatch_and_reply
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher executor_callback))
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher executor_callback)
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, *_new_args)
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher return f(_args, *_kwargs)
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher return f(_args, *_kwargs)
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher return f(_args, **kwargs)
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/cinder/cinder/volume/manager.py", line 1106, in initialize_connection
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher raise exception.VolumeBackendAPIException(data=err_msg)
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: (u'Driver initialize connection failed (error: %(err)s).', {'err': u"'hostname'"})
2015-05-12 13:22:12.861 TRACE oslo_messaging.rpc.dispatcher
2015-05-12 13:22:12.876 ERROR oslo_messaging._drivers.common [req-ae836f09-b688-4803-b710-bbc2b46cef20 None None] Returning exception Bad or unexpected response from the storage volume backend API: (u'Driver initialize connection failed (error: %(err)s).', {'err': u"'hostname'"}) to caller

createVLUN() API failes with LUN==0, while CLI allows it.

I'm currently working with a legacy code that expects LUN==0, but I cannot make 3PAR storage create such VLUN mapping from WSAPI.

I believe this is not a 3PAR box limitation, as I CAN make one from CLI:

cli% createhost testhost
Setting default host persona 2 (Generic-ALUA)
cli% createvv default testvv 1G
cli% createvlun testvv 0 testhost
Warning: Host testhost has no active paths.  Template may be created but no active VLUNs will be created.
cli% showvlun -v testvv
Active VLUNs
no vluns listed

VLUN Templates
Lun VVName HostName -Host_WWN/iSCSI_Name- Port Type
  0 testvv testhost ----------------       --- host
---------------------------------------------------
  1 total

As you can see in the result, VLUN mapping can be created with Lun==0 from CLI.

OTOH, here's what happens from WSAPI:

In [3]: cl.createVolume("testvv", "default", 1024)
In [4]: cl.createHost("testhost")
In [9]: cl.createVLUN("testvv", 0, "testhost")
---------------------------------------------------------------------------
HTTPBadRequest                            Traceback (most recent call last)
/var/tmp/gio-boot/gio/storage/hp3par.py in <module>()
----> 1 cl.createVLUN("testvv", 0, "testhost")

/opt/rh/python33/root/usr/lib/python3.3/site-packages/hp3parclient-3.2.2-py3.3.egg/hp3parclient/client.py in createVLUN(self, volumeName, lun, hostname, portPos, noVcn, overrideLowerPriority, auto)
   2012             info['lun'] = 0
   2013
-> 2014         headers, body = self.http.post('/vluns', body=info)
   2015         if headers:
   2016             location = headers['location'].replace('/api/v1/vluns/', '')

/opt/rh/python33/root/usr/lib/python3.3/site-packages/hp3parclient-3.2.2-py3.3.egg/hp3parclient/http.py in post(self, url, **kwargs)
    287                          will be an object
    288         """
--> 289         return self._cs_request(url, 'POST', **kwargs)
    290
    291     def put(self, url, **kwargs):

/opt/rh/python33/root/usr/lib/python3.3/site-packages/hp3parclient-3.2.2-py3.3.egg/hp3parclient/http.py in _cs_request(self, url, method, **kwargs)
    232         try:
    233             resp, body = self._time_request(self.api_url + url, method,
--> 234                                             **kwargs)
    235             return resp, body
    236         except exceptions.HTTPUnauthorized as ex:

/opt/rh/python33/root/usr/lib/python3.3/site-packages/hp3parclient-3.2.2-py3.3.egg/hp3parclient/http.py in _time_request(self, url, method, **kwargs)
    208     def _time_request(self, url, method, **kwargs):
    209         start_time = time.time()
--> 210         resp, body = self.request(url, method, **kwargs)
    211         self.times.append(("%s %s" % (method, url),
    212                            start_time, time.time()))

/opt/rh/python33/root/usr/lib/python3.3/site-packages/hp3parclient-3.2.2-py3.3.egg/hp3parclient/http.py in request(self, *args, **kwargs)
    202
    203         if resp.status >= 400:
--> 204             raise exceptions.from_response(resp, body)
    205
    206         return resp, body

HTTPBadRequest: Bad request (HTTP 400) 40 - invalid input: some or all required parameters are missing - lun

So WSAPI thinks "lun" is missing when lun==0. And when I change it to 1, it passes:

In [10]: cl.createVLUN("testvv", 1, "testhost")
Out[10]: 'testvv,1,testhost'

Is there any limitation in creating VLUN with LUN==0, or is this a WSAPI issue?

Backward compatibility on session generation broken on 3PAR WSAPI v3

Curl command fails on v3

cho-suse:~ # curl -k -H "Content-type: application/json" -H "Accept: application/json" -X POST -d'{"user": "3paradm", "password":"3pardata"}' https://15.129.163.204:8080/api/v3/credentials
{"message":"json: cannot unmarshal string into Go struct field LoginRequest.password of type []string","resourceUri":""}

Against v1, api running on the array which supports v3/v1 , the same payload works

cho-suse:~ # curl -k -H "Content-type: application/json" -H "Accept: application/json" -X POST -d'{"user": "3paradm", "password":"3pardata"}' https://15.129.163.204:8080/api/v1/credentials
{"key":"0-5b3a9c6b96eb33e13268de7a971141df-e77e465b"}

What we have found further is , the password text has to be send in a JSON string array format

cho-suse:~ # curl -k -H "Content-type: application/json" -H "Accept: application/json" -X POST -d'{"user": "3paradm", "password":["3","p","a","r","d","a","t","a"]}' https://15.129.163.204:8080/api/v3/credentials
{"key":"e1a6dc3c812fde7c34e9e015d60cc76062ef017177b9a5fa"}cho-suse:~ #

cho-suse:~ # curl -k -H "Content-type: application/json" -H "Accept: application/json" -X POST -d'{"user": "3paradm", "password":["3pardata"]}' https://15.129.163.204:8080/api/v3/credentials
{"key":"527b3234531c01cb5d47fbcc511f046c5baed15c170168b9"}

WSAPI Sessionkey missing on one of the DELETE request during detach volume

Detach volume issue a series of rest commands, one of the delete request is missing WSAPI SessionKey which results in 403 invalid session key from WSAPI and leaves the session dangling.

REQ: curl -i https://10.10.20.241:8080/api/v1/volumes/osv-poaJoIFQT-.WbT5abgbIcg/objectKeyValues/HPQ-CS-instance_uuid DELETE -H "Accept: application/json" -H "User-Agent: python-3parclient"
�[00m �[00;33mfrom (pid=3843) _http_log_req /usr/local/lib/python2.7/dist-packages/hp3parclient/http.py:159�[00m
RESP:{'connection': 'close',
'content-type': 'application/json',
'date': 'Thu, 04 Sep 2014 17:24:03 GMT',
'server': 'hp3par-wsapi',
'status': '403'}

2014-09-04 10:24:11.173 �[00;32mDEBUG hp3parclient.http [�[01;36mreq-f35c04d9-85c1-4043-bd32-0017f4e57005 �[00;36md58b2f359802452884da20ddf2d7ecf9 b67d33ea3945422c96f16299f87ca5f0�[00;32m] �[01;35m�[00;32mRESP:{'connection': 'close',
'content-type': 'application/json',
'date': 'Thu, 04 Sep 2014 17:24:03 GMT',
'server': 'hp3par-wsapi',
'status': '403'}
�[00m �[00;33mfrom (pid=3843) _http_log_resp /usr/local/lib/python2.7/dist-packages/hp3parclient/http.py:166�[00m
RESP BODY:{"code":6,"desc":"invalid session key"}

A good request example
2014-09-04 10:24:11.649 �[00;32mDEBUG hp3parclient.http [�[01;36mreq-f35c04d9-85c1-4043-bd32-0017f4e57005 �[00;36md58b2f359802452884da20ddf2d7ecf9 b67d33ea3945422c96f16299f87ca5f0�[00;32m] �[01;35m�[00;32m
REQ: curl -i https://10.10.20.241:8080/api/v1/volumes/osv-poaJoIFQT-.WbT5abgbIcg/objectKeyValues/HPQ-CS-instance_uuid DELETE -H "X-Hp3Par-Wsapi-Sessionkey: 1bc-046640f4d011d1cffd1853340d9f2e33-34a00856" -H "Accept: application/json" -H "User-Agent: python-3parclient"
�[00m �[00;33mfrom (pid=3843) _http_log_req /usr/local/lib/python2.7/dist-packages/hp3parclient/http.py:159�[00m
RESP:{'cache-control': 'no-cache',
'connection': 'close',
'date': 'Thu, 04 Sep 2014 17:24:04 GMT',
'pragma': 'no-cache',
'server': 'hp3par-wsapi',
'status': '200'}

2014-09-04 10:24:11.680 �[00;32mDEBUG hp3parclient.http [�[01;36mreq-f35c04d9-85c1-4043-bd32-0017f4e57005 �[00;36md58b2f359802452884da20ddf2d7ecf9 b67d33ea3945422c96f16299f87ca5f0�[00;32m] �[01;35m�[00;32mRESP:{'cache-control': 'no-cache',
'connection': 'close',
'date': 'Thu, 04 Sep 2014 17:24:04 GMT',
'pragma': 'no-cache',
'server': 'hp3par-wsapi',
'status': '200'}
�[00m �[00;33mfrom (pid=3843) _http_log_resp /usr/local/lib/python2.7/dist-packages/hp3parclient/http.py:166�[00m
RESP BODY:

pop from empty list error in docker volume plugin initialization

When the FC Driver is initialized on an 3PAR array where the iscsi ports are offline or not there, we get the following error on HPE 3PAR Docker Volume plugin error.

MountVolume.SetUp failed for volume "sc1-90ad0972-9ce0-11e9-85b5-566f4fb90000" : mount command failed, status: Failure, reason: HPE Docker Volume Plugin Mount Failed: ('Initialize Connection Failed: connection info retrieval failed, error is: ', 'pop from empty list')

You can't just compare version strings like this

current_wsapi_version = '{}.{}.{}'.format(api_version.get('major'),
api_version.get('minor'),
api_version.get('revision'))
if current_wsapi_version >= self.WSAPI_MIN_VERSION_COMPRESSION_SUPPORT:

The WSAPI version on primera is now 1.10.0. Which since the 'minimum' version is 1.5.0, means that the code thinks the minimum version is not met and breaks any attempt at a creation of a compressed lun. Please fix ASAP

upgrade of python-3parclient from 4.2.5 to 4.2.7 is failing

On Ubuntu 16.04

lsb_release -a

No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.2 LTS
Release: 16.04
Codename: xenial

pip install -U python-3parclient

Collecting python-3parclient
Downloading https://files.pythonhosted.org/packages/9e/03/a236d0efec29db8e8765681d81b1f6aad5c040490cf1425c2236620f0fae/python-3parclient-4.2.7.tar.gz (109kB)
100% |████████████████████████████████| 112kB 122kB/s
Requirement already up-to-date: paramiko in /usr/local/lib/python2.7/dist-packages (from python-3parclient)
Collecting eventlet (from python-3parclient)
Downloading https://files.pythonhosted.org/packages/75/5c/09d277692cb7d44258062a46b553067662b530266776e5cb0306e7a83770/eventlet-0.23.0-py2.py3-none-any.whl (409kB)
100% |████████████████████████████████| 419kB 169kB/s
Requirement already up-to-date: requests in /usr/local/lib/python2.7/dist-packages (from python-3parclient)
Requirement already up-to-date: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from paramiko->python-3parclient)
Requirement already up-to-date: bcrypt>=3.1.3 in /usr/local/lib/python2.7/dist-packages (from paramiko->python-3parclient)
Requirement already up-to-date: cryptography>=1.5 in /usr/local/lib/python2.7/dist-packages (from paramiko->python-3parclient)
Requirement already up-to-date: pynacl>=1.0.1 in /usr/local/lib/python2.7/dist-packages (from paramiko->python-3parclient)
Requirement already up-to-date: enum34; python_version < "3.4" in /usr/local/lib/python2.7/dist-packages (from eventlet->python-3parclient)
Requirement already up-to-date: greenlet>=0.3 in /usr/local/lib/python2.7/dist-packages (from eventlet->python-3parclient)
Requirement already up-to-date: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests->python-3parclient)
Requirement already up-to-date: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests->python-3parclient)
Requirement already up-to-date: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests->python-3parclient)
Requirement already up-to-date: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests->python-3parclient)
Requirement already up-to-date: six>=1.4.1 in /usr/local/lib/python2.7/dist-packages (from bcrypt>=3.1.3->paramiko->python-3parclient)
Requirement already up-to-date: cffi>=1.1 in /usr/local/lib/python2.7/dist-packages (from bcrypt>=3.1.3->paramiko->python-3parclient)
Requirement already up-to-date: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.5->paramiko->python-3parclient)
Requirement already up-to-date: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.5->paramiko->python-3parclient)
Requirement already up-to-date: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.1->bcrypt>=3.1.3->paramiko->python-3parclient)
Building wheels for collected packages: python-3parclient
Running setup.py bdist_wheel for python-3parclient ... done
Stored in directory: /root/.cache/pip/wheels/12/ac/9c/48f7783850b3934dcd5483a252fbaa83f3f63674e8da0c92e6
Successfully built python-3parclient
Installing collected packages: eventlet, python-3parclient
Found existing installation: eventlet 0.22.1
Uninstalling eventlet-0.22.1:
Successfully uninstalled eventlet-0.22.1
Found existing installation: python-3parclient 4.2.5
Uninstalling python-3parclient-4.2.5:
Successfully uninstalled python-3parclient-4.2.5
Successfully installed eventlet python-3parclient
Traceback (most recent call last):
File "/usr/bin/pip", line 11, in
sys.exit(main())
File "/usr/lib/python2.7/dist-packages/pip/init.py", line 217, in main
return command.main(cmd_args)
File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 246, in main
pip_version_check(session)
File "/usr/lib/python2.7/dist-packages/pip/utils/outdated.py", line 102, in pip_version_check
installed_version = get_installed_version("pip")
File "/usr/lib/python2.7/dist-packages/pip/utils/init.py", line 866, in get_installed_version
working_set = pkg_resources.WorkingSet()
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 619, in init
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 675, in add_entry
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 1942, in find_eggs_in_zip
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 1463, in has_metadata
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 1824, in _has
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 1704, in zipinfo
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 1644, in load
OSError: [Errno 2] No such file or directory: '/usr/local/lib/python2.7/dist-packages/python_3parclient-4.2.5-py2.7.egg'

Proper Error Handling Missing at client initialization

https://github.com/hpe-storage/python-3parclient/blob/master/hpe3parclient/client.py#L201-L204
try: api_version = self.getWsApiVersion() except exceptions as ex: ex_desc = ex.get_description()

error at "self.getWsApiVersion()" results in

11:23:49 E   requests.exceptions.ProxyError: HTTPSConnectionPool(host='192.168.67.2', port=8080): Max retries exceeded with url: /api (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 503 Service Unavailable',)))
11:23:49 
11:23:49 During handling of the above exception, another exception occurred:
11:23:49 hpe_3par_api_manage_test.py:48: in <module>
11:23:49     hpe_3par_cli = HPE3ParClient(HPE3PAR_API_URL, True, False, None, True)
11:23:49 /usr/lib/python3.6/site-packages/hpe3parclient/client.py:203: in __init__
11:23:49     except exceptions as ex:
11:23:49 E   TypeError: catching classes that do not inherit from BaseException is not allowed
11:23:49 ------------------------------- Captured stderr --------------------------------
11:23:49 
11:23:49 REQ: curl -i https://192.168.67.2:8080/api -X GET -H "User-Agent: python-3parclient" -H "Accept: application/json"
11:23:49 

(HTTP 500) 1 - internal server error when trying to login

Hello,
I'm trying to login to a 3PAR but I got the HTTP 5000 error:

REST_URL='https://10.250.7.6:8080/api/v1'
REST_URL
'https://10.250.7.6:8080/api/v1'
cl = client.HPE3ParClient(REST_URL)

REQ: curl -i https://10.250.7.6:8080/api -X GET -H "User-Agent: python-3parclient" -H "Accept: application/json"

/home/gabrield/venv/venv3.6_wsapi/lib64/python3.6/site-packages/urllib3/connectionpool.py:847: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
RESP:{'Date': 'Tue, 19 Mar 2019 06:55:31 GMT'
'Server': 'hp3par-wsapi'
'Cache-Control': 'no-cache'
'Pragma': 'no-cache'
'Content-Type': 'application/json'
'Connection': 'close'
'status': '200'
'content-location': 'https://10.250.7.6:8080/api'}

RESP BODY:{"major":1,"minor":5,"revision":3,"build":30202750}

cl.login(username, password)

REQ: curl -i https://10.250.7.6:8080/api/v1/credentials -X POST -H "User-Agent: python-3parclient" -H "Accept: application/json" -H "Content-Type: application/json"

REQ BODY: {"user": "gabrield", "password": "********"}

/home/gabrield/venv/venv3.6_wsapi/lib64/python3.6/site-packages/urllib3/connectionpool.py:847: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
RESP:{'Date': 'Tue, 19 Mar 2019 06:55:52 GMT'
'Server': 'hp3par-wsapi'
'Content-Type': 'application/json'
'Connection': 'close'
'status': '500'
'content-location': 'https://10.250.7.6:8080/api/v1/credentials'}

RESP BODY:{"code":1,"desc":"internal server error"}

Traceback (most recent call last):
File "", line 1, in
File "/home/gabrield/venv/python-3parclient/hpe3parclient/client.py", line 294, in login
self.http.authenticate(username, password, optional)
File "/home/gabrield/venv/python-3parclient/hpe3parclient/http.py", line 124, in authenticate
resp, body = self.post('/credentials', body=info)
File "/home/gabrield/venv/python-3parclient/hpe3parclient/http.py", line 376, in post
return self._cs_request(url, 'POST', **kwargs)
File "/home/gabrield/venv/python-3parclient/hpe3parclient/http.py", line 321, in _cs_request
**kwargs)
File "/home/gabrield/venv/python-3parclient/hpe3parclient/http.py", line 297, in _time_request
resp, body = self.request(url, method, **kwargs)
File "/home/gabrield/venv/python-3parclient/hpe3parclient/http.py", line 262, in request
raise exceptions.from_response(resp, body)
hpe3parclient.exceptions.HTTPInternalServerError: Internal Server Error (HTTP 500) 1 - internal server error

The WSAPI server is enabled:
showwsapi -d
-----------------WSAPI Server Configuration-----------------
service State : Enabled
server State : Active
HTTP State : Disabled
HTTP Port : 8008
HTTPS State : Enabled
HTTPS Port : 8080
Number of sessions In Use : 0
System Resource Usage : 72
Version : 1.5.3
Session Timeout : 15 Minutes
Policy : tls_strict
API URL : https://10.250.7.6:8080/api/v1

Can you please tell me what am I doing wrong?

Client does not capture 3Par errors for Manila

The client is ignoring errors returned by a 3Par array when requested manila actions cannot be performed. It's necessary to set "hp3par_debug = True" in manila.conf to see the errors that 3Par is returning.

Example log from manual test of manila snapshot-create:

2015-05-06 10:00:04.332 DEBUG hp3parclient.ssh [req-2ad10d4b-b1a6-4918-bf1d-1c41d7bb6124 b8788c43632b4d2d8dd8f88bb66ef9cb 5efc26263da6404ca9b0d53a3f894b0b] SSH CMD = ['createfsnap', '-f', '-fpg', 'smsfpg', 'smsvfs', 'osf-nfs-5efc26263da6404ca9b0d53a3f894b0b', u'osf-233d8c18-827d-42db-9704-a4654b411afa'] from (pid=45676) run /usr/local/lib/python2.7/dist-packages/hp3parclient/ssh.py:241
Running cmd (SSH): createfsnap -f -fpg smsfpg smsvfs osf-nfs-5efc26263da6404ca9b0d53a3f894b0b osf-233d8c18-827d-42db-9704-a4654b411afa
2015-05-06 10:00:04.333 DEBUG hp3parclient.ssh [req-2ad10d4b-b1a6-4918-bf1d-1c41d7bb6124 b8788c43632b4d2d8dd8f88bb66ef9cb 5efc26263da6404ca9b0d53a3f894b0b] Running cmd (SSH): createfsnap -f -fpg smsfpg smsvfs osf-nfs-5efc26263da6404ca9b0d53a3f894b0b osf-233d8c18-827d-42db-9704-a4654b411afa from (pid=45676) _ssh_execute /usr/local/lib/python2.7/dist-packages/hp3parclient/ssh.py:262
2015-05-06 10:00:04.333 DEBUG paramiko.transport [req-2ad10d4b-b1a6-4918-bf1d-1c41d7bb6124 b8788c43632b4d2d8dd8f88bb66ef9cb 5efc26263da6404ca9b0d53a3f894b0b] [chan 1161] Max packet in: 32768 bytes from (pid=45676) _log /usr/local/lib/python2.7/dist-packages/paramiko/channel.py:1072
2015-05-06 10:00:04.371 DEBUG paramiko.transport [-] [chan 1161] Max packet out: 32768 bytes from (pid=45676) _log /usr/local/lib/python2.7/dist-packages/paramiko/channel.py:1072
2015-05-06 10:00:04.371 DEBUG paramiko.transport [-] Secsh channel 1161 opened. from (pid=45676) _log /usr/local/lib/python2.7/dist-packages/paramiko/transport.py:1430
2015-05-06 10:00:04.401 DEBUG paramiko.transport [-] [chan 1161] Sesch channel 1161 request ok from (pid=45676) _log /usr/local/lib/python2.7/dist-packages/paramiko/channel.py:1072
2015-05-06 10:00:04.403 DEBUG paramiko.transport [-] [chan 1161] Sesch channel 1161 request ok from (pid=45676) _log /usr/local/lib/python2.7/dist-packages/paramiko/channel.py:1072
2015-05-06 10:00:05.224 DEBUG paramiko.transport [-] [chan 1161] EOF received (1161) from (pid=45676) _log /usr/local/lib/python2.7/dist-packages/paramiko/channel.py:1072
2015-05-06 10:00:05.225 DEBUG paramiko.transport [-] [chan 1161] EOF sent (1161) from (pid=45676) _log /usr/local/lib/python2.7/dist-packages/paramiko/channel.py:1072
Result was 0
2015-05-06 10:00:05.226 DEBUG hp3parclient.ssh [req-2ad10d4b-b1a6-4918-bf1d-1c41d7bb6124 b8788c43632b4d2d8dd8f88bb66ef9cb 5efc26263da6404ca9b0d53a3f894b0b] Result was 0 from (pid=45676) _ssh_execute /usr/local/lib/python2.7/dist-packages/hp3parclient/ssh.py:289
OUT = task {endTime 2015-05-06T16:57:36.273Z location /provisioningGroup/smsfpg/virtualServer/smsvfs/fileStore/osf-nfs-5efc26263da6404ca9b0d53a3f894b0b/snapshots/FailedToCreateSnapShot response {No such file or directory error code -2} startTime 2015-05-06T16:57:36.272Z state EXCEPTION}
2015-05-06 10:00:05.227 DEBUG hp3parclient.ssh [req-2ad10d4b-b1a6-4918-bf1d-1c41d7bb6124 b8788c43632b4d2d8dd8f88bb66ef9cb 5efc26263da6404ca9b0d53a3f894b0b] OUT = task {endTime 2015-05-06T16:57:36.273Z location /provisioningGroup/smsfpg/virtualServer/smsvfs/fileStore/osf-nfs-5efc26263da6404ca9b0d53a3f894b0b/snapshots/FailedToCreateSnapShot response {No such f from (pid=45676) run /usr/local/lib/python2.7/dist-packages/hp3parclient/ssh.py:250
2015-05-06 10:00:05.228 DEBUG manila.share.drivers.hp.hp_3par_mediator [req-2ad10d4b-b1a6-4918-bf1d-1c41d7bb6124 b8788c43632b4d2d8dd8f88bb66ef9cb 5efc26263da6404ca9b0d53a3f894b0b] createfsnap result=['task {endTime 2015-05-06T16:57:36.273Z location /provisioningGroup/smsfpg/virtualServer/smsvfs/fileStore/osf-nfs-5efc26263da6404ca9b0d53a3f894b0b/snapshots/FailedToCreateSnapShot response {No such file or directory error code -2} startTime 2015-05-06T16:57:36.272Z state EXCEPTION}\r'] from (pid=45676) create_snapshot /opt/stack/manila/manila/share/drivers/hp/hp_3par_mediator.py:415

Replace httplib2 with requests lib

Customer ran into issue with default SSL lib using in httplib2, it's a known defect for along time and doesn't look to be fixed soon, the customer had to change httplib2 to use ssl.PROTOCOL_TLSv1.

We should use requests lib instead of httplib2, some of openstack has already moved to this version.
The customer has requests==2.3.0 in there envrionment

WSAPI API (createHostSet) fails with a certain character in a comment

I'm trying to embed JSON-encoded string (for operational reasons) in a comment field, and API call is failing with a HTTP 500 error.

Here is a CLI example which passes without an error:

cli% createhostset -comment {["some", "json", "data"]} fooset
cli% showhostset -d fooset
Id Name   Members Comment
46 fooset --      ["some", "json", "data"]

At this point, I can successfully read above hostset info over WSAPI:

In [28]: cl.getHostSets()
Out[28]:
{'members': [{'comment': '["some", "json", "data"]',
   'id': 47,
   'name': 'fooset'}],
 'total': 1}

However, when I try to do the same thing (= create a hostset over WSAPI), it fails.
It seems either "[" or "]" is causing this, but any printable chracters up to 255 characters should be supported as stated in the documentation (CLI reference, createhostset command).

In [32]: data = '["some", "json", "data"]'
In [33]: cl.createHostSet("fooset", comment=data)
---------------------------------------------------------------------------
HTTPInternalServerError                   Traceback (most recent call last)
/d/src/gio-boot/gio/storage/hp3par.py in <module>()
----> 1 cl.createHostSet("fooset", comment=data)

/opt/rh/python33/root/usr/lib/python3.3/site-packages/hp3parclient-3.2.2-py3.3.egg/hp3parclient/client.py in createHostSet(self, name, domain, comment, setmembers)
   1266             info = self._mergeDict(info, members)
   1267
-> 1268         response, body = self.http.post('/hostsets', body=info)
   1269         if response is not None and 'location' in response:
   1270             host_set_id = response['location'].rsplit(

/opt/rh/python33/root/usr/lib/python3.3/site-packages/hp3parclient-3.2.2-py3.3.egg/hp3parclient/http.py in post(self, url, **kwargs)
    287                          will be an object
    288         """
--> 289         return self._cs_request(url, 'POST', **kwargs)
    290
    291     def put(self, url, **kwargs):

/opt/rh/python33/root/usr/lib/python3.3/site-packages/hp3parclient-3.2.2-py3.3.egg/hp3parclient/http.py in _cs_request(self, url, method, **kwargs)
    232         try:
    233             resp, body = self._time_request(self.api_url + url, method,
--> 234                                             **kwargs)
    235             return resp, body
    236         except exceptions.HTTPUnauthorized as ex:

/opt/rh/python33/root/usr/lib/python3.3/site-packages/hp3parclient-3.2.2-py3.3.egg/hp3parclient/http.py in _time_request(self, url, method, **kwargs)
    208     def _time_request(self, url, method, **kwargs):
    209         start_time = time.time()
--> 210         resp, body = self.request(url, method, **kwargs)
    211         self.times.append(("%s %s" % (method, url),
    212                            start_time, time.time()))

/opt/rh/python33/root/usr/lib/python3.3/site-packages/hp3parclient-3.2.2-py3.3.egg/hp3parclient/http.py in request(self, *args, **kwargs)
    202
    203         if resp.status >= 400:
--> 204             raise exceptions.from_response(resp, body)
    205
    206         return resp, body

HTTPInternalServerError: Internal Server Error (HTTP 500) 1 - internal server error

As the error is HTTP 500, could this be an issue with the WSAPI server running on a 3PAR box?

setQOSRule doesn't work if max_bw is not defined

There's an bad indentation in setQOSRule at line :

            cmd.append('vvset:' + set_name)

If only max_io is defined, the "cmd" generated looks like
setqos -io 12345
and is incorrectly interpreted by 3par array.

def setQOSRule(self, set_name, max_io=None, max_bw=None):
        cmd = ['setqos']
        if max_io is not None:
            cmd.extend(['-io', '%s' % max_io])
        if max_bw is not None:
            cmd.extend(['-bw', '%sM' % max_bw])
            cmd.append('vvset:' + set_name)
        result = self._run(cmd)

paramiko.ssh_exception.SSHException: Error connecting via ssh: 'HPE3PARSSHClient' object has no attribute 'san_privatekey'

StoreServ

  • added public key of host via addsshkey

Script at host
root@hLinux:/3parpython# cat 3parprivatekeyssh.py
import logging
import os
import paramiko
from random import randint
import time
import re
from eventlet import greenthread
from hpe3parclient import client, exceptions, http, ssh

sshclient = ssh.HPE3PARSSHClient(ip="x.x.x.x", login="user", password=None,port=22,conn_timeout=None,privatekey='/home/xxxxx/.ssh/id_rsa')
sshclient.open()
output = sshclient._ssh_execute('showvv')
print "Output s",output
print "ssh session started"
sshclient.close()
print "ssh session closed"

Error :
root@hLinux:/3parpython# python 3parprivatekeyssh.py
No handlers could be found for logger "hpe3parclient.ssh"
Traceback (most recent call last):
File "hpe3par.py", line 38, in
sshclient.open()
File "/usr/local/lib/python2.7/dist-packages/hpe3parclient/ssh.py", line 140, in open
raise paramiko.SSHException(msg)
paramiko.ssh_exception.SSHException: Error connecting via ssh: 'HPE3PARSSHClient' object has no attribute 'san_privatekey'

Snapshot promotion

Hi

i might be dumb, but I cant find a way to promote a snapshot using this API. I can do it manually in the web interface of the 3par, but I miss the automation here.

Is it a missing function in the API, or did I overlook something?

Greetings
Joerg

exception received when creating vlun

When creating vLUN using autoLUN option an error is thrown:

hpe3parclient.exceptions.HTTPConflict: Conflict (HTTP 409) 99 - invalid operation: LUN number and persona capability conflict

This is caused in specific cases:

  1. host persona VMware 11 is used
  2. LUN IDs 0-255 are used.

Root cause is introduction of PE LUN for host persona VMware 11 (LUN ID). Since HPE didn't provide a fix yet for 3PAR OS, we would like to as for enhancement of 3PARclient code in following way for autoLUN option.

1step.
autoLun = True
lun=0
maxAutoLun=0

if receive error "Conflict (HTTP 409) 99 - invalid operation: LUN number and persona capability conflict" ignore error and continue 2step

2step.
autoLun = True
lun=257
maxAutoLun=0

thanks,
Miro

Peer persistence?

We are looking to use openstack cinder for managing our 3PAR storage server(s).

While setting up a PoC environment I appear to be unable to setup peer-persistence. We are using SYNC mode for our volumes, and I am able to successfully create remoteCopyGroups and sync replicated volumes through cinder (and thus the 3parclient). I do not see any code in neither Cinder driver nor the 3parclient to support adding a new RCG to the list of groups enabled for Peer Persistence.

Is this functionality available in any way? Should we be solving this differently (i.e. some kind of default configuration on the 3PAR to have RCG's by default peer-persistent), or was this particular feature just not implemented? We can implement the feature in cinder if necessary, but I would prefer to be able to at least use the upstream 3parclient.

Thank you in advance for any feedback/pointers you can provide.

findVolumeSet broken

Hi

the code of findVolumeSet hardcodes to only return the first result of volumesets to which a volume is exported.
3Par allows to export to many vvsets and there are situations where this must be (say, VM live migration with VM storage on 3Par volumes, clustered filesystems, ...).

There should be a way to have all vvsets listed in which a volume exists.

bye
Joerg

RFE: Add vlan tagged iscsi ports support

When a user vlan tags iscsi ports on 3par systems this driver can't find the port information required to connect initiators and fails out. I've attached a file with the ouput of the showport and showport -iscsivlan commands from the 3par system I have under test and the cinder debug logs and error:

hp3par_client_error_iscsi_port_vlans.txt

I'm willing to write the update for this feature. I wanted to have some discussion around the way the owners would prefer to see it done. I see two options:

  1. Add a self.ssh.run('showport -iscsivlan') to the getPorts() method, massage the output into an array of port objects similar to what it gets from the api call, and append it to the members key of that object. Or,

  2. Add a hp3par_iscsi_vlans_enabled=true flag to cinder_override.conf and create tie-ins to getPorts() to only return the output of one command or the other.

I think I'd prefer 1 since it would let people use vlans without any extra work being done by the end user. I'm open to additional options as well.

Had to disable WS API check to run it

Hello,

I had to disable the code below to use it because of Invalid 3PAR WS API. My systemVersion is 3.1.3.202

181 # try:
182 api_version = self.getWsApiVersion()
183 # except Exception:
184 # pass
185 # msg = ('Either, the 3PAR WS is not running or the'
186 # ' version of the WS is invalid.')
187 # raise exceptions.UnsupportedVersion(msg)
188
189 # Note the build contains major, minor, maintenance and build
190 # e.g. 30102422 is 3 01 02 422
191 # therefore all we need to compare is the build
192 # if (api_version is None or
193 # api_version['build'] < self.HP3PAR_WS_MIN_BUILD_VERSION):
194 # raise exceptions.UnsupportedVersion(
195 # 'Invalid 3PAR WS API, requires version, %s' %
196 # self.HP3PAR_WS_MIN_BUILD_VERSION_DESC)

getVLUN() method is misleading

getVLUN(volumeName) method receives a volume name as its unique parameter and is described in the API as

    Get information about a VLUN.

    :param volumeName: The volume name of the VLUN to find
    :returns: VLUN

This description seems to imply that a volume can only have one associated VLUN which frequently is not the case (a volume can be exported to one or several hosts or one or several hosts sets).
Indeed, in its implementation, this method gets the list of VLUNs matching the volumeName but only returns the first one (on purpose, as stated by the comment "Return the first VLUN found for the volume"):

# Return the first VLUN found for the volume.
for vlun in body.get('members', []):
return vlun
else:
vluns = self.getVLUNs()
if vluns:
for vlun in vluns['members']:
if vlun['volumeName'] == volumeName:
return vlun

Ideally, this method should request enough parameters to univocally reference the VLUN to get (similar feature in WSAPI requires volume name and host name in versions up to up to 1.2, and volume name, lun ID and a combination of hostname and/or port in recent versions). As maybe a signature change will cause more harm that good, at least its behavior should be stated clearly in the doc to avoid confusion.

Related note: as this library does not expose WSAPI's filtering support, the only current way to get an actual list of the VLUNs of a volume is to use getVLUNs() to get ALL the volumes and filter it client-side.

Exception blocking log message output.

There seems to be a problem recently added to the hp3parclient package which is preventing an error message from being placed in the log. Instead of the intended message, we see a message about an exception being thrown.

Details follow:


This is the function that works. It is located in /usr/local/lib/python2.7/dist-packages/hp3parclient/client.py at around line 175. Git branch says this is Juno.

def init(self, api_url):
self.api_url = api_url
self.http = http.HTTPJSONRESTClient(self.api_url)
api_version = None
self.ssh = None
try:
api_version = self.getWsApiVersion()
except Exception:
msg = ('Either, the 3PAR WS is not running or the'
' version of the WS is invalid.')
raise exceptions.UnsupportedVersion(msg)

    # Note the build contains major, minor, maintenance and build
    # e.g. 30102422 is 3 01 02 422
    # therefore all we need to compare is the build
    if (api_version is None or
       api_version['build'] < self.HP3PAR_WS_MIN_BUILD_VERSION):
        raise exceptions.UnsupportedVersion(
            'Invalid 3PAR WS API, requires version, %s' %
            self.HP3PAR_WS_MIN_BUILD_VERSION_DESC)

This is the function that doesn’t, located in the same place. Git branch says this is master.

def __init__(self, api_url, debug=False):
    self.api_url = api_url
    self.http = http.HTTPJSONRESTClient(self.api_url)
    api_version = None
    self.ssh = None

    self.debug_rest(debug)

    try:
        api_version = self.getWsApiVersion()
    except Exception as ex:
        ex_desc = ex.get_description()
        if (ex_desc and "Unable to find the server at" in ex_desc or
                "Only absolute URIs are allowed" in ex_desc):
            raise exceptions.HTTPBadRequest(ex_desc)
        else:
            msg = ('Error: \'%s\' - Error communicating with the 3PAR WS. '
                   'Check proxy settings. If error persists, either the '
                   '3PAR WS is not running or the version of the WS is '
                   'not supported.') % ex_desc
            raise exceptions.UnsupportedVersion(msg)

    # Note the build contains major, minor, maintenance and build
    # e.g. 30102422 is 3 01 02 422
    # therefore all we need to compare is the build
    if (api_version is None or
       api_version['build'] < self.HP3PAR_WS_MIN_BUILD_VERSION):
        raise exceptions.UnsupportedVersion(
            'Invalid 3PAR WS API, requires version, %s' %
            self.HP3PAR_WS_MIN_BUILD_VERSION_DESC)

The line causing the error is the if statement within the except statement.

if (ex_desc and "Unable to find the server at" in ex_desc or
"Only absolute URIs are allowed" in ex_desc)

The problem seems to be that in order of operations, and is evaluated before or, so the second time python checks for a string in ex_desc, an exception is thrown because ex_desc is NoneType.

"getVolume()" method is returning "None"

Hi,

I am using this library for communicating to the array over web-service/REST APIs.
in a case, when I try to get a volume for knowing its information, i'm not able to get the volume's details.
The data responded to me is "None"

Then I checked at array side & checked using CLI , the volume is present on the Array.

Can someone please let me know what could go wrong? this is really serious problem for me.

Attributed Error when print Exception

Hi there,

When I use client.deleteVolume(name=volume_name) I encountered below failure.
I tried to find the root cause and found that it was because I want to store the exception into a variable in the catch block and the exception's str throw an exception.

code snippet:

try:
	cl.deleteVolume(name=vv_name)
except Exception as e:
	message = "failed to remove virtual volume, name=%s, error=%s. " % (vv_name, e)

Traceback (most recent call last):
.......
File "/opt/virtualenvs/3par/lib/python2.7/site-packages/hpe3parclient/exceptions.py",
line 105, in str
if self.http_status:AttributeError: 'Timeout' object has no attribute 'http_status'

Def "createVolume" optional support for tdvv vv's not documented for python-3parclient (4.2.2)

InForm OS versions ------------------
Release version 3.2.2.390 (MU2)
Patches: P11,P14,P15,P19

Def createVolume Call eg. -----------------------
cl.createVolume("vol44v", "SSD_r1", 10000, {'id':93,'snapCPG':'SSD_r5','tdvv':True,'usrSpcAllocWarningPct':70})

Result - (cl.getVolume("vol44v") --------Was Created Successfully-----------
u'baseId': 93,
u'capacityEfficiency': {u'compaction': 20.0, u'deduplication': 0.0},
u'copyType': 1,
u'id': 93,
u'name': u'vol44v',
u'provisioningType': 6,

Directory Location : /usr/local/lib/python2.7/dist-packages/hpe3parclient ----------
def createVolume(self, name, cpgName, sizeMiB, optional=None):
"""Create a new volume. Removed left comments for brevity

        optional = {
         """removed rest informartion for brevity
         'tpvv': True,                # True: Create TPVV
         """removed rest informartion for brevity
        }

INT_SERV_ERR 1 500 returned when using getVolumes() API call

Hi,
we have a python script that pulls volume capacity information from the 3Par arrays to be stored in InfluxDB. The script worked fine with the old hp3parclient on our 3par 7400s and 8450 until the 3par 8450 was upgraded to 3.3.1.460 (MU3)+P50 .
If I ran the script now, it returns values up until a point where it give this response:
Unable to get volume capacity
Internal Server Error (HTTP 500) 1 - internal server error

the actual call that fails (values in <> not shown for privacyreasons):
REQ: curl -i http://:8008/api/v1/volumes -X GET -H "Accept: application/json" -H "X-Hp3Par-Wsapi-Sessionkey: " -H "User-Agent: python-3parclient"

Anyone encountered this? Anything changed in that OS version that changes API behavior?

Allow client override to disable "InsecureRequestWarning" log messages

With the release of the new client, the following error message is constantly logged:

/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised.
See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)

Currently there is now way to suppress this warning, making the log file bogged down with these messages. There should be a an override flag that allows you to willingly disabled these log messages.

TypeError when parsing JSON, caused by httplib2/Python3 returning 'bytes' object instead of a string.

Hi.

I'm now trying to develop software on top of hp3pardata API, and noticed it won't work with Python3.
Even the following simplest code fails with a TypeError:

#!/usr/bin/python
# -*- coding: utf-8 -*-
import hp3parclient
from hp3parclient.client import *
cl = HP3ParClient("http://172.16.4.161:8008/api/v1")

The root cause was that httplib2.request() method now returns a 'bytes' object instead of a 'string' due to a change in Python3 (I'm using Python 3.3). This broke JSON parser when 'bytes' object is passed.

Quick fix is to turn HTTP response body back into string as I posted in #11 .

Best Regards,

python-3parclient.git install issues

[root@dlseelm-bb6006 pike]# pip install --upgrade python-3parclient
'Collecting python-3parclient
Using cached python-3parclient-4.2.5.tar.gz
Collecting paramiko (from python-3parclient)
Using cached paramiko-2.4.0-py2.py3-none-any.whl
Collecting eventlet (from python-3parclient)
Using cached eventlet-0.22.0-py2.py3-none-any.whl
Collecting requests (from python-3parclient)
Using cached requests-2.18.4-py2.py3-none-any.whl
Collecting cryptography>=1.5 (from paramiko->python-3parclient)
Using cached cryptography-2.1.4.tar.gz
Complete output from command python setup.py egg_info:
error in cryptography setup command: Invalid environment marker: python_version < '3'

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-uPI1BK/cryptography/

ssh.HPE3PARSSHClient shownode incomplete output

We are using ssh.HPE3PARSSHClient for monitoring scripts.
For 2-node systems using command shownode we are experiencing incomplete output. Node 0 and also the header is missing. This is not happening all the time, about 6 times per day.

Complete output:

,​,​,​,​,​,​,​Control,​Data,​Cache
Node,​----Name----,​-State-,​Master,​InCluster,​-Service_LED,​---LED---,​Mem(MB),​Mem(MB),​Available(%)
0,​CZ29190K73-0,​OK,​Yes,​Yes,​Off,​GreenBlnk,​16384,​16384,​100
1,​CZ29190K73-1,​OK,​No,​Yes,​Off,​GreenBlnk,​16384,​16384,​100

Incomplete output:

1,​CZ29190K73-1,​OK,​No,​Yes,​Off,​GreenBlnk,​16384,​16384,​100

We have proven there is no problem on the system. In the same time we executed the command in a different way, the output was always complete. The problem does not occur for 4-node systems.

Function getting the data:

def get_data_ssh(array, cmd_list, severity_rc):

    cl = ssh.HPE3PARSSHClient(array, username, password, conn_timeout=timeout_ssh)

    try:
        cl.open()

    except paramiko.ssh_exception.SSHException:
        print('CRITICAL: Login to {} via ssh failed '.format(array))
        print(os.linesep + 'Process runtime: ' + str(time.time() - start_time) + ' seconds')
        exit(severity_rc)

    data = cl.run(cmd_list)
    cl.close()
    return data

Have privilege to fetch volume(s) in chunks

When an array has huge number of volume(s) the GET call takes lot of time to return the information. For which 3PAR has a solution to return the information in chunks. Just a thought that it will be great to have feature.

PS: I will be grateful if I can help in accomplishing this.

Missing an "L" to urllib (client.py, line 34)

There's an "L" missing to "urllib" line 34

try:
# For Python 3.0 and later
from urlib.parse import quote
except ImportError:
# Fall back to Python 2's urllib2
from urllib2 import quote

SSH multi-line-parser fails on slow connections

On slow SSH connections both parsers, default and multi-line, fail because the responses are somewhat different than what they expect:

  • We don't get the initial stdin dump of the commands we've sent before the first prompt.
  • We get empty prompts with no command, as if we had entered an additional enter key.

Fixing the default parser would be more complicated, but the multi-line one should be able to support these cases.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.