Test should be update so the can be run in working directory. Currently when running 'python test_cdmi_object.py' the test will fail:

ERROR: test_create_object_cdmi_multipart (main.TestCDMIObject)

Traceback (most recent call last):
File "test_cdmi_object.py", line 172, in test_create_object_cdmi_multipart
fp = open(path + os.sep + 'desert.jpg', 'rb')
IOError: [Errno 2] No such file or directory: '/desert.jpg'

ERROR: test_create_object_non_cdmi_multipart (main.TestCDMIObject)

Traceback (most recent call last):
File "test_cdmi_object.py", line 216, in test_create_object_non_cdmi_multipart
fp = open(os.sep.join([path, 'desert.jpg']), 'rb')
IOError: [Errno 2] No such file or directory: '/desert.jpg'

Running test from 'python cdmi/test_cdmi_object.py' works.

Also tests should make use of os.sep instead of '/'

This might not always be the case - e.g. when keystone and swift proxy are on different host. Maybe we should asume swift proxy == localhost & auth_host == keystone ip?

See:

 conn = httplib.HTTPConnection(self.conf.get('auth_host'), self.conf.get('access_port'))

If you're happy with this I can make the changes accordingly.

Hi,
I installed cdmi to use with swift and keystone. But curl command is always get errors. So I run run_tests.py in cdmi install dir, and end with errors. Please help!
I followed the readme file, and installed the cdmi by python, then configured proxy-server.conf at /etc/swift. Restart the proxy-server. Here is my proxy-server.conf
///////////////////////////////////////////proxy-server.conf/////////////////////////////////////
[DEFAULT]
bind_ip = 0.0.0.0
bind_port = 8080
swift_dir = /etc/swift
workers = 8
user = swift

cert_file = /etc/swift/syncsort.cer

key_file = /etc/swift/private.pem

log_name = swift
log_facility = LOG_LOCAL0
log_level = DEBUG

[pipeline:main]
pipeline = catch_errors healthcheck cache swift3 s3token authtoken keystone cdmi proxy-server

[app:proxy-server]
use = egg:swift#proxy
account_autocreate = true
log_level = DEBUG

[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory

the host must point to your keystone server

auth_host = 10.200.33.30
auth_port = 35357
auth_protocol=http
service_host = 10.200.33.30
service_port = 5000
admin_token = ADMIN
admin_user = admin
admin_password = openstack
admin_tenant_name = adminTenant
signing_dir = /etc/swift

[filter:keystone]
paste.filter_factory = keystone.middleware.swift_auth:filter_factory
operator_roles = adminRole, swiftoperator
is_admin = true
reseller_prefix=AUTH

[filter:healthcheck]
use = egg:swift#healthcheck

[filter:cache]
use = egg:swift#memcache
memcache_servers = 127.0.0.1:11211

[filter:domain_remap]
use = egg:swift#domain_remap

[filter:catch_errors]
use = egg:swift#catch_errors

[filter:swift3]
use = egg:swift3#swift3

[filter:cdmi]
use = egg:cdmiapi#cdmiapp

[filter:s3token]
paste.filter_factory = keystone.middleware.s3_token:filter_factory
auth_port = 35357
auth_host = 10.200.33.30
auth_protocol = http
reseller_prefix=AUTH
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////
And here is my test.conf at /etc/swift/
//////////////////////////////////////////test.conf//////////////////////////////////////////////////////////
[func_test]

sample config

auth_host = 10.200.33.30
auth_port = 5000
access_port = 8080
auth_ssl = no
auth_prefix = /v2.0/tokens

account = adminTenant
username = admin
password = openstack

Primary functional test account (needs admin access to the account)

account = test

username = tester

password = testing

User on a second account (needs admin access to the account)

account2 = test2

username2 = tester2

password2 = testing2

User on same account as first, but without admin access

username3 = tester3

password3 = testing3

Default constraints if not defined here, the test runner will try

to set them from /etc/swift/swift.conf. If that file isn't found,

the test runner will skip tests that depend on these values.

Note that the cluster must have "sane" values for the test suite to pass.

max_file_size = 5368709122

max_meta_name_length = 128

max_meta_value_length = 256

max_meta_count = 90

max_meta_overall_size = 4096

max_object_name_length = 1024

container_listing_limit = 10000

account_listing_limit = 10000

max_account_name_length = 256

max_container_name_length = 256

collate = C

[unit_test]
fake_syslog = False

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////
And here is my log when run run_tests.py
/////////////////////////////////////////runtest.log////////////////////////////////////////////////////////
************* Module cdmi
C0111: 1,0: Missing docstring
W0403: 17,0: Relative import 'cdmi', should be 'cdmi.cdmi'
C0111: 37,4:filter_factory.cdmi_filter: Missing docstring
************* Module cdmi.cdmi
W0403: 31,0: Relative import 'cdmiapp.cdmibase', should be 'cdmi.cdmiapp.cdmibase'
W0403: 33,0: Relative import 'cdmiapp.cdmibase', should be 'cdmi.cdmiapp.cdmibase'
W0403: 34,0: Relative import 'cdmiapp.cdmicontrollers', should be 'cdmi.cdmiapp.cdmicontrollers'
W0403: 36,0: Relative import 'cdmiapp.cdmicommoncontroller', should be 'cdmi.cdmiapp.cdmicommoncontroller'
W0403: 38,0: Relative import 'cdmiapp.noncdmicontrollers', should be 'cdmi.cdmiapp.noncdmicontrollers'
W0403: 40,0: Relative import 'cdmiapp.cdmiutils', should be 'cdmi.cdmiapp.cdmiutils'
W0613: 50,0:CdmiMiddleware.init: Unused argument 'args'
W0613: 50,0:CdmiMiddleware.init: Unused argument 'kwargs'
C0103: 59,4:CdmiMiddleware.get_container_controller_by_version: Invalid name "get_container_controller_by_version" (should match [a-z_][a-z0-9_]{2,30}$)
C0111: 59,4:CdmiMiddleware.get_container_controller_by_version: Missing docstring
R0201: 59,4:CdmiMiddleware.get_container_controller_by_version: Method could be a function
C0103: 66,4:CdmiMiddleware.get_object_controller_by_version: Invalid name "get_object_controller_by_version" (should match [a-z_][a-z0-9_]{2,30}$)
C0111: 66,4:CdmiMiddleware.get_object_controller_by_version: Missing docstring
R0201: 66,4:CdmiMiddleware.get_object_controller_by_version: Method could be a function
C0111: 73,4:CdmiMiddleware.get_controller: Missing docstring
R0914: 73,4:CdmiMiddleware.get_controller: Too many local variables (21/15)
C0103:132,16:CdmiMiddleware.get_controller: Invalid name "d" (should match [a-z_][a-z0-9_]{2,30}$)
C0103:157,16:CdmiMiddleware.get_controller: Invalid name "d" (should match [a-z_][a-z0-9_]{2,30}$)
C0103:166,16:CdmiMiddleware.get_controller: Invalid name "d" (should match [a-z_][a-z0-9_]{2,30}$)
C0103:172,16:CdmiMiddleware.get_controller: Invalid name "d" (should match [a-z_][a-z0-9_]{2,30}$)
C0103:177,12:CdmiMiddleware.get_controller: Invalid name "d" (should match [a-z_][a-z0-9_]{2,30}$)
W0612: 90,12:CdmiMiddleware.get_controller: Unused variable 'content_length'
W0612: 81,12:CdmiMiddleware.get_controller: Unused variable 'content_is_object'
R0912: 73,4:CdmiMiddleware.get_controller: Too many branches (26/12)
R0915: 73,4:CdmiMiddleware.get_controller: Too many statements (68/50)
W0142:200,25:CdmiMiddleware.call: Used * or ** magic
C0111:226,4:filter_factory.cdmi_filter: Missing docstring
W0611: 44,0: Unused import split_path
W0611: 41,0: Unused import Request
W0611: 31,0: Unused import Consts
W0611: 41,0: Unused import Response
R0801: 1,0: Similar lines in 2 files
==cdmi.init:29
==cdmi.cdmi:218
cdmi_root = conf.setdefault('cdmi_root', 'cdmi')
cdmi_root = cdmi_root.lstrip('/ ').rstrip('/ ')
conf['cdmi_root'] = cdmi_root
conf['cdmi_root_length'] = len(cdmi_root.split('/'))
conf.setdefault('cdmi_version_supported', '1.0.1')
conf.setdefault('cdmi_capability_id', 'cdmi_capabilities')

def cdmi_filter(app):
    return CdmiMiddleware(app, conf)

return cdmi_filter

cdmi/cdmiapp/cdmicommoncontroller.py:384:57: W291 trailing whitespace
1 W291 trailing whitespace
cdmi/cdmi.py:31: 'Consts' imported but unused
cdmi/cdmi.py:41: 'Request' imported but unused
cdmi/cdmi.py:41: 'Response' imported but unused
cdmi/cdmi.py:44: 'split_path' imported but unused
cdmi/cdmi.py:81: local variable 'content_is_object' is assigned to but never used
cdmi/cdmi.py:90: local variable 'content_length' is assigned to but never used
cdmi/cdmiapp/cdmicontrollers.py:16: 'Controller' imported but unused
cdmi/cdmiapp/cdmicontrollers.py:22: 'unquote' imported but unused
cdmi/cdmiapp/cdmicontrollers.py:22: 'quote' imported but unused
cdmi/cdmiapp/cdmicontrollers.py:23: 'Response' imported but unused
cdmi/cdmiapp/cdmicontrollers.py:24: 'get_logger' imported but unused
cdmi/cdmiapp/cdmicontrollers.py:25: 'split_path' imported but unused
cdmi/cdmiapp/cdmicontrollers.py:26: 'http_connect_raw' imported but unused
cdmi/cdmiapp/cdmicommoncontroller.py:21: 'get_logger' imported but unused
cdmi/cdmiapp/cdmicommoncontroller.py:26: 'mimetypes' imported but unused
cdmi/cdmiapp/cdmicommoncontroller.py:291: undefined name 'Excetpion'
cdmi/cdmiapp/cdmiutils.py:23: 'HTTPConnection' imported but unused
cdmi/cdmiapp/cdmiutils.py:173: undefined name 'HTTPSConnection'
cdmi/cdmiapp/noncdmicontrollers.py:16: 'Controller' imported but unused
cdmi/cdmiapp/noncdmicontrollers.py:16: 'Consts' imported but unused
cdmi/cdmiapp/noncdmicontrollers.py:22: 'unquote' imported but unused
cdmi/cdmiapp/noncdmicontrollers.py:23: 'split_path' imported but unused
cdmi/cdmiapp/noncdmicontrollers.py:24: 'Response' imported but unused

Pychecker report

Warnings...

[system path]/httplib.py:807: INTERNAL ERROR -- STOPPED PROCESSING FUNCTION --
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/pychecker/warn.py", line 242, in _checkFunction
_checkCode(code, codeSource)
File "/usr/lib/python2.7/dist-packages/pychecker/warn.py", line 155, in _checkCode
raise NotImplementedError('No DISPATCH member for op %r' % op)
NotImplementedError: No DISPATCH member for op 50

[system path]/urlparse.py:352: (dict) shadows builtin

[system path]/swift/common/bufferedhttp.py:101: Overridden method (getresponse) doesn't match signature in class (httplib.HTTPConnection)

[system path]/swift/common/utils.py:716: Function (import) doesn't support **kwArgs

[system path]/webob/request.py:154: No class attribute (environ) found
[system path]/webob/request.py:161: No class attribute (environ) found
[system path]/webob/request.py:164: No class attribute (environ) found
[system path]/webob/request.py:166: No class attribute (environ) found
[system path]/webob/request.py:167: No class attribute (environ) found
[system path]/webob/request.py:190: No class attribute (environ) found
[system path]/webob/request.py:206: No class attribute (environ) found
[system path]/webob/request.py:207: No class attribute (environ) found
[system path]/webob/request.py:216: No class attribute (environ) found
[system path]/webob/request.py:218: No class attribute (environ) found
[system path]/webob/request.py:220: No class attribute (environ) found
[system path]/webob/request.py:233: No class attribute (environ) found
[system path]/webob/request.py:378: No class attribute (environ) found
[system path]/webob/request.py:379: No class attribute (environ) found
[system path]/webob/request.py:380: No class attribute (environ) found
[system path]/webob/request.py:381: No class attribute (environ) found
[system path]/webob/request.py:384: No class attribute (environ) found
[system path]/webob/request.py:388: No class attribute (environ) found
[system path]/webob/request.py:400: No class attribute (environ) found
[system path]/webob/request.py:401: No class attribute (environ) found
[system path]/webob/request.py:402: No class attribute (environ) found
[system path]/webob/request.py:403: No class attribute (environ) found
[system path]/webob/request.py:404: No class attribute (environ) found
[system path]/webob/request.py:407: No class attribute (environ) found
[system path]/webob/request.py:421: No class attribute (environ) found
[system path]/webob/request.py:422: No class attribute (environ) found
[system path]/webob/request.py:429: No class attribute (environ) found
[system path]/webob/request.py:441: No class attribute (environ) found
[system path]/webob/request.py:442: No class attribute (environ) found
[system path]/webob/request.py:443: No class attribute (environ) found
[system path]/webob/request.py:446: No class attribute (environ) found
[system path]/webob/request.py:466: No class attribute (environ) found
[system path]/webob/request.py:467: No class attribute (environ) found
[system path]/webob/request.py:469: No class attribute (environ) found
[system path]/webob/request.py:471: No class attribute (environ) found
[system path]/webob/request.py:473: No class attribute (environ) found
[system path]/webob/request.py:474: No class attribute (environ) found
[system path]/webob/request.py:588: No class attribute (environ) found
[system path]/webob/request.py:673: No class attribute (environ) found
[system path]/webob/request.py:684: No class attribute (environ) found
[system path]/webob/request.py:711: No class attribute (environ) found
[system path]/webob/request.py:715: No class attribute (environ) found
[system path]/webob/request.py:774: (input) shadows builtin
[system path]/webob/request.py:817: No class attribute (environ) found
[system path]/webob/request.py:818: No class attribute (environ) found
[system path]/webob/request.py:837: No class attribute (environ) found
[system path]/webob/request.py:849: No class attribute (environ) found
[system path]/webob/request.py:862: No class attribute (environ) found
[system path]/webob/request.py:869: No class attribute (environ) found
[system path]/webob/request.py:1022: No class attribute (environ) found
[system path]/webob/request.py:1033: Function return types are inconsistent
[system path]/webob/request.py:1170: Invalid arguments to (setattr), got 4, expected 3
[system path]/webob/request.py:1175: No class attribute (environ) found
[system path]/webob/request.py:1177: Invalid arguments to (getattr), got 3, expected 2
[system path]/webob/request.py:1179: No class attribute (environ) found
[system path]/webob/request.py:1183: Invalid arguments to (delattr), got 3, expected 2
[system path]/webob/request.py:1187: No class attribute (environ) found

[system path]/webob/response.py:162: Invalid arguments to (str), got 2, expected 1
[system path]/webob/response.py:166: Statement appears to have no effect
[system path]/webob/response.py:980: Function return types are inconsistent

cdmi/cdmi.py:50: Parameter (args) not used
cdmi/cdmi.py:50: Parameter (kwargs) not used
cdmi/cdmi.py:82: Local variable (content_is_object) not used
cdmi/cdmi.py:90: Local variable (content_length) not used

cdmi/cdmiapp/cdmibase.py:65: Parameter (account_name) not used
cdmi/cdmiapp/cdmibase.py:65: Parameter (env) not used
cdmi/cdmiapp/cdmibase.py:65: Parameter (kwargs) not used
cdmi/cdmiapp/cdmibase.py:73: Parameter (env) not used
cdmi/cdmiapp/cdmibase.py:73: Parameter (kwargs) not used
cdmi/cdmiapp/cdmibase.py:82: Parameter (env) not used
cdmi/cdmiapp/cdmibase.py:82: Parameter (start_response) not used
cdmi/cdmiapp/cdmibase.py:181: Parameter (account_name) not used
cdmi/cdmiapp/cdmibase.py:181: Parameter (kwargs) not used
cdmi/cdmiapp/cdmibase.py:186: Parameter (start_response) not used
cdmi/cdmiapp/cdmibase.py:217: Parameter (kwargs) not used
cdmi/cdmiapp/cdmibase.py:222: Parameter (start_response) not used

cdmi/cdmiapp/cdmicommoncontroller.py:26: Imported module (mimetypes) not used
cdmi/cdmiapp/cdmicommoncontroller.py:39: Parameter (kwargs) not used
cdmi/cdmiapp/cdmicommoncontroller.py:67: Parameter (start_response) not used
cdmi/cdmiapp/cdmicommoncontroller.py:111: Parameter (start_response) not used
cdmi/cdmiapp/cdmicommoncontroller.py:278: Parameter (valid_units) not used
cdmi/cdmiapp/cdmicommoncontroller.py:291: No global (Excetpion) found
cdmi/cdmiapp/cdmicommoncontroller.py:308: Parameter (start_response) not used
cdmi/cdmiapp/cdmicommoncontroller.py:397: Parameter (env) not used
cdmi/cdmiapp/cdmicommoncontroller.py:397: Parameter (start_response) not used
cdmi/cdmiapp/cdmicommoncontroller.py:493: Parameter (start_response) not used

cdmi/cdmiapp/cdmiutils.py:95: Parameter (logger) not used
cdmi/cdmiapp/cdmiutils.py:117: Setting method to itself has no effect
cdmi/cdmiapp/cdmiutils.py:118: Setting path to itself has no effect
cdmi/cdmiapp/cdmiutils.py:152: Parameter (get_body) not used
cdmi/cdmiapp/cdmiutils.py:152: Parameter (logger) not used
cdmi/cdmiapp/cdmiutils.py:152: Parameter (method) not used
cdmi/cdmiapp/cdmiutils.py:152: Parameter (query_string) not used
cdmi/cdmiapp/cdmiutils.py:173: No global (HTTPSConnection) found
cdmi/cdmiapp/cdmiutils.py:174: Object (req) has no attribute (server_portport)

7 errors suppressed, use -#/--limit to increase the number of errors displayed
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////
So what is the problem? Please help! Thank you in advance!

Sincerely,
Peng.Shi

Hi,

it seems that the Swift API, which is ultimately called for requests to the CDMI API, has an upper page size limit of 10000 elements. We have a container that has more than 10000 elements under a single (pseudo) path.

Containers (pseudo-directories) with more than 10000 elements must be listed through pagination (last element of a page given as marker to start the next page).

The implementation of cdmiutils.check_resource() does not do this at the moment. It merely lists the first 10000 elements and returns them.

Cheers,
Björn