softlayer / jumpgate Goto Github PK

Request

• Update core bootstrap styles and scripts to Bootstrap 3.0.3
• This includes:
  • css > bootstrap.min.css
  • scripts > affix.js
  • scripts > application.js
  • scripts > bootstrap.min.js
  • scripts > collapse.js
  • scripts > holder.js
  • scripts > scrollspy.js

Acceptance Criteria

• Verify that the new bootstrap.css does not introduce conflicts with our pages.css or landing.css custom stylesheets

The current implementation uses lastTransaction and activeTransaction in order to get up to two 'events' that have happened on a VM. This might pass some tests but ultimately is not useful at all for auditing/debugging. A new implementation could use SoftLayer_Event Log in order to provide this information. There are some comments already in jumpgate/compute/drivers/sl/endpoints/instance_actions.py which might help with this.

SoftLayer's Object Storage uses swift itself, so the actual API wouldn't need to be emulated. What would be nice to be emulated is keystone-like authentication for SoftLayer's Object Storage instead of the swift 1.0-style authentication that is currently used.

The primary issue with doing this is that Object Storage accounts don't map 1-to-1 with SoftLayer accounts. One SoftLayer account could have multiple Object Storage accounts.

We could have tenants which are specific to specific object storage accounts and when authenticating to jumpgate-keystone it would actually authenticate and return a valid tokens from swift itself. The reason we'd need to that is that we cannot make ad-hoc tokens that are valid with swift without adding a middleware to swift. This would require auth endpoints for all swift clusters to be configured with jumpgate-keystone or have some way to look it up using the normal SoftLayer API (that might actually exist currently).

Authentication would look like (at least for the dev jumpgate setup):

swift --os-auth-url=http://127.0.0.1:5000/v2.0 --os-username=USERNAME --os-password=API_KEY --os-tenant-name=SLOS12345-1 --os-region-name=dal05 stat -v

currently, SL image driver does not support image metadata feature. we can support this feature by using Tags or notes fields of Block_Device_Template_Group. But there are some pros and cons of these approaches from Softlayer side. The maximun length of tag field is only 20 characters. But once set, only authorized user can edit this. however notes field do not have such limitation on size of characters but it's an editable object.

SL API allows to create a CCI with SAN disk storage by setting localDiskFlag to false.

Now Jumpgate allows to create a CCI with a local disk only. It would be great to have SAN support as well.

1. Document what tools exist for Openstack and determine the most common API calls
2. Determine the range of support for each of these tools and document caveats when using Jumpgate with the SoftLayer driver.

• Language Bindings/CLI Clients
• Other Integrations/Tooling
  • Vagrant-openstack
  • RightScale
  • ironfan
  • salt-cloud (now a part of salt)
  • libcloud
  • jclouds
  • knife-openstack

• Listing VLANs
• Listing Subnets.
• VLAN detail
• Subnet detail
• ???

There's a lot of discovery to do for this for the actual functionality. The network manager is a good start as far as seeing what the SoftLayer API supports (and how) and, as always, the API docs for OpenStack projects is a good reference along with the quantum CLI client and, finally, tempest.

The jumpgate compute driver for nova is returning the incorrect flavor mapping on 'server show'.

For example I have the following CCI which I created through jumpgate (from the instance order info on SL web UI):

Name: Cloud Server
Domain: nova-jumpgate-dal.jumpgate.com Quantity: 1
 Hourly Setup
Data Center
Dallas 1
2 x 2.0 GHz Cores
2 GB

I created this server via nova / jumpgate with flavor 3 which corresponds to:

| 3  | 2 vCPU, 2GB ram, 100GB | 2048      | 100  | 0         |      | 2     | 1           | True      |

But when I show it via nova / jumpgate I get:

root@jump:~# nova show nova-jumpgate-dal
+-----------------------------+-------------------------------------------------------------------------+
| Property                    | Value                                                                   |
+-----------------------------+-------------------------------------------------------------------------+
| OS-EXT-AZ:availability_zone | 3                                                                       |
| OS-EXT-STS:power_state      | 1                                                                       |
| OS-EXT-STS:task_state       | -                                                                       |
| OS-EXT-STS:vm_state         | ACTIVE                                                                  |
| accessIPv4                  |                                                                         |
| accessIPv6                  |                                                                         |
| created                     | 2014-03-22T12:19:15-06:00                                               |
| flavor                      | 1 vCPU, 1GB ram, 25GB (1)                                               |
| hostId                      | 3979788                                                                 |
| id                          | 3979788                                                                 |
| image                       | frozen.ubuntu.storage.2013/06/25 (7a938011-a7c2-4e4e-9394-4c6456a24d07) |
| image_name                  |                                                                         |
| name                        | nova-jumpgate-dal                                                       |
| private network             | MY_PRIVATE_IP                                                            |
| public network              | MY_PUBLIC_IP                                                           |
| security_groups             | default                                                                 |
| status                      | ACTIVE                                                                  |
| tenant_id                   | 278184                                                                  |
| updated                     | 2014-03-22T12:23:12-06:00                                               |
| user_id                     | 170512                                                                  |
+-----------------------------+-------------------------------------------------------------------------+

As you can see above, jumpgate incorrectly shows the instance as 'flavor 1'.

Haven't had time to dig into this one, but appears to be a looming issue.

In the current jumpgate impl, the domain name is always jumpgate.com as per: https://github.com/softlayer/jumpgate/blob/master/jumpgate/compute/drivers/sl/servers.py#L200

Clearly this is something that is slated to be updated given the 'TODO' in the code.

This task is to track such work.

A few ideas on how we might consider more dynamic fqdn on boot server:

• IMHO there should be a conf option to specify the default domain name to use. For example in jumpgate.conf

[DEFAULT]
default_domain_name = provider.com

This value should be used as the domain name when non is specified on the boot server call.

• In the jumpgate handling for boot server call we could:
  • Accept a server name which is in FQDN format. If the body['server']['name'] is in FQDN format, we split the hostname from the domain name and use the 1st portion for the hostname and the later for the domain name (i.e. <hostname>.<domain_name_may_include_dots>)
  • Optionally consider supporting obtaining the host/domain name from user_data (http://docs.openstack.org/user-guide/content/user-data.html).. Based on the nova user_data it appears the way this is done for openstack proper is to pass down as user data. Thus we "could" check for user data in the server request body, base64 decode it and use as host/domain name.... On the fence about this one, but wanted to present it for completeness.

Right now we print in order to display request events. Normally this is handled by a legit server like HAProxy or Apache but for development and production debugging we need to leverage python's logging more.

I've been trying to get the openstack (horizon) dashboard to work with jumpgate.

As part of that effort we are missing the jumpgate support for 'show image' via glance client. As shown in the glance client here: https://github.com/openstack/python-glanceclient/blob/master/glanceclient/v1/images.py#L115 the following URI is used to get metadata about a single image:

HEAD /v1/images/{image_id}

The response to this request should return the metadata properties about an image.

Here's a some details about what openstack proper returns for the request:

HTTP/1.1 200 OK
content-length: 0
x-image-meta-status: active
x-image-meta-owner: 203023b0d3694c23a185003aec8f7a68
x-image-meta-name: cirros-0.3.1-x86_64-uec
x-image-meta-container_format: ami
x-image-meta-created_at: 2014-03-22T18:55:15
etag: f8a2eeee2dc65b3d9b6e63678955bd83
location: http://50.97.34.61:9292/v1/images/904f7415-5342-41b4-a56e-d625e103eba3
x-image-meta-min_ram: 0
x-image-meta-updated_at: 2014-03-22T18:55:16
x-image-meta-id: 904f7415-5342-41b4-a56e-d625e103eba3
x-image-meta-property-ramdisk_id: 74a42fde-2f37-4025-ab97-8fd20d855017
date: Sun, 23 Mar 2014 10:29:16 GMT
x-image-meta-property-kernel_id: ba5f40c4-c0f6-4129-8020-908cb9b56187
x-openstack-request-id: req-9f0e8bcc-9824-4b84-bbee-cc3bce7d7b06
x-image-meta-deleted: False
x-image-meta-checksum: f8a2eeee2dc65b3d9b6e63678955bd83
x-image-meta-protected: False
x-image-meta-min_disk: 0
x-image-meta-size: 25165824
x-image-meta-is_public: True
content-type: text/html; charset=UTF-8
x-image-meta-disk_format: ami

Even if we return a very limited subset of these properties I believe this will suffice for initial support of the API.

Right now, the SL controller for 'servers' knows how to handle metadata as passed in via a server boot command. This obviously gets created a CCI user metadata on the provision call in SL.

I would like to propose that support is added for personality in the same manner.

Obviously the translation from nova boot REST call into CCI provision w/personality needs to standardize the format of the user data since any given boot server command may have both.

My initial suggestion is to consider a keyed json object put into the CCI user meta... Something like:

{
    "metadata": {...user metadata k/v pairs...},
    "user_data" : THE_BASE64_ENCODED_METADATA_FROM_BOOT_CALL,
    "personality": [...the personality json from boot call...]
}

Given the suggested format, logic within the CCI can pull down the json CCI data and parse accordingly. Maybe you think a less structured format is more versatile?? For example:

metadata=<THE_METADATA>
userdata=<THE_BASE64_ENCODED_USERDATA>
personality=<THE_PERSONALITY_JSON>

I can see some advantages to a more simple approach like this (i.e. easy to parse in bash scripts), but I prefer the JSON style.

Should be very easy to add user_data support.

Snippet of boot call so you can see the JSON format:

curl -i http://myhost:5000/compute/v2/278184/servers -X POST -H "X-Auth-Project-Id: 12234" -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: myid:MY_API_KEY" -d '{"server": {"name": "rds-poc", "imageRef": "092d0096-a440-46e3-bc64-8ee25a36eda7", "flavorRef": "3", "max_count": 1, "min_count": 1, "personality": [{"path": "/etc/guest_info", "contents": "W0RFRkFVTFRdCmd1ZXN0X2lkPTIxNjQyNTM4LTQ3OTYtNDBmMy04ZDM0LTE0MTc5ZDg1MmFiNQpk\nYXRhc3RvcmVfbWFuYWdlcj1teXNxbAp0ZW5hbnRfaWQ9Mjc4MTg0Cg==\n"}]}}'

I believe we should make the identity module grab the catalog information from a config file similarly to how keystone represents their templated backend, for example: https://github.com/openstack/keystone/blob/master/etc/default_catalog.templates.

This would prevent the need to hard code anything and allow much more configurability to our simple identity imitation.

All of the OpenStack REST APIs I've worked with allow a REST API consumer to query they root URI (i.e. http://host:/) to obtain information about the supported REST API versions for the endpoint.

For example in keystone paste ini: https://github.com/openstack/keystone/blob/master/etc/keystone-paste.ini#L85

The other services such as nova, cinder, etc. support this as well.

Although the openstack clients are smart enough to default the API version if a GET on the root URI is invalid/not implemented, it would be beneficial if the jumpgate compat layer returned a proper response here.

What you get with jumpgate right now is something like this:

WARNING:jumpgate.common.nyi:UNKNOWN PATH: GET /
INFO:jumpgate.common.hooks:GET /  501 Not Implemented [ReqId: req-daad5364-81f4-11e3-8654-0695fb7590bd]
127.0.0.1 - - [20/Jan/2014:11:03:59] "GET / HTTP/1.1" 501 95 "-" "python-requests/1.2.2 CPython/2.7.3 Linux/3.2.0-58-virtual"
INFO:gunicorn.access:127.0.0.1 - - [20/Jan/2014:11:03:59] "GET / HTTP/1.1" 501 95 "-" "python-requests/1.2.2 CPython/2.7.3 Linux/3.2.0-58-virtual"

Likely this is an enhancement.

Based on what I'm seeing right now, the token generated by the jumpgate layer is unsecure as it contains the plain text SL username and API token...

Source is here: https://github.com/softlayer/jumpgate/blob/master/jumpgate/common/sl/auth.py#L26

This results in a plan text token as my_sl_username:my_sl_api_key... Although this is transferred over the wire via HTTP headers and should be via SSL, many of the openstack clients will use that token in an API call to validate the token with the identity service like this: GET http://host:port/vX.Y/tokens/{token}. As such the {token} in that URI is the plain text SL user id and API key which can be intercepted by unauthorized parties over the wire.

IMHO -- this is not secure and the token generated by jumpgate should not have any plain text credential info.

Having issues creating a server with a specific network. Is that supported?

I keep getting RESP BODY: {"badRequest": {"message": "Invalid network", "code": "400"}}
I'm using the network id returned from nova net-list or neutron net-list.

The ability to resize an instance (CCI) is an important aspect to managing compute. This feature is obviously supported in native nova instances, but no such support exists in jumpgate.

This issue/enhancement is intended to track that support -- adding the ability to resize a CCI a la jumpgate.

Related issues: softlayer py client upgrade

Right now, the SL controller for 'servers' knows how to handle metadata as passed in via a server boot command. This obviously gets created a CCI user metadata on the provision call in SL.

I would like to propose that support is added for user_data in the same manner.

Obviously the translation from nova boot REST call into CCI provision w/user data needs to standardize the format of the user data since any given boot server command may have both.

My initial suggestion is to consider a keyed json object put into the CCI user meta... Something like:

{
    "metadata": {...user metadata k/v pairs...},
    "user_data" : THE_BASE64_ENCODED_METADATA_FROM_BOOT_CALL
}

Given the suggested format, logic within the CCI can pull down the json CCI user data and parse accordingly. Maybe you think a less structured format is more versatile?? For example:

metadata=<THE_METADATA>
userdata=<THE_BASE64_ENCODED_USERDATA>

I can see some advantages to a more simple approach like this (i.e. easy to parse in bash scripts), but I prefer the JSON style.

Should be very easy to add user_data support.

Snippet of boot call so you can see the JSON format:

curl -i http://myhost:5000/compute/v2/278184/servers -X POST -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: mysluser:THE_API_KEY" -d '{"server": {"name": "xud-test", "imageRef": "092d0096-a440-46e3-bc64-8ee25a36eda7", "flavorRef": "3", "user_data": "ewogICAgImlkIjoiYWJjMTIzIiwKICAgICJ0ZW5hbnQiOiIxMjMyMSIKfQo=", "max_count": 1, "min_count": 1}}'

I was munking with jumpgate + novaclient and creating some CCIs based on the flavors provided by the jumpgate support.

Based on what I'm seeing, the CCIs created via jumpgate are always created with LOCAL storage in SL... So for example:

nova boot --flavor 2 --image MY_STANDARD_IMAGE_ID --availability-zone wdc01

I will get a CCI with LOCAL storage. Best I can tell SAN storage is not an option via jumpgate, but maybe I missed it.

Based on the OpenStack constructs, IMHO the flavors returned by jumpgate should use OS-FLV-EXT-DATA:ephemeral rather than disk as they do now (for ex).

OS-FLV-EXT-DATA:ephemeral in a nova openstack world signifies the storage is LOCAL. Some existing openstack code in fact checks for flavor.ephemeral using the novaclient flavor manager. Also see here: http://docs.openstack.org/user-guide-admin/content/dashboard_manage_flavors.html

For example here's some openstack trove code which expects if local disk is being used that its returned as ephemeral. I know you do not support trove per say, but I wanted to provide an example of how existing code expects the openstack constructs to be followed.

Thanks

Since Jumpgate will add one more layer on top of the SoftLayer API, latency and responsiveness is a concern of mine for the project. In the softlayer-python bindings there exists a TimedClient, which times each API call made using that client.

My idea for this feature is to used SoftLayer.TimedClient for API calls and report on API calls in an after hook if debugging is enabled. With that information, it will be a bit easier to determine how much of the Jumpgate response times are coming from the overhead of Jumpgate and the overhead of the calls to the SoftLayer API. It would also be good to have the entire request/response verb/path/headers/body available to debug both Jumpgate and potential bugs on the SoftLayer API side as well.

A number of the openstack components/CLIs expect to be able to validate a token using GET /vX.Y/tokens/{tokenId}. This is used when the consumer has a reference to an identity token, but they want to make sure it's still valid.. See: http://docs.openstack.org/api/openstack-identity-service/2.0/content/GET_validateToken_v2.0_tokens__tokenId__Token_Operations.html

Currently jumpgate does not support this route/service -- would be great if it could be added.

Likely an enhancement request.

As per the API [1], token can be created using tokes which are generated earlier. A pull request #50 submitted, earlier for this issue, which was waiting for merge #48. Now that #48 has been merged we can modify it to allow create tokens based on older tokens.

[1] http://api.openstack.org/api-ref-identity.html#identity-v2

This issue is to track project on implementing the V3 keystone API. This is mostly covered by the API reference site. This will only cover porting existing functionality in Jumpgate to the V3 version. Which endpoints are implemented is documented on the jumpgate documentation site.

Allow the logging level to be set and/or have a debug option which spews debug information. Getting the SoftLayer API request/response chain is fairly important for this.

This would be used for testing and deploying.

$ pip install -r requirements.txt

If you follow the install instructions for jumpgate (http://softlayer.github.io/jumpgate/get-started/) you still need to manually install the softlayer-python package (i.e. pip install softlayer). It seems this should be included in the requirements.txt rather than having the user manually determine its missing and have to install it separately.

Openstack allows to list the endpoints associated with a specified token using v2.0/tokens/​{tokenId}​/endpoints.
Though service-catalog is supported by JG, but JG fails when some other services like savanna tries to get endpoint-list using a specific token.

To best test this, I feel like re-using the fixtures defined in the softlayer/softlayer-python project is best. Unfortunately, they're not yet in a place where they can easily be exported. This issue might require that to happen.

Right now the jumpgate code has hardcoded quota-sets. See: https://github.com/softlayer/jumpgate/blob/master/jumpgate/compute/drivers/sl/quota_sets.py#L4

While I understand these defaults taken mimic those of nova's default quotas, in reality we need a more flexible means to set these quotas in jumpgate.

While it's entirely possible to (enhance) support setting quotas dynamically, jumpgates stateless nature does not fit well into a dynamic API model for setting these.

Therefore I'm thinking an initial approach may be to support setting the quotas in the jumpgate.conf file and use them in the quota response.

For example in the conf

[compute]
quota_sets:instances = 100
quota_sets:cores = 1000
# etc

One could imagine a more elaborate scheme which is fine. The point here is that the hard-coded static constants aren't very flexible and limit consumption by clients.

We either need to do one of the following:

• implement this: falconry/falcon#55
• find another solution for handling exceptions. The current method is wrapping everything in a try-except block in a very flat way or to figure out how to layer on our own exception handler independent of Falcon

Hi,
I'm noticing when i delete my VM, it goes to a BUILD state then to a SHUTOFF state. That Build state is confusing and probably should not be there.

Wissam

In the current authN in jumpgate, there is no validation that the given tenantId in the authN request body matches the account id in softlayer. As a result I can use a tenant_id in the openstack clients which has no relation to my SL account and authN just fine if my userid and password (or api key is valid). In such cases a response token from jumpgate will return the valid SL account id -- ignoring the tenant id I passed in.

The mitigation is fairly simple -- update the get_new_token to validate the SL account id against the request tenant..

For example:

    # If the 'password' is the right length, treat it as an API api_key
    if len(credential) == 64:
        client = Client(username=username, api_key=credential)
        user = client['Account'].getCurrentUser(mask=USER_MASK)
        if str(user['accountId']) != lookup(credentials, 'auth', 'tenantId'):
            raise Unauthorized('Invalid username, password or tenant id')
        return {'username': username,
                'api_key': credential,
                'auth_type': 'api_key',
                'tenant_id': str(user['accountId']),
                'expires': time.time() + (60 * 60 * 24)}, user
    else:
        client = Client()
        client.auth = None
        try:
            userId, tokenHash = client.authenticate_with_password(username,
                                                                  credential)
            user = client['Account'].getCurrentUser(mask=USER_MASK)
            if str(user['accountId']) != lookup(credentials, 'auth', 'tenantId'):
                raise Unauthorized('Invalid username, password or tenant id')
            return {'userId': userId,
                    'tokenHash': tokenHash,
                    'auth_type': 'token',
                    'tenant_id': str(user['accountId']),
                    'expires': time.time() + (60 * 60 * 24)}, user
        except SoftLayerAPIError as e:

Assuming you guys agree with this issue/approach, I can code something more formal up and deliver tomorrow.

If you setup jumpgate + novaclient and then do a nova flavor-list you get the following flavors back:

+----+------------------------+-----------+------+-----------+------+-------+-------------+-----------+
| ID | Name                   | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
+----+------------------------+-----------+------+-----------+------+-------+-------------+-----------+
| 1  | 1 vCPU, 1GB ram, 25GB  | 1024      | 25   | 0         |      | 1     | 1           | True      |
| 2  | 1 vCPU, 1GB ram, 100GB | 1024      | 100  | 0         |      | 1     | 1           | True      |
| 3  | 2 vCPU, 2GB ram, 100GB | 2048      | 100  | 0         |      | 2     | 1           | True      |
| 4  | 4 vCPU, 4GB ram, 100GB | 4096      | 100  | 0         |      | 4     | 1           | True      |
| 5  | 8 vCPU, 8GB ram, 100GB | 8192      | 100  | 0         |      | 8     | 1           | True      |
| 1  | 1 vCPU, 1GB ram, 25GB  | 1024      | 25   | 0         |      | 1     | 1           | True      |
+----+------------------------+-----------+------+-----------+------+-------+-------------+-----------+

Notice how there are 2 identical flavors with id 1.

I didn't poke at this, but likely due to FLAVORS[None] = FLAVORS[1]

Due to the current state of eventlet and gevent, it appears that the best deployment strategy at the moment (evented: http://docs.gunicorn.org/en/latest/design.html#choosing-a-worker-type) has to involve Python 2.7. However, Jumpgate currently exclusively works with Python 3.

I don't believe we're using too much that is exclusive to Python 3.3 currently. The biggest things that come to mind are:

• Not using __init__.py files
• Relative imports

Main libraries:

• Requests (Supports 2.6, 2.7, 3.3)
• SoftLayer bindings (Supports 2.6, 2.7, 3.3)
• Falcon (Supports 2.6, 2.7, 3.3)

I would like to make jumpgate authN & token generation pluggable which would permit schemes like facaded authN whereupon we authN POST token requests against a non-SLIAPI service and then perform SLAPI "run as" operations on behalf of the user.

The scheme I was thinking is as follows:

• Create a 'TokenAuthDriver' interface which promises a single method get_new_token(credentials) and should validate the credentials + return a properly formed jumpgate token just like jumpgate authN & token generation does today
• Create a base class of the TokenAuthDriver which is something like SLTokenAuthDriver which effectively implements jumpgate authN & token generation as-is... This is the "default" driver.
• Expose a new jumpgate.conf config property like token_auth_driver which specifies the driver class to use for token authN. The default in jumpgate will be of course the SL base class for backwards compat.
• In the jumpgate logic, add a little simple magic to use the specified token_auth_driver from the conf; again defaulting to the SL driver if not specified.

The above is similar to how OpenStack does pluggable drivers as I'm sure you know.

@beittenc @sudorandom -- any objections here?? I'm about to start working on a PR.

The issue here is that when Virtual_Guest::createArchiveTransaction is called, it returns a transaction instead of an empty blockdevicetemplategroup with an identifying global identifier. This is an issue because the call to create an image from a running server expects to get back a global identifier for the image which it can look up the status of, similarly to how CCIs are provisioned. Currently, SoftLayer's Jumpgate driver waits for the entire capture during the createImage API call, which currently takes minutes.

After #3 is complete this is less of a problem, but this issue likely leads to usability issues on the part of tooling since one API call will sit on the line for minutes for the image capture to complete.

With the latest identity changes there are 2 bugs:

• Tokens can be generated which are non-url friendly. This impacts the consumers ability to use the token on a GET /tokens/{token_id} call.
• The API pipeline should allow unauthenticated calls to GET /tokens/{token_id} to validate an existing token

Hello, any thoughts on what could cause this error?

{"computeFault": {"message": "SoftLayer_Exception_User_Customer_AccountLocked", "code": "500", "details": "Account has been locked for 30 minutes."}}

Seems to happen on all my nova clis against jumpgate... but I can run the sl cli fine using the same account.

OpenStack keystone provides the notion of an admin_token which is effectively a shared secret between keystone and other openstack services. Many services integrating with keystone internally use the admin_token to make internal calls to keystone on behalf of the user - for example to validate a user token (ex see here). For example see the keystone middleware which validates the admin_token in a request here.

We need a similar notion in jumpgate in order to ease integration with openstack services using jumpgate. This jumpgate admin_token should permit authz to the jumpgate facing APIs -- it does not need to map into a SLAPI. For example, I should be able to validate a user (jumpgate based) token using the GET /v2.0/tokens/{token_id} API in jumpgate whereupon I can specify the admin_token secret key as my X-Auth-Token header and get into the jumpgate API to validate the token.

I will be submitting a PR in a min which implements this support and get your feedback.

nova servers should have their id property returned as a string. see sample server response here: http://api.openstack.org/api-ref-compute.html

however in the current jumpgate code its returned as a number... over the wire in json it does not have quotes around it and thus is marshalled into a 'number' on the client side... this causes problems for some code as the code expects the server.id to be a string...

simple fix... PR coming.

When user gets the details of the server (VM) she invokes

GET v2/​{tenant_id}​/servers/​{server_id}​

The current SL driver responds with both primaryBackendIpAddress and primaryIpAddress [1]. However, there is no way to differentiate between them.

By using OS-EXT-IPS:type [2] we can let user determine which one is which (private or public). Many services expects fixed for internal IP. We can use that and for primaryIpAddress we can use public as type.

[1] https://github.com/softlayer/jumpgate/blob/master/jumpgate/compute/drivers/sl/servers.py#L404
[2] https://bugs.launchpad.net/openstack-api-site/+bug/1128562

It does not appear that jumpgate's compute (server) driver is honoring the marker and limit params on a GET /servers call.

As per http://api.openstack.org/api-ref-compute-v2.html a GET v2/​{tenant_id}​/servers should support a limit and marker as follows:

marker (Optional)   query   csapi:UUID  
UUID of the server at which you want to set a marker.

limit (Optional)    query   xsd:int 
Integer value for the limit of values to return.

Based on the current jumpgate compute sl driver I do see some code for these around here: https://github.com/softlayer/jumpgate/blob/master/jumpgate/compute/drivers/sl/servers.py#L274

However when I try to use limit and marker I'm getting back my entire list of servers..

For example:

GET http://JUMPGATE_IP:JUMPGATE_PORT/compute/v2/MY_ACCOUNT/servers/detail?limit=21&marker=3912018&project_id=MY_ACCOUNT

Returns all CCIs in my account rather than a max of 21 as requested above.

I didn't have time to dig into this one yet, but I will try to find more details as time permits.

As the softlaye-python allowing user to specify proxy, the SL driver should allow this.

Related pull request: allowing proxy

as per the nova server response, a server json should have a server.user_id which is the creator of the server (as I understand). See sample response here: http://docs.openstack.org/api/openstack-compute/2/content/List_Servers-d1e2078.html

Is this something we can get from the SLAPI?? If possible maybe jumpgate can just return the SL username as the server.user_id

With the introduction of SL client support for proxy in jumpgate (#65) the out of the box jumpgate does not work due to a backleveled version of SL py client getting installed.

When you send a jumpgate request you'll get:

POST /v2.0/tokens  500 Internal Server Error [ReqId: req-b8f4e91e-b1cc-11e3-afe5-0676dee8c029]
127.0.0.1 - - [22/Mar/2014:09:17:38] "POST /v2.0/tokens HTTP/1.1" 500 101 "-" "python-novaclient"
Unexpected Error
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/falcon-0.1.8-py2.7.egg/falcon/api.py", line 96, in __call__
    responder(req, resp, **params)  # pragma: no cover
  File "/usr/local/lib/python2.7/dist-packages/falcon-0.1.8-py2.7.egg/falcon/api_helpers.py", line 262, in do_before
    responder(req, resp, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/falcon-0.1.8-py2.7.egg/falcon/api_helpers.py", line 262, in do_before
    responder(req, resp, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/falcon-0.1.8-py2.7.egg/falcon/api_helpers.py", line 262, in do_before
    responder(req, resp, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/falcon-0.1.8-py2.7.egg/falcon/api_helpers.py", line 261, in do_before
    action(req, resp, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/jumpgate-0.1-py2.7.egg/jumpgate/common/hooks/sl/client.py", line 10, in bind_client
    proxy=cfg.CONF['softlayer']['proxy'])
TypeError: __init__() got an unexpected keyword argument 'proxy'

The problem is that the softlayer py client installed via jumpgate's setup is backleveled and does not support the proxy kwarg.

If you install the softlayer py client from git hub manually (via git clone and then python install) the proxy kwargs is there.

I believe the solution here is to publish the latest version of sl py client to pypy or document in jumpgate that you need to install it manually.