Giter Club home page Giter Club logo

gui-backend's People

Contributors

amercer1 avatar amit4111989 avatar c-mart avatar cdosborn avatar cjlarose avatar dleehr avatar edwins avatar erbriones avatar jchansen avatar jmatt avatar julianpistorius avatar lenards avatar mgatto avatar mlent avatar netscruff avatar pns005 avatar prosif avatar rob-baron avatar steve-gregory avatar tharon-c avatar xuhang57 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

Forkers

rob-baron

gui-backend's Issues

celery does not always close all of its worker processes after a full-stop.

With the error that it was unable to allocate memory.

snippet of error log:
File "/opt/env/atmo/local/lib/python2.7/site-packages/billiard/pool.py", line 1237, in _repopulate_pool
self._create_worker_process(self._avail_index())
File "/opt/env/atmo/local/lib/python2.7/site-packages/celery/concurrency/asynpool.py", line 415, in _create_worker_process
return super(AsynPool, self)._create_worker_process(i)
File "/opt/env/atmo/local/lib/python2.7/site-packages/billiard/pool.py", line 1068, in _create_worker_process
w.start()
File "/opt/env/atmo/local/lib/python2.7/site-packages/billiard/process.py", line 137, in start
self._popen = Popen(self)
File "/opt/env/atmo/local/lib/python2.7/site-packages/billiard/forking.py", line 99, in init
self.pid = os.fork()
MemoryError: [Errno 12] Cannot allocate memory

Also this one:

File "/opt/env/atmo/lib/python2.7/site-packages/rest_framework/settings.py", line 154, in perform_import
return [import_from_string(item, setting_name) for item in val]
File "/opt/env/atmo/lib/python2.7/site-packages/rest_framework/settings.py", line 170, in import_from_string
raise ImportError(msg)
ImportError: Could not import 'api.renderers.BrowsableAPIRenderer' for API setting 'DEFAULT_RENDERER_CLASSES'. ImportError: /opt/env/atmo/lib/python2.7/site-packages/pandas/_sparse.so: failed to map segment from shared object: Cannot allocate memory.

Sometimes this will disappear just by waiting.

Initial login, every provider has an admin account

https://trello.com/c/oqYvRTAR/56-working-around-the-provider-account-on-the-initial-startup

When we are logging into the system, there is a login account associated with each provider that is used to:

  1. fetch images
  2. create user accounts as users login into the provider.

At this point my install has started crashing as I haven't specified an appropriate account. Currently this is stopping in a celery process and yielding a 500 error in the web browser. I am not sure why this wasn't an issue last week.

Cannot use Chrome to view the website due to an unknown Cert problem

It works on the Firefox. But not Chrome

Since the SSL_* variables names changed to TLS_… More details in the variables.yml.dist in MOC-Atmosphere-clank repo.

Possible solutions here:

  • revert to a backup site.conf, etc. that points to our valid certs/keys/etc
  • if after re-clank and updating variables to TLS_…things _still_dont work, we would need some help for triaging the SSL cert troubles

identity definition

Identity should only be used in the mapping between an atmosphere user and a provider.

Currently for open stack, the project name is also related through the credential field. Unfortunately when a new login is done, a new identity is created (same default project).

So it would be cleaner to model this part as an inheritance hierarchy:

identity
    Key identity_id
    keystone unstopped token should be associated here.

provider_project
    key: identity_id, provider_project_id
    (keystone scoped token should be associated here)

instance
    key: identity_id, provider_project_id, instance_id

This way an identity has many projects, and a project has many instances

For now, I am removing the openstack project from the identity - this way there is not a separate identity for each project.

This simplifies the login process as we only need to store the current credentials at one identity (as opposed to n identities). Furthermore, it simplifies the login process and the switching between projects.

Removal of Admin Account

There are a few functions the Admin Account is used for that can be done with using either the user account or data that is currently in the database. These are:

1) initial retrieval of images
2) creation of user accounts and quotas when a user logs in.

1 can be done with data that is in the database when there is no valid auth token (before someone has logged in).

I have coded for 2 with the previous version of our code base and am in the process to confirm that is fully merged with our current code base.

Associate instances to the correct project

Although open stack compute nodes are associated with an open stack project, they are not automatically associated with an atmosphere project. After atmosphere projects are synced with openstack projects, the compute nodes will should be placed into the correct project.

sometimes the cached driver is not fetched

Here is the snippet from the log:

R = retval = fun(*args, **kwargs)

File "/opt/env/atmo/local/lib/python2.7/site-packages/celery/app/trace.py", line 438, in protected_call
return self.run(*args, **kwargs)
File "service/tasks/monitoring.py", line 583, in monitor_instances_for
instance_map = _get_instance_owner_map(provider, users=users)
File "service/monitoring.py", line 411, in _get_instance_owner_map
all_instances = get_cached_instances(provider=provider, identity=account_identity, force=True)
File "service/cache.py", line 108, in get_cached_instances
instances_method = cached_driver.list_instances
AttributeError: 'NoneType' object has no attribute 'list_instances'

Am checking the cached_driver object to ensure that it is not None.

More general handling of urls with service endpoints

Currently the database does not seem to support different urls for different services. For example, the nova url is assumed to be used for neutron, glance and swift. Although this is ok for the most common openstack configuration, it is not a general solution.

This falls between the GUI-Backend and rtwo.

Before we go live on other openstack clouds, this should be handled.

UnboundLocalError for monitor_size_for()

In [3]: monitor_sizes_for(4)
/opt/env/atmo/local/lib/python2.7/site-packages/libcloud/httplib_ssl.py:229: UserWarning: SSL certificate verification is disabled, this can pose a security risk. For more information how to enable the SSL certificate verification, please visit the libcloud documentation.
  warnings.warn(libcloud.security.VERIFY_SSL_DISABLED_MSG)

---------------------------------------------------------------------------
UnboundLocalError                         Traceback (most recent call last)
<ipython-input-3-18bd2a358b11> in <module>()
----> 1 monitor_sizes_for(4)

/opt/env/atmo/local/lib/python2.7/site-packages/celery/local.pyc in __call__(self, *a, **kw)
    186
    187     def __call__(self, *a, **kw):
--> 188         return self._get_current_object()(*a, **kw)
    189
    190     def __len__(self):

/opt/env/atmo/local/lib/python2.7/site-packages/celery/app/task.pyc in __call__(self, *args, **kwargs)
    418             if self.__self__ is not None:
    419                 return self.run(self.__self__, *args, **kwargs)
--> 420             return self.run(*args, **kwargs)
    421         finally:
    422             self.pop_request()

/opt/dev/atmosphere/service/tasks/monitoring.pyc in monitor_sizes_for(provider_id, print_logs)
    708     # Non-End dated sizes on this provider
    709     db_sizes = Size.objects.filter(only_current(), provider=provider)
--> 710     all_sizes = admin_driver.list_sizes()
    711     seen_sizes = []
    712     for cloud_size in all_sizes:

/opt/env/atmo/local/lib/python2.7/site-packages/rtwo/driver.pyc in list_sizes(self, *args, **kwargs)
    274         return self.provider.sizeCls.get_sizes(
    275             self.provider,
--> 276             super(EshDriver, self).list_sizes)
    277
    278     def list_locations(self, *args, **kwargs):

/opt/env/atmo/local/lib/python2.7/site-packages/rtwo/models/size.pyc in get_sizes(cls, provider, lc_list_sizes_method)
     67         cached_sizes = cls.sizes.get(identifier)
     68         if not cached_sizes or not cls.lc_sizes:
---> 69             cls.lc_sizes = lc_list_sizes_method()
     70             logger.debug("Caching %s sizes for identifier:%s" %
     71                          (len(cls.lc_sizes), identifier))

/opt/env/atmo/local/lib/python2.7/site-packages/rtwo/driver.pyc in list_sizes(self, *args, **kwargs)
    168                      % (self.provider.identifier,
    169                      self.identity.credentials['key']))
--> 170         return self._connection.list_sizes()
    171
    172     def list_locations(self, *args, **kwargs):

/opt/env/atmo/local/lib/python2.7/site-packages/libcloud/compute/drivers/openstack.pyc in list_sizes(self, location)
    297     def list_sizes(self, location=None):
    298         return self._to_sizes(
--> 299             self.connection.request('/flavors/detail').object)
    300
    301     def list_locations(self):

/opt/env/atmo/local/lib/python2.7/site-packages/rtwo/drivers/openstack_facade.pyc in request(self, action, params, data, headers, method, max_attempts)
     98                         action=action,
     99                         params=params, data=data,
--> 100                         method=method, headers=headers)
    101                 return response
    102             except (BaseHTTPError, httplib.HTTPException, socket.error,

/opt/env/atmo/local/lib/python2.7/site-packages/libcloud/common/openstack.pyc in request(self, action, params, data, headers, method, raw)
    200                                                             method=method,
    201                                                             headers=headers,
--> 202                                                             raw=raw)
    203
    204     def _get_auth_url(self):

/opt/env/atmo/local/lib/python2.7/site-packages/libcloud/common/base.pyc in request(self, action, params, data, headers, method, raw)
    707                                        False) or RETRY_FAILED_HTTP_REQUESTS
    708
--> 709         action = self.morph_action_hook(action)
    710         self.action = action
    711         self.method = method

/opt/env/atmo/local/lib/python2.7/site-packages/libcloud/common/openstack.pyc in morph_action_hook(self, action)
    267
    268     def morph_action_hook(self, action):
--> 269         self._populate_hosts_and_request_paths()
    270         return super(OpenStackBaseConnection, self).morph_action_hook(action)
    271

/opt/env/atmo/local/lib/python2.7/site-packages/libcloud/common/openstack.pyc in _populate_hosts_and_request_paths(self)
    300                 kwargs = {}
    301
--> 302             osa = osa.authenticate(**kwargs)  # may throw InvalidCreds
    303
    304             self.auth_token = osa.auth_token

/opt/env/atmo/local/lib/python2.7/site-packages/libcloud/common/openstack_identity.pyc in authenticate(self, force)
   1050             body = 'code: %s body:%s' % (response.status, response.body)
   1051         else:
-> 1052             raise MalformedResponseError('Malformed response', body=body,
   1053                                          driver=self.driver)
   1054

UnboundLocalError: local variable 'body' referenced before assignment

Currently, we could add flavors by hands. Just the python function fails.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.