juju / theblues Goto Github PK

Python library for the juju charmstore (v4)

License: GNU Lesser General Public License v3.0

Makefile 2.84% Python 97.16%

theblues's Introduction

Juju is an open source application orchestration engine that enables any application operation (deployment, integration, lifecycle management) on any infrastructure (Kubernetes or otherwise) at any scale (development or production) in the same easy way (typically, one line of code), through special operators called ‘charms’.


👉	Juju	Learn how to quickly deploy, integrate, and manage charms on any cloud with Juju. It's as simple as `juju deploy foo`, `juju integrate foo bar`, ..., on any cloud.

	Charmhub	Sample our existing charms on Charmhub. A charm can be a cluster (OpenStack, Kubernetes), a data platform (PostgreSQL, MongoDB, etc.), an observability stack (Canonical Observability Stack), an MLOps solution (Kubeflow), and so much more.

	Charm SDK	Write your own charm! Juju is written in Go, but our SDK supports easy charm development in Python.

Give it a try!

Let's use Juju to deploy, configure, and integrate some Kubernetes charms:

Set up

You will need a cloud and Juju. The quickest way is to use a Multipass VM launched with the charm-dev blueprint.

Install Multipass: Linux | macOS | Windows. On Linux:

sudo snap install multipass

Use Multipass to launch an Ubuntu VM with the charm-dev blueprint:

multipass launch --cpus 4 --memory 8G --disk 30G --name tutorial-vm charm-dev

Open a shell into the VM:

multipass shell tutorial-vm

Verify that you have Juju and two localhost clouds:

juju clouds

Bootstrap a Juju controller into the MicroK8s cloud:

juju bootstrap microk8s tutorial-controller

Add a workspace, or 'model':

juju add-model tutorial-model

Deploy, configure, and integrate a few things

Deploy Mattermost:

juju deploy mattermost-k8s

See more: Charmhub | mattermost-k8s

Deploy PostgreSQL:

juju deploy postgresql-k8s --channel 14/stable --trust

See more: Charmhub | postgresql-k8s

Enable security in your PostgreSQL deployment:

juju deploy tls-certificates-operator
juju config tls-certificates-operator generate-self-signed-certificates="true" ca-common-name="Test CA"
juju integrate postgresql-k8s tls-certificates-operator

Integrate Mattermost with PostgreSQL:

juju integrate mattermost-k8s postgresql-k8s:db

Watch your deployment come to life:

juju status --watch 1s

(Press Ctrl-C to quit. Drop the --watch 1s flag to get the status statically. Use the --relations flag to view more information about your integrations.)

Test your deployment

When everything is in active or idle status, note the IP address and port of Mattermost and pass them to curl:

curl <IP address>:<port>/api/v4/system/ping

You should see the output below:

{"AndroidLatestVersion":"","AndroidMinVersion":"","IosLatestVersion":"","IosMinVersion":"","status":"OK"}

Congratulations!

You now have a Kubernetes deployment consisting of a Mattermost backed by PosgreSQL with TLS-encrypted traffic!

Clean up

Delete your Multipass VM:

multipass delete --purge tutorial-vm

Uninstall Multipass: Linux | macOS | Windows. On Linux:

snap remove multipass

Next steps

Learn more

Read our user docs
Read our developer docs

Chat with us

Read our Code of conduct and:

Join our chat: Matrix
Join our forum: Discourse

File an issue

Report a Juju bug on Launchpad
Raise a general https://juju.is/docs documentation issue on Github | juju/docs

Make your mark

Read our documentation contributor guidelines and help improve a doc
Read our codebase contributor guidelines and help improve the codebase

theblues's People

Contributors

Stargazers

Watchers

theblues's Issues

API version should be set by the library instead of supplied by user

The comments and repo summary say to use v4 of the API, but, as reported in #43, that now fails with the current version of this library.

The library should have the supported API version in the code, instead of relying on it being provided by the user, so that a given release will always reference the API version that it works with.

Charm store / methods reject full entity IDs

The output from CharmStore.entityId() cannot be used to reference a charm:

>>> from theblues.charmstore import CharmStore
>>> cs = CharmStore('https://api.jujucharms.com/v4')

>>> cs.entityId('wiki-simple')
'cs:bundle/wiki-simple-4'

>>> cs.entityId(cs.entityId('wiki-simple'))
Traceback (most recent call last):
  File "/home/johnsca/juju/python-libjuju/.tox/py35/lib/python3.5/site-packages/theblues/charmstore.py", line 57, in _get
    response.raise_for_status()
  File "/home/johnsca/juju/python-libjuju/.tox/py35/lib/python3.5/site-packages/requests/models.py", line 862, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api.jujucharms.com/v4/cs:bundle/wiki-simple-4/meta/any

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/johnsca/juju/python-libjuju/.tox/py35/lib/python3.5/site-packages/theblues/charmstore.py", line 325, in entityId
    data = self._get(_add_channel(url, channel))
  File "/home/johnsca/juju/python-libjuju/.tox/py35/lib/python3.5/site-packages/theblues/charmstore.py", line 61, in _get
    raise EntityNotFound(url)
theblues.errors.EntityNotFound: https://api.jujucharms.com/v4/cs:bundle/wiki-simple-4/meta/any

>>> cs.files('wiki-simple', filename='bundle.yaml')
'https://api.jujucharms.com/v4/wiki-simple/archive/bundle.yaml'

>>> cs.files(cs.entityId('wiki-simple'), filename='bundle.yaml')
Traceback (most recent call last):
  File "/home/johnsca/juju/python-libjuju/.tox/py35/lib/python3.5/site-packages/theblues/charmstore.py", line 57, in _get
    response.raise_for_status()
  File "/home/johnsca/juju/python-libjuju/.tox/py35/lib/python3.5/site-packages/requests/models.py", line 862, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api.jujucharms.com/v4/cs:bundle/wiki-simple-4/meta/manifest

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/johnsca/juju/python-libjuju/.tox/py35/lib/python3.5/site-packages/theblues/charmstore.py", line 273, in files
    manifest = self._get(manifest_url)
  File "/home/johnsca/juju/python-libjuju/.tox/py35/lib/python3.5/site-packages/theblues/charmstore.py", line 61, in _get
    raise EntityNotFound(url)
theblues.errors.EntityNotFound: https://api.jujucharms.com/v4/cs:bundle/wiki-simple-4/meta/manifest

CharmStore.entity() fails with "unrecognized metadata name "published""

Attempting to access any entity fails with: unrecognized metadata name "published"

>>> from theblues.charmstore import CharmStore
>>> cs = CharmStore('https://api.jujucharms.com/v4')

>>> cs.entityId('ubuntu')
'cs:xenial/ubuntu-8'

>>> cs.entity('ubuntu')
ERROR:root:Error during request: https://api.jujucharms.com/v4/ubuntu/meta/any?include=bundle-machine-count&include=bundle-metadata&include=bundle-unit-count&include=bundles-containing&include=charm-config&include=charm-metadata&include=common-info&include=extra-info&include=owner&include=revision-info&include=published&include=stats&include=resources&include=supported-series&include=terms status code:(500) message: {"Message":"unrecognized metadata name \"published\"","Code":""}
Traceback (most recent call last):
  File "/home/johnsca/juju/python-libjuju/.tox/py35/lib/python3.5/site-packages/theblues/charmstore.py", line 57, in _get
    response.raise_for_status()
  File "/home/johnsca/juju/python-libjuju/.tox/py35/lib/python3.5/site-packages/requests/models.py", line 862, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://api.jujucharms.com/v4/ubuntu/meta/any?include=bundle-machine-count&include=bundle-metadata&include=bundle-unit-count&include=bundles-containing&include=charm-config&include=charm-metadata&include=common-info&include=extra-info&include=owner&include=revision-info&include=published&include=stats&include=resources&include=supported-series&include=terms

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/johnsca/juju/python-libjuju/.tox/py35/lib/python3.5/site-packages/theblues/charmstore.py", line 135, in entity
    return self._meta(entity_id, includes, channel=channel)
  File "/home/johnsca/juju/python-libjuju/.tox/py35/lib/python3.5/site-packages/theblues/charmstore.py", line 106, in _meta
    data = self._get(url)
  File "/home/johnsca/juju/python-libjuju/.tox/py35/lib/python3.5/site-packages/theblues/charmstore.py", line 72, in _get
    message)
theblues.errors.ServerError: (500, '{"Message":"unrecognized metadata name \\"published\\"","Code":""}', 'Error during request: https://api.jujucharms.com/v4/ubuntu/meta/any?include=bundle-machine-count&include=bundle-metadata&include=bundle-unit-count&include=bundles-containing&include=charm-config&include=charm-metadata&include=common-info&include=extra-info&include=owner&include=revision-info&include=published&include=stats&include=resources&include=supported-series&include=terms status code:(500) message: {"Message":"unrecognized metadata name \\"published\\"","Code":""}')

theblues.errors.ServerError: Request timed out: https://api.jujucharms.com/v4...timeout: 3.05

We have been seeing sustained and noticeable charm store api timeouts in OpenStack Charm CI. This causes false test failures in our charm dev/test gate.

Can this be made more resilient and/or configurable, such as retry threshold and timeout backoffs?

DEBUG:runner:Traceback (most recent call last):
DEBUG:runner:  File "/tmp/bundletester-MyqTGN/ceph-osd/tests/gate-basic-xenial-mitaka", line 22, in <module>
DEBUG:runner:    deployment = CephOsdBasicDeployment(series='xenial')
DEBUG:runner:  File "/tmp/bundletester-MyqTGN/ceph-osd/tests/basic_deployment.py", line 40, in __init__
DEBUG:runner:    self._add_services()
DEBUG:runner:  File "/tmp/bundletester-MyqTGN/ceph-osd/tests/basic_deployment.py", line 72, in _add_services
DEBUG:runner:    other_services)
DEBUG:runner:  File "/tmp/bundletester-MyqTGN/ceph-osd/tests/charmhelpers/contrib/openstack/amulet/deployment.py", line 147, in _add_services
DEBUG:runner:    other_services)
DEBUG:runner:  File "/tmp/bundletester-MyqTGN/ceph-osd/tests/charmhelpers/contrib/amulet/deployment.py", line 67, in _add_services
DEBUG:runner:    constraints=svc.get('constraints'))
DEBUG:runner:  File "/var/lib/jenkins/checkout/0/ceph-osd/.tox/func27-smoke/local/lib/python2.7/site-packages/amulet/deployer.py", line 208, in add
DEBUG:runner:    service_name, charm, branch=branch, series=service['series'])
DEBUG:runner:  File "/var/lib/jenkins/checkout/0/ceph-osd/.tox/func27-smoke/local/lib/python2.7/site-packages/amulet/charm.py", line 57, in fetch
DEBUG:runner:    series=series)
DEBUG:runner:  File "/var/lib/jenkins/checkout/0/ceph-osd/.tox/func27-smoke/local/lib/python2.7/site-packages/amulet/charm.py", line 42, in get_charm
DEBUG:runner:    return Charm(with_series(charm_path, series))
DEBUG:runner:  File "/var/lib/jenkins/checkout/0/ceph-osd/.tox/func27-smoke/local/lib/python2.7/site-packages/charmstore/lib.py", line 155, in __init__
DEBUG:runner:    super(Charm, self).__init__(id, api)
DEBUG:runner:  File "/var/lib/jenkins/checkout/0/ceph-osd/.tox/func27-smoke/local/lib/python2.7/site-packages/charmstore/lib.py", line 105, in __init__
DEBUG:runner:    AVAILABLE_INCLUDES).get('Meta')
DEBUG:runner:  File "/var/lib/jenkins/checkout/0/ceph-osd/.tox/func27-smoke/local/lib/python2.7/site-packages/theblues/charmstore.py", line 107, in _meta
DEBUG:runner:    data = self._get(url)
DEBUG:runner:  File "/var/lib/jenkins/checkout/0/ceph-osd/.tox/func27-smoke/local/lib/python2.7/site-packages/theblues/charmstore.py", line 78, in _get
DEBUG:runner:    raise ServerError(message)

DEBUG:runner:theblues.errors.ServerError: Request timed out: https://api.jujucharms.com/v4/~openstack-charmers-next/xenial/ceph-mon/meta/any?include=bundle-machine-count&include=bundle-metadata&include=bundle-unit-count&include=bundles-containing&include=charm-actions&include=charm-config&include=charm-metadata&include=common-info&include=extra-info&include=revision-info&include=stats&include=supported-series&include=manifest&include=tags&include=promulgated&include=perm&include=id timeout: 3.05

AttributeError: 'ConnectionError' object has no attribute 'message'

The original failure that caused this was a DNS resolution failure, but in attempting to turn that into a ServerError, theblues failed with this. Using something like str(exc) would be safest, but you could also do something with getattr(). I think some exception types use msg rather than message, as well.

AttributeError: 'ConnectionError' object has no attribute 'message'
  File "conjureup/controllers/deploy/tui.py", line 34, in do_deploy
    await asyncio.gather(*tasks)
  File "conjureup/juju.py", line 622, in deploy_service
    app_inst = await app.juju.client.deploy(**deploy_args)
  File "juju/model.py", line 1081, in deploy
    entity = await self.charmstore.entity(entity_url, channel=channel)
  File "juju/model.py", line 1926, in coro
    return await self.loop.run_in_executor(None, method)
  File "concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "theblues/charmstore.py", line 136, in entity
    return self._meta(entity_id, includes, channel=channel)
  File "theblues/charmstore.py", line 107, in _meta
    data = self._get(url)
  File "theblues/charmstore.py", line 83, in _get
    message=exc.message)

Set minimum requirement for dependencies

Fixing requirements to specific versions of requests and libcharmstore creates some challenges when trying to interop theblues with other python modules that require newer versions of requests.

Please could a minimum version requirement be set rather than a fixed version.