chaostoolkit / chaostoolkit-kubernetes Goto Github PK
View Code? Open in Web Editor NEWKubernetes driver extension of the Chaos Toolkit probes and actions API
Home Page: https://chaostoolkit.org/drivers/kubernetes/
License: Apache License 2.0
Kubernetes driver extension of the Chaos Toolkit probes and actions API
Home Page: https://chaostoolkit.org/drivers/kubernetes/
License: Apache License 2.0
Currently, chaostoolkit-kubernetes has a method terminate_pods
that supports to kill only one pod or all pods at a time (with and without randomization). It will be great if there is a provision to specify a percentage of pods that can be killed.
Receive the following error when running drain_nodes function.
`failed: TypeError: Got an unexpected keyword argument 'include_uninitialized' to method list_pod_for_all_namespaces'
Installed chaostoolkit-kubernetes using pip3 with Python 3.7.4. Using version chaostoolkit version 1.2.0
Below is the action I'm using:
39 │ "method": [
40 │ {
41 │ "type": "action",
42 │ "name": "drain_node",
43 │ "provider": {
44 │ "type": "python",
45 │ "module": "chaosk8s.node.actions",
46 │ "func": "drain_nodes",
47 │ "arguments": {
48 ~ │ "name": "aks-agentpool-24917760-1"
49 │ }
50 │ }
51 │ }
52 │ ],
Suggest removing it.
I would like to use chaostoolkit to test statefullsets the same way that it is possible to test deployments in kubernetes.
in the CTK cli, the extension summary is supposed to start with the prefix "Chaos Toolkit Extension for "
chaostoolkit-kubernetes/setup.py
Line 25 in b5de1aa
A new function to remove k8s service would be useful:
"provider": {
"type": "python",
"module": "chaosk8s.service.actions",
"func": "remove_service",
"arguments": {
"label_selector": "app=myapp-greeting",
"ns": "default"
}
}
FYI @adrianco
OS: Win10
Python: 3.7.5 But it's the same for 3.7.7
I'm facing a problem when I try to execute pytest
λ pytest
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "c:\users\devil\appdata\local\programs\python\python37\lib\site-packages\_pytest\main.py", line 187, in wrap_session
INTERNALERROR> config._do_configure()
INTERNALERROR> File "c:\users\devil\appdata\local\programs\python\python37\lib\site-packages\_pytest\config\__init__.py", line 820, in _do_configure
INTERNALERROR> self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
INTERNALERROR> File "c:\users\devil\appdata\local\programs\python\python37\lib\site-packages\pluggy\hooks.py", line 308, in call_historic
INTERNALERROR> res = self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR> File "c:\users\devil\appdata\local\programs\python\python37\lib\site-packages\pluggy\manager.py", line 93, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "c:\users\devil\appdata\local\programs\python\python37\lib\site-packages\pluggy\manager.py", line 87, in <lambda>
INTERNALERROR> firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR> File "c:\users\devil\appdata\local\programs\python\python37\lib\site-packages\pluggy\callers.py", line 208, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "c:\users\devil\appdata\local\programs\python\python37\lib\site-packages\pluggy\callers.py", line 80, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File "c:\users\devil\appdata\local\programs\python\python37\lib\site-packages\pluggy\callers.py", line 187, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "C:\Users\devil\AppData\Local\Programs\Python\Python37\lib\site-packages\pytest_sugar.py", line 176, in pytest_configure
INTERNALERROR> sugar_reporter = SugarTerminalReporter(standard_reporter)
INTERNALERROR> File "C:\Users\devil\AppData\Local\Programs\Python\Python37\lib\site-packages\pytest_sugar.py", line 214, in __init__
INTERNALERROR> self.writer = self._tw
INTERNALERROR> AttributeError: can't set attribute
λ python --version
Python 3.7.5
λ pip --version
pip 20.0.2 from c:\users\devil\appdata\local\programs\python\python37\lib\site-packages\pip (python 3.7)
It's happening also on travis : https://github.com/tdevilleduc/chaostoolkit-kubernetes/runs/592029162
I've manage to get rid of this early today but reinstall python (downgrade to 3.7.5) but it appends again. The only thing I've done by then is to rebase my repo from this one
Did you ever add this behaviour ?
Allow passing the Kubernetes context so that we use the right one when connecting.
tejendrabhandari@cloudshell:/chaostoolkitexp/go-demo-8 (fluted-polymer-273010)$ kubectl apply -f k8s/terminate-pods/pod.yaml -n go-demo-8
pod/go-demo-8 created
tejendrabhandari@cloudshell:/chaostoolkitexp/go-demo-8 (fluted-polymer-273010)$ pip install -U chaostoolkit-kubernetes
Python 2 is deprecated. Upgrade to pip3 as soon as possible.
See https://cloud.google.com/python/docs/python2-sunset
To suppress this warning, create an empty ~/.cloudshell/no-pip-warning file.
The command will automatically proceed in 5 seconds or on any key.
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Defaulting to user installation because normal site-packages is not writeable
Collecting chaostoolkit-kubernetes
Downloading chaostoolkit_kubernetes-0.1.0-py2.py3-none-any.whl (5.6 kB)
Requirement already satisfied, skipping upgrade: pycodestyle in /usr/local/lib/python2.7/dist-packages (from chaostoolkit-kubernetes) (2.5.0)
Collecting chaostoolkit-lib
Downloading chaostoolkit_lib-0.1.0-py2.py3-none-any.whl (15 kB)
Requirement already satisfied, skipping upgrade: pyyaml in /home/tejendrabhandari/.local/lib/python2.7/site-packages (from chaostoolkit-kubernetes) (5.3.1)
Requirement already satisfied, skipping upgrade: kubernetes in /home/tejendrabhandari/.local/lib/python2.7/site-packages (from chaostoolkit-kubernetes) (11.0.0)
Requirement already satisfied, skipping upgrade: logzero in /home/tejendrabhandari/.local/lib/python2.7/site-packages (from chaostoolkit-lib->chaostoolkit-kubernetes) (1.5.0)....
....
Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests->kubernetes->chaostoolkit-kubernetes) (2.9)
Requirement already satisfied, skipping upgrade: pyasn1>=0.1.3 in /usr/local/lib/python2.7/dist-packages (from rsa<4.1,>=3.1.4->google-auth>=1.0.1->kubernetes->chaostoo
lkit-kubernetes) (0.4.8)
Installing collected packages: chaostoolkit-lib, chaostoolkit-kubernetes
Successfully installed chaostoolkit-kubernetes-0.1.0 chaostoolkit-lib-0.1.0
after succssfull install
Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests->kubernetes->chaostoolkit-kubernetes) (2.9)
Requirement already satisfied, skipping upgrade: pyasn1>=0.1.3 in /usr/local/lib/python2.7/dist-packages (from rsa<4.1,>=3.1.4->google-auth>=1.0.1->kubernetes->chaostoolkit-kubernetes) (0.4.8)
Installing collected packages: chaostoolkit-lib, chaostoolkit-kubernetes
Successfully installed chaostoolkit-kubernetes-0.1.0 chaostoolkit-lib-0.1.0
Can we be backward compatible with very old cluster that don't support apps/v1 endpoint for deployment?
see #65
Hi, we have been trying the probe count pods and faced inconsistency.
When we give a range as tolerance it is increasing the pod count and also the return of pod count is incorrect.
Another point phase check is also inconsistent, it does not recognise the running pods and prompting as pending even if they are running state when its checked with kubectl.
�[36m[2019-02-01 14:49:52 DEBUG]�[39m The Chaos Toolkit settings file could not be found at '/home/jakob/.chaostoolkit/settings.yaml'.
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Building activity cache...
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Cached 2 activities
�[32m[2019-02-01 14:49:52 INFO]�[39m Validating the experiment's syntax
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Loading configuration...
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Loading secrets...
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Secrets loaded
�[32m[2019-02-01 14:49:52 INFO]�[39m Experiment looks valid
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Clearing activities cache
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Building activity cache...
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Cached 2 activities
�[32m[2019-02-01 14:49:52 INFO]�[39m Running experiment: Test janie-nginx Resilience - at least one pod
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Loading configuration...
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Loading secrets...
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Secrets loaded
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Initializing controls
�[32m[2019-02-01 14:49:52 INFO]�[39m Steady state hypothesis: Prometheus running as expected
�[32m[2019-02-01 14:49:52 INFO]�[39m Probe: count_pods
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Activity 'count_pods' loaded from '/usr/lib/python3.7/site-packages/chaosk8s/pod/probes.py'
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Using Kubernetes context: default
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Found 2 pods matching label 'app=janie-nginx' in ns 'chaos'
�[36m[2019-02-01 14:49:52 DEBUG]�[39m => succeeded with '2'
�[36m[2019-02-01 14:49:52 DEBUG]�[39m allowed tolerance is [1, 3]
�[32m[2019-02-01 14:49:52 INFO]�[39m Steady state hypothesis is met!
�[32m[2019-02-01 14:49:52 INFO]�[39m Action: terminate_pods
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Activity 'terminate_pods' loaded from '/usr/lib/python3.7/site-packages/chaosk8s/pod/actions.py'
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Using Kubernetes context: default
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Found 2 pods labelled 'app=janie-nginx' in ns chaos
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Pod 'janie-nginx-5795fbf867-l6l4b' match pattern
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Pod 'janie-nginx-5795fbf867-vcv2p' match pattern
�[36m[2019-02-01 14:49:52 DEBUG]�[39m Picked pods 'janie-nginx-5795fbf867-l6l4b,janie-nginx-5795fbf867-vcv2p' to be terminated
�[36m[2019-02-01 14:49:52 DEBUG]�[39m => succeeded without any result value
�[32m[2019-02-01 14:49:52 INFO]�[39m Pausing after activity for 5s...
�[32m[2019-02-01 14:49:57 INFO]�[39m Steady state hypothesis: Prometheus running as expected
�[32m[2019-02-01 14:49:57 INFO]�[39m Probe: count_pods
�[36m[2019-02-01 14:49:57 DEBUG]�[39m Activity 'count_pods' loaded from '/usr/lib/python3.7/site-packages/chaosk8s/pod/probes.py'
�[36m[2019-02-01 14:49:57 DEBUG]�[39m Using Kubernetes context: default
�[36m[2019-02-01 14:49:58 DEBUG]�[39m Found 2 pods matching label 'app=janie-nginx' in ns 'chaos'
�[36m[2019-02-01 14:49:58 DEBUG]�[39m => succeeded with '2'
�[36m[2019-02-01 14:49:58 DEBUG]�[39m allowed tolerance is [1, 3]
�[32m[2019-02-01 14:49:58 INFO]�[39m Steady state hypothesis is met!
�[32m[2019-02-01 14:49:58 INFO]�[39m Let's rollback...
�[32m[2019-02-01 14:49:58 INFO]�[39m No declared rollbacks, let's move on.
�[32m[2019-02-01 14:49:58 INFO]�[39m Experiment ended with status: completed
�[36m[2019-02-01 14:49:58 DEBUG]�[39m Cleaning up controls
�[36m[2019-02-01 14:49:58 DEBUG]�[39m Clearing activities cache
�[36m[2019-02-01 14:51:00 DEBUG]�[39m ###############################################################################
�[36m[2019-02-01 14:51:00 DEBUG]�[39m Running command 'run'
�[36m[2019-02-01 14:51:00 DEBUG]�[39m Using settings file '/home/jakob/.chaostoolkit/settings.yaml'
�[33m[2019-02-01 14:51:01 WARNING]�[39m
There is a new version (1.0.0rc3) of the chaostoolkit available.
You may upgrade by typing:
$ pip install -U chaostoolkit
Please review changes at https://github.com/chaostoolkit/chaostoolkit/blob/master/CHANGELOG.md
�[36m[2019-02-01 14:51:01 DEBUG]�[39m The Chaos Toolkit settings file could not be found at '/home/jakob/.chaostoolkit/settings.yaml'.
�[36m[2019-02-01 14:51:01 DEBUG]�[39m Building activity cache...
�[36m[2019-02-01 14:51:01 DEBUG]�[39m Cached 2 activities
�[32m[2019-02-01 14:51:01 INFO]�[39m Validating the experiment's syntax
�[36m[2019-02-01 14:51:01 DEBUG]�[39m Loading configuration...
�[36m[2019-02-01 14:51:01 DEBUG]�[39m Loading secrets...
�[36m[2019-02-01 14:51:01 DEBUG]�[39m Secrets loaded
�[32m[2019-02-01 14:51:01 INFO]�[39m Experiment looks valid
�[36m[2019-02-01 14:51:01 DEBUG]�[39m Clearing activities cache
�[36m[2019-02-01 14:51:01 DEBUG]�[39m Building activity cache...
�[36m[2019-02-01 14:51:01 DEBUG]�[39m Cached 2 activities
�[32m[2019-02-01 14:51:01 INFO]�[39m Running experiment: Test janie-nginx Resilience - at least one pod
�[36m[2019-02-01 14:51:01 DEBUG]�[39m Loading configuration...
�[36m[2019-02-01 14:51:01 DEBUG]�[39m Loading secrets...
�[36m[2019-02-01 14:51:01 DEBUG]�[39m Secrets loaded
�[36m[2019-02-01 14:51:01 DEBUG]�[39m Initializing controls
�[32m[2019-02-01 14:51:01 INFO]�[39m Steady state hypothesis: Prometheus running as expected
�[32m[2019-02-01 14:51:01 INFO]�[39m Probe: count_pods
�[36m[2019-02-01 14:51:01 DEBUG]�[39m Activity 'count_pods' loaded from '/usr/lib/python3.7/site-packages/chaosk8s/pod/probes.py'
�[36m[2019-02-01 14:51:01 DEBUG]�[39m Using Kubernetes context: default
�[36m[2019-02-01 14:51:03 DEBUG]�[39m Found 2 pods matching label 'app=janie-nginx' in ns 'chaos'
�[36m[2019-02-01 14:51:03 DEBUG]�[39m => succeeded with '2'
�[36m[2019-02-01 14:51:03 DEBUG]�[39m allowed tolerance is [1, 2]
�[32m[2019-02-01 14:51:03 INFO]�[39m Steady state hypothesis is met!
�[32m[2019-02-01 14:51:03 INFO]�[39m Action: terminate_pods
�[36m[2019-02-01 14:51:03 DEBUG]�[39m Activity 'terminate_pods' loaded from '/usr/lib/python3.7/site-packages/chaosk8s/pod/actions.py'
�[36m[2019-02-01 14:51:03 DEBUG]�[39m Using Kubernetes context: default
�[36m[2019-02-01 14:51:03 DEBUG]�[39m Found 2 pods labelled 'app=janie-nginx' in ns chaos
�[36m[2019-02-01 14:51:03 DEBUG]�[39m Pod 'janie-nginx-5795fbf867-7wjdt' match pattern
�[36m[2019-02-01 14:51:03 DEBUG]�[39m Pod 'janie-nginx-5795fbf867-zkbvf' match pattern
�[36m[2019-02-01 14:51:03 DEBUG]�[39m Picked pods 'janie-nginx-5795fbf867-7wjdt,janie-nginx-5795fbf867-zkbvf' to be terminated
�[36m[2019-02-01 14:51:03 DEBUG]�[39m => succeeded without any result value
�[32m[2019-02-01 14:51:03 INFO]�[39m Pausing after activity for 10s...
�[32m[2019-02-01 14:51:13 INFO]�[39m Steady state hypothesis: Prometheus running as expected
�[32m[2019-02-01 14:51:13 INFO]�[39m Probe: count_pods
�[36m[2019-02-01 14:51:13 DEBUG]�[39m Activity 'count_pods' loaded from '/usr/lib/python3.7/site-packages/chaosk8s/pod/probes.py'
�[36m[2019-02-01 14:51:13 DEBUG]�[39m Using Kubernetes context: default
�[36m[2019-02-01 14:51:14 DEBUG]�[39m Found 4 pods matching label 'app=janie-nginx' in ns 'chaos'
�[36m[2019-02-01 14:51:14 DEBUG]�[39m => succeeded with '4'
�[36m[2019-02-01 14:51:14 DEBUG]�[39m allowed tolerance is [1, 2]
[2019-02-01 14:51:14 CRITICAL] Steady state probe 'count_pods' is not in the given tolerance so failing this experiment
�[32m[2019-02-01 14:51:14 INFO]�[39m Let's rollback...
�[32m[2019-02-01 14:51:14 INFO]�[39m No declared rollbacks, let's move on.
�[32m[2019-02-01 14:51:14 INFO]�[39m Experiment ended with status: deviated
�[32m[2019-02-01 14:51:14 INFO]�[39m The steady-state has deviated, a weakness may have been discovered
�[36m[2019-02-01 14:51:14 DEBUG]�[39m Cleaning up controls
�[36m[2019-02-01 14:51:14 DEBUG]�[39m Clearing activities cache
�[36m[2019-02-01 14:52:21 DEBUG]�[39m ###############################################################################
�[36m[2019-02-01 14:52:21 DEBUG]�[39m Running command 'run'
�[36m[2019-02-01 14:52:21 DEBUG]�[39m Using settings file '/home/jakob/.chaostoolkit/settings.yaml'
�[33m[2019-02-01 14:52:22 WARNING]�[39m
There is a new version (1.0.0rc3) of the chaostoolkit available.
You may upgrade by typing:
---------------____________________________------------------------------_____________________________
�[36m[2019-02-01 14:31:47 DEBUG]�[39m Activity 'pods_in_phase' loaded from '/usr/lib/python3.7/site-packages/chaosk8s/pod/probes.py'
�[36m[2019-02-01 14:31:47 DEBUG]�[39m Using Kubernetes context: default
�[36m[2019-02-01 14:31:47 DEBUG]�[39m Found 4 pods matching label 'app=janie-nginx' in ns 'chaos'
�[36m[2019-02-01 14:31:47 DEBUG]�[39m Activity failed
Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/chaoslib/provider/python.py", line 57, in run_python_activity
return func(**arguments)
File "/usr/lib/python3.7/site-packages/chaosk8s/pod/probes.py", line 105, in pods_in_phase
name=label_selector, s=d.status.phase, p=phase))
chaoslib.exceptions.ActivityFailed: pod 'app=janie-nginx' is in phase 'Pending' but should be 'Running'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/chaoslib/activity.py", line 224, in run_activity
result = run_python_activity(activity, configuration, secrets)
File "/usr/lib/python3.7/site-packages/chaoslib/provider/python.py", line 62, in run_python_activity
sys.exc_info()[2])
File "/usr/lib/python3.7/site-packages/chaoslib/provider/python.py", line 57, in run_python_activity
return func(**arguments)
File "/usr/lib/python3.7/site-packages/chaosk8s/pod/probes.py", line 105, in pods_in_phase
name=label_selector, s=d.status.phase, p=phase))
chaoslib.exceptions.ActivityFailed: chaoslib.exceptions.ActivityFailed: pod 'app=janie-nginx' is in phase 'Pending' but should be 'Running'
�[31m[2019-02-01 14:31:47 ERROR]�[39m => failed: chaoslib.exceptions.ActivityFailed: pod 'app=janie-nginx' is in phase 'Pending' but should be 'Running'
�[33m[2019-02-01 14:31:47 WARNING]�[39m Probe terminated unexpectedly, so its tolerance could not be validated
[2019-02-01 14:31:47 CRITICAL] Steady state probe 'pods_in_phase' is not in the given tolerance so failing this experiment
�[32m[2019-02-01 14:31:47 INFO]�[39m Let's rollback...
�[32m[2019-02-01 14:31:47 INFO]�[39m No declared rollbacks, let's move on.
�[32m[2019-02-01 14:31:47 INFO]�[39m Experiment ended with status: deviated
�[32m[2019-02-01 14:31:47 INFO]�[39m The steady-state has deviated, a weakness may have been discovered
�[36m[2019-02-01 14:31:47 DEBUG]�[39m Cleaning up controls
�[36m[2019-02-01 14:31:47 DEBUG]�[39m Clearing activities cache
openshift v3.7.42
kubernetes v1.7.6+a08f5eeb62
chaos, version 0.14.0
[2018-06-22 10:53:57 INFO] [activity:158] Action: scale_microservice
[2018-06-22 10:53:57 DEBUG] [activity:225] Activity failed
Reason: Internal Server Error
HTTP response headers: HTTPHeaderDict({'Content-Length': '117', 'Date': 'Fri, 22 Jun 2018 10:53:57 GMT', 'Content-Type': 'application/json', 'Cache-Control': 'no-store'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"unrecognized type: int32","code":500}
similar to #42
Code:
"type": "action",
"name": "stop-provider-service",
"provider": {
"type": "python",
"module": "chaosk8s.actions",
"func": "kill_microservice",
"arguments": {
"name": "my-provider-app"
}
Output:
[2019-08-30 09:40:44 ERROR] => failed: TypeError: delete_namespaced_deployment() takes 3 positional arguments but 4 were given
I got this code from the Sample github but the kill_microservice function does not seem to be working correctly.
I am seeing issues when I am trying to run experiment on micro service hosted on minikube .
I see this behavior for actions which accepts name
as mandatory parameter in the request:
Here is the link to my GitHub repository which host the code for sample spring boot micro service that I am deploying on local minikube and trying to run chaos experiments. All the probes are successful . Only experiments are failing.
Error when running chaos:
kubect service description:
The journal.json file generated as a result of random pod deletion experiment is missing details about the targeted pod.
When using terminate_pods and defining only a NS, it will also try to match the labes to none:
Config:
method:
- name: terminate-pods
type: action
provider:
func: terminate_pods
module: chaosk8s.pod.actions
type: python
arguments:
ns: 'myapp'
Error log:
[2018-10-05 03:05:15 DEBUG] Using Kubernetes context: Default
[2018-10-05 03:05:15 DEBUG] Found 0 pods labelled 'None'
[2018-10-05 03:05:15 DEBUG] Activity failed
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/chaoslib/provider/python.py", line 50, in run_python_activity
return func(**arguments)
File "/usr/local/lib/python3.5/dist-packages/chaosk8s/pod/actions.py", line 58, in terminate_pods
pods = [pods[0]]
IndexError: list index out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/chaoslib/activity.py", line 218, in run_activity
result = run_python_activity(activity, configuration, secrets)
File "/usr/local/lib/python3.5/dist-packages/chaoslib/provider/python.py", line 55, in run_python_activity
sys.exc_info()[2])
File "/usr/local/lib/python3.5/dist-packages/chaoslib/provider/python.py", line 50, in run_python_activity
return func(**arguments)
File "/usr/local/lib/python3.5/dist-packages/chaosk8s/pod/actions.py", line 58, in terminate_pods
pods = [pods[0]]
chaoslib.exceptions.ActivityFailed: IndexError: list index out of range
[2018-10-05 03:05:15 ERROR] => failed: IndexError: list index out of range
Setting a label_selector however, gets things to work.
That probe is probably better located in chaosk8s.pod.probes
.
the following probe fails with error:
Steady state probe 'fetch-ziptranslatorservice-zipservice-pod-logs' is not in the given tolerance so failing this experiment
{
"name": "fetch-ziptranslatorservice-zipservice-pod-logs",
"type": "probe",
"tolerance": true,
"provider": {
"func": "read_pod_logs",
"arguments": {
"label_selector": "name=zipservice",
"last": "30m",
"ns": "core"
},
"type": "python",
"module": "chaosk8s.pod.probes"
}
}
I work behind a corporate proxy with windows 10
So i use 3 env variables:
When i'm using chaostoolkit-kubernetes i cannot contact my minikube cluster
[2019-12-20 15:55:19 INFO] Validating the experiment's syntax
[2019-12-20 15:55:20 INFO] Experiment looks valid
[2019-12-20 15:55:20 INFO] Running experiment: Le pod MySQL KO ne doit pas perturber le service
[2019-12-20 15:55:20 INFO] Steady state hypothesis: Les APIs répondent
[2019-12-20 15:55:20 INFO] Probe: Swagger OK
[2019-12-20 15:55:20 INFO] Probe: GET /api/users OK
[2019-12-20 15:55:20 INFO] Probe: GET /api/users/{id} OK
[2019-12-20 15:55:20 INFO] Probe: GET /api/workstations OK
[2019-12-20 15:55:20 INFO] Probe: GET /api/workstations/{id} OK
[2019-12-20 15:55:20 INFO] Steady state hypothesis is met!
[2019-12-20 15:55:20 INFO] Action: Suppression du pod MYSQL
2019-12-20 15:55:23,855 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden'))': /apis/apps/v1/namespaces/default/deployments?labelSelector=module%3Dmysql
2019-12-20 15:55:23,911 WARNING Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden'))': /apis/apps/v1/namespaces/default/deployments?labelSelector=module%3Dmysql
2019-12-20 15:55:23,998 WARNING Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden'))': /apis/apps/v1/namespaces/default/deployments?labelSelector=module%3Dmysql
[2019-12-20 15:55:24 ERROR] => failed: urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='145.42.9.20', port=8443): Max retries exceeded with url: /apis/apps/v1/namespaces/default/deployments?labelSelector=module%3Dmysql (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden')))
[2019-12-20 15:55:24 INFO] Steady state hypothesis: Les APIs répondent
[2019-12-20 15:55:24 INFO] Probe: Swagger OK
[2019-12-20 15:55:24 INFO] Probe: GET /api/users OK
[2019-12-20 15:55:24 INFO] Probe: GET /api/users/{id} OK
[2019-12-20 15:55:24 INFO] Probe: GET /api/workstations OK
[2019-12-20 15:55:24 INFO] Probe: GET /api/workstations/{id} OK
[2019-12-20 15:55:24 INFO] Steady state hypothesis is met!
[2019-12-20 15:55:24 INFO] Let's rollback...
[2019-12-20 15:55:24 INFO] No declared rollbacks, let's move on.
[2019-12-20 15:55:24 INFO] Experiment ended with status: completed
The only way to make this work is too remove HTTP_PROXY env variable
But I need it for other use cases
Is it possible to have NO_PROXY en variable support inchaostoolkit-kubernetes ?
I tried to make it myself but i'm newbie in Python
I am getting this error:
TypeError: delete_namespaced_pod() takes 3 positional arguments but 4 were given
when trying to kill a pod with latest chaostolkit-kubernetes ..
here is the json:
{
"version": "1.0.0",
"title": "System is resilient to provider's failures",
"description": "Can our consumer survive gracefully a provider's failure?",
"method": [
{
"type": "probe",
"name": "count-pods-before-termination",
"provider": {
"type": "python",
"module": "chaosk8s.pod.probes",
"func": "count_pods",
"arguments": {
"label_selector": "app=nginx",
"ns": "ns1"
}
},
"pauses": {
"after": 10
}
},
{
"type": "action",
"name": "Terminate_pod",
"provider": {
"type": "python",
"module": "chaosk8s.pod.actions",
"func": "terminate_pods",
"arguments": {
"label_selector": "app=nginx",
"ns": "ns1"
}
},
"pauses": {
"after": 2
}
},
{
"type": "probe",
"name": "count-pods-after-termintation",
"provider": {
"type": "python",
"module": "chaosk8s.pod.probes",
"func": "count_pods",
"arguments": {
"label_selector": "app=nginx",
"ns": "ns1"
}
},
"pauses": {
"after": 10
}
}
],
"rollbacks": [
]
}
Is it possible to ignore .idea folder ?
I use Intellij and i need to ignore thoses files every times i create a branch on chaostoolkit-kubernetes project
Do i need to create a pull request for that ?
In the documentation it states look for deployment by "name" but when I give the deployment name it always fails stating microservice not found.
I want an action to activate port-forwarding to a service or pod during the experiment. This will allow me to hit private endpoints that are not exposed. Currently, I manually set up the port forwarding using kubectl before the experiment begins. https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/
I would like to add it as an action.
I tried to use the terminate_pods
action, but without enough permissions on my K8s cluster.
However, the experiment did go till the end with "completed" status.
Is it intended behavior when a function breaks that the CTK continues other methods ?
I would have thought to have a failure at some point
[2020-10-01 12:54:06 INFO] Action: terminate-pods
[2020-10-01 12:54:06 ERROR] => failed: kubernetes.client.rest.ApiException: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Audit-Id': '1c05c067-58fc-4d95-ae0b-240ed4c3709d', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'Date': 'Thu, 01 Oct 2020 12:54:06 GMT', 'Content-Length': '310'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User \"system:serviceaccount:chaostoolkit-run:chaostoolkit-bsm9b\" cannot list resource \"pods\" in API group \"\" in the namespace \"chaosiq-console\"","reason":"Forbidden","details":{"kind":"pods"},"code":403}
[2020-10-01 12:54:06 INFO] Pausing after activity for 2s...
[2020-10-01 12:54:11 INFO] Steady state hypothesis: Application is normal
[2020-10-01 12:54:13 INFO] Probe: application-must-respond-normally
[2020-10-01 12:54:15 INFO] Steady state hypothesis is met!
[2020-10-01 12:54:16 INFO] Let's rollback...
[2020-10-01 12:54:18 INFO] No declared rollbacks, let's move on.
[2020-10-01 12:54:19 INFO] Experiment ended with status: completed
Is it possible to have multiple iteration of single action in an experiment?
Say I want to run terminate-db-pod several times after each 600 sec.
{ "name": "all-our-microservices-should-be-healthy", "type": "probe", "tolerance": "true", "provider": { "type": "python", "module": "chaosk8s.probes", "func": "microservice_available_and_healthy", "arguments": { "name": "myapp", "ns": "myns" } } }, { "type": "action", "name": "terminate-db-pod", "provider": { "type": "python", "module": "chaosk8s.pod.actions", "func": "terminate_pods", "arguments": { "label_selector": "app=my-app", "name_pattern": "my-app-[0-9]$", "rand": true, "ns": "default" } }, "pauses": { "after": 5 } }
Hi
Im chaostoolkit-lib 0.21.0. Python 3.6.0.
Last week we changed our auth method in kubernetes. We use jwt tokens now.
There is known issue for python and oidc tokens
For now ive disabled using of kube configuration and authenticate with plain token only.
But this situation have to be addressed somehow.
$ chaos discover chaostoolkit-kubernetes --no-install
Traceback (most recent call last):
File "/usr/bin/chaos", line 6, in
from chaostoolkit.main import cli
File "/usr/lib/python2.7/site-packages/chaostoolkit/main.py", line 20
def run(report_path: str, dry: bool, path: str):
^
SyntaxError: invalid syntax
First, I've successfully installed chaostoolkit and chaostoolkit-kubernetes from pip in my venv :
⇒ python3 -m venv chaostk
⇒ source chaostk/bin/activate
⇒ pip install chaostoolkit
⇒ pip install chaostoolkit-kubernetes
I am trying to use chaostoolkit-kubernetes on my k8s cluster, but I always get the following errors :
⇒ which chaos
~/venv/chaostk/bin/chaos
⇒ chaos --verbose discover chaostoolkit-kubernetes
[2020-10-28 11:05:33 DEBUG] [cli:74] ###############################################################################
[2020-10-28 11:05:33 DEBUG] [cli:75] Running command 'discover'
[2020-10-28 11:05:33 DEBUG] [cli:79] Using settings file '/home/xat/.chaostoolkit/settings.yaml'
[2020-10-28 11:05:33 DEBUG] [settings:26] The Chaos Toolkit settings file could not be found at '/home/xat/.chaostoolkit/settings.yaml'.
[2020-10-28 11:05:33 INFO] [package:22] Attempting to download and install package 'chaostoolkit-kubernetes'
[2020-10-28 11:05:34 DEBUG] [package:31] Requirement already up-to-date: chaostoolkit-kubernetes in /home/xat/venv/chaostk/lib/python3.8/site-packages (0.24.0)
Requirement already satisfied, skipping upgrade: pyyaml in /home/xat/venv/chaostk/lib/python3.8/site-packages (from chaostoolkit-kubernetes) (5.3.1)
Requirement already satisfied, skipping upgrade: chaostoolkit-lib>=0.20.0 in /home/xat/venv/chaostk/lib/python3.8/site-packages (from chaostoolkit-kubernetes) (1.15.0)
Requirement already satisfied, skipping upgrade: dateparser in /home/xat/venv/chaostk/lib/python3.8/site-packages (from chaostoolkit-kubernetes) (0.7.6)
Requirement already satisfied, skipping upgrade: kubernetes in /home/xat/venv/chaostk/lib/python3.8/site-packages (from chaostoolkit-kubernetes) (12.0.0)
Requirement already satisfied, skipping upgrade: logzero in /home/xat/venv/chaostk/lib/python3.8/site-packages (from chaostoolkit-kubernetes) (1.5.0)
Requirement already satisfied, skipping upgrade: requests>=2.21 in /home/xat/venv/chaostk/lib/python3.8/site-packages (from chaostoolkit-lib>=0.20.0->chaostoolkit-kubernetes) (2.24.0)
Requirement already satisfied, skipping upgrade: tzlocal in /home/xat/venv/chaostk/lib/python3.8/site-packages (from dateparser->chaostoolkit-kubernetes) (2.1)
Requirement already satisfied, skipping upgrade: regex!=2019.02.19 in /home/xat/venv/chaostk/lib/python3.8/site-packages (from dateparser->chaostoolkit-kubernetes) (2020.10.23)
Requirement already satisfied, skipping upgrade: pytz in /home/xat/venv/chaostk/lib/python3.8/site-packages (from dateparser->chaostoolkit-kubernetes) (2020.1)
Requirement already satisfied, skipping upgrade: python-dateutil in /home/xat/venv/chaostk/lib/python3.8/site-packages (from dateparser->chaostoolkit-kubernetes) (2.8.1)
Requirement already satisfied, skipping upgrade: certifi>=14.05.14 in /home/xat/venv/chaostk/lib/python3.8/site-packages (from kubernetes->chaostoolkit-kubernetes) (2020.6.20)
Requirement already satisfied, skipping upgrade: google-auth>=1.0.1 in /home/xat/venv/chaostk/lib/python3.8/site-packages (from kubernetes->chaostoolkit-kubernetes) (1.22.1)
Requirement already satisfied, skipping upgrade: websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 in /home/xat/venv/chaostk/lib/python3.8/site-packages (from kubernetes->chaostoolkit-kubernetes) (0.57.0)
Requirement already satisfied, skipping upgrade: requests-oauthlib in /home/xat/venv/chaostk/lib/python3.8/site-packages (from kubernetes->chaostoolkit-kubernetes) (1.3.0)
Requirement already satisfied, skipping upgrade: setuptools>=21.0.0 in /home/xat/venv/chaostk/lib/python3.8/site-packages (from kubernetes->chaostoolkit-kubernetes) (41.6.0)
Requirement already satisfied, skipping upgrade: urllib3>=1.24.2 in /home/xat/venv/chaostk/lib/python3.8/site-packages (from kubernetes->chaostoolkit-kubernetes) (1.25.11)
Requirement already satisfied, skipping upgrade: six>=1.9.0 in /home/xat/venv/chaostk/lib/python3.8/site-packages (from kubernetes->chaostoolkit-kubernetes) (1.15.0)
Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in /home/xat/venv/chaostk/lib/python3.8/site-packages (from requests>=2.21->chaostoolkit-lib>=0.20.0->chaostoolkit-kubernetes) (2.10)
Requirement already satisfied, skipping upgrade: chardet<4,>=3.0.2 in /home/xat/venv/chaostk/lib/python3.8/site-packages (from requests>=2.21->chaostoolkit-lib>=0.20.0->chaostoolkit-kubernetes) (3.0.4)
Requirement already satisfied, skipping upgrade: pyasn1-modules>=0.2.1 in /home/xat/venv/chaostk/lib/python3.8/site-packages (from google-auth>=1.0.1->kubernetes->chaostoolkit-kubernetes) (0.2.8)
Requirement already satisfied, skipping upgrade: cachetools<5.0,>=2.0.0 in /home/xat/venv/chaostk/lib/python3.8/site-packages (from google-auth>=1.0.1->kubernetes->chaostoolkit-kubernetes) (4.1.1)
Requirement already satisfied, skipping upgrade: rsa<5,>=3.1.4; python_version >= "3.5" in /home/xat/venv/chaostk/lib/python3.8/site-packages (from google-auth>=1.0.1->kubernetes->chaostoolkit-kubernetes) (4.6)
Requirement already satisfied, skipping upgrade: oauthlib>=3.0.0 in /home/xat/venv/chaostk/lib/python3.8/site-packages (from requests-oauthlib->kubernetes->chaostoolkit-kubernetes) (3.1.0)
Requirement already satisfied, skipping upgrade: pyasn1<0.5.0,>=0.4.6 in /home/xat/venv/chaostk/lib/python3.8/site-packages (from pyasn1-modules>=0.2.1->google-auth>=1.0.1->kubernetes->chaostoolkit-kubernetes) (0.4.8)
[2020-10-28 11:05:34 INFO] [package:40] Package downloaded and installed in current environment
[2020-10-28 11:05:34 DEBUG] [cli:444] Failed to discover chaostoolkit-kubernetes
Traceback (most recent call last):
File "/home/xat/.local/lib/python3.8/site-packages/chaoslib/discovery/package.py", line 99, in get_importname_from_package
dist = importlib_metadata.distribution(package_name)
File "/usr/lib64/python3.8/importlib/metadata.py", line 504, in distribution
return Distribution.from_name(distribution_name)
File "/usr/lib64/python3.8/importlib/metadata.py", line 177, in from_name
raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: chaostoolkit-kubernetes
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/xat/.local/lib/python3.8/site-packages/chaostoolkit/cli.py", line 439, in discover
discovery = disco(
File "/home/xat/.local/lib/python3.8/site-packages/chaoslib/discovery/discover.py", line 33, in discover
package = load_package(package_name)
File "/home/xat/.local/lib/python3.8/site-packages/chaoslib/discovery/package.py", line 47, in load_package
name = get_importname_from_package(package_name)
File "/home/xat/.local/lib/python3.8/site-packages/chaoslib/discovery/package.py", line 101, in get_importname_from_package
raise DiscoveryFailed(
chaoslib.exceptions.DiscoveryFailed: Package chaostoolkit-kubernetes not found
[2020-10-28 11:05:34 CRITICAL] [cli:445] Package chaostoolkit-kubernetes not found
I also tried to run chaos discover chaostoolkit-aws
that works well.
Thanks for the support.
Steps:
pip install chaostoolkit-kubernetes
command on my local windows machineOutput:
[2020-09-01 18:52:53 INFO] Validating the experiment's syntax
[2020-09-01 18:52:53 INFO] Experiment looks valid
[2020-09-01 18:52:53 INFO] Running experiment: Do we remain available in face of pod going down?
[2020-09-01 18:52:53 INFO] Steady state hypothesis: Verifying service remains healthy
[2020-09-01 18:52:53 INFO] Probe: all-our-microservices-should-be-healthy
[2020-09-01 18:52:57 ERROR] => failed: urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='xxxxxxxx', port=443): Max retries exceeded with url: /k8s/clusters/xxxxx/apis/apps/v1/namespaces/xxxxx/deployments?fieldSelector=metadata.name%3Dxxxxx (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)')))
[2020-09-01 18:52:57 WARNING] Probe terminated unexpectedly, so its tolerance could not be validated
[2020-09-01 18:52:57 CRITICAL] Steady state probe 'all-our-microservices-should-be-healthy' is not in the given tolerance so failing this experiment
[2020-09-01 18:52:57 INFO] Experiment ended with status: failed
Can someone help what could be missing?
Hello Team,
I have 2 services running inside K8s but they are not exposed to the outer world through ingress/loadbalancer. We are using it inside K8s with gRPC. Is there a way to test the service endpoints for those services or "microservice_available_and_healthy" is enough?
Currently, the way resources like pods and services are selected is via an hardcoded label_selector. It is too restrictive in most cases. Make the label_selector an argument for the activities.
The discussion started here. This issue is here to share thoughts and plan a small rework to clean up the repository.
The plugin is actually a bit messy because of the background. The most annoying things (making it hard for a user to understand what to use and when) is that, at the beginning of the project, some actions and probes meant to be high-level. This is also why the project still contains unnamespaced actions and probes. Also, as I mentioned in the PR linked above, some functions have the same prefix but don't work with the same resources (cf. microservice_available_and_healthy
function which looks for a deployment vs. microservice_is_not_available
which looks for pods). Finally, some of them may be a bit redundant.
As you can see, the actions and probes included recently are namespaced per resource (pod
and node
actually).
Here are two things I think we should do for now:
microservice_
prefix anymore since it does not relate to a Kubernetes resource)We should still keep backward compatibility with existing functions and mark them as deprecated to remove them in a next release.
Hi,
Feel free to reject or accept my read me doc update recommendation.
if you are hitting from outside of cluster. download kubernetes config file and place in your home folder in the below format in Windows -> C:\Users\username\.kube\config, Chaos toolkit automatically picks up from this folder
More Detailed Notes for GOOGLE_APPLICATION_CREDENTIALS
Create Google Cloud Service Account and download the json file and place it in the following format for google cloud.
open command prompt in windows and enter the following command
set GOOGLE_APPLICATION_CREDENTIALS="C:\Users\username\micro-primer-####.json
chaosk8s/__init__.py
line 91 has:
configuration.cert_file = lookup("KUBERNETES_CA_CERT_FILE")
and
chaosk8s/__init__.py
line 100 - 101 have:
configuration.cert_file = lookup("KUBERNETES_CERT_FILE")
configuration.key_file = lookup("KUBERNETES_KEY_FILE")
I believe line 91 is a bug since Kube client configuration takes ssl_ca_cert
for CA cert.
PR #93 should fix it.
Could maintainers take a look at it, please?
Thanks!
It would be interesting to add grace_period_seconds
paramater to the V1DeleteOptions at chaosk8s/pod/actions.py so we can choose whether we want to "kill" a pod (terminate it as soon as possible, i.e. grace_period_seconds=0) instead of gracefully terminating it.
{
"name": "restart-node",
"type": "action",
"provider": {
"func": "restart_node",
"type": "python",
"module": "chaosazure.aks.actions"
}
}
Will the above action restart all the nodes in the given AKS?
Looks like exception is thrown when using the CHAOSTOOLKIT_IN_POD
environment variable in the kubernetes:
stack trace from the logs:
Traceback (most recent call last):
File "/usr/local/bin/chaos", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/chaostoolkit/cli.py", line 106, in run
experiment = load_experiment(
File "/usr/local/lib/python3.8/site-packages/chaoslib/loader.py", line 96, in load_experiment
parsed = parse_experiment_from_file(experiment_source)
File "/usr/local/lib/python3.8/site-packages/chaoslib/loader.py", line 38, in parse_experiment_from_file
return json.load(f)
File "/usr/local/lib/python3.8/json/__init__.py", line 293, in load
return loads(fp.read(),
File "/usr/local/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.8/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 8 column 3 (char 213)
Using the option to connect to the kubernetes cluster via local configuation file does not set a proxy. I have a working example below:
import os
from kubernetes import client, config
from kubernetes.client import Configuration
# Configs can be set in Configuration class directly or using helper utility
proxy_url = os.getenv('HTTP_PROXY', None)
config.load_kube_config()
if proxy_url:
client.Configuration._default.proxy = proxy_url
v1 = client.CoreV1Api()
print("Listing pods with their IPs:")
ret = v1.list_pod_for_all_namespaces(watch=False)
for i in ret.items:
print("%s\t%s\t%s" % (i.status.pod_ip, i.metadata.namespace, i.metadata.name))
This looks similar to #34
It appears actions.py
has several incorrect instances of body
as an argument.
If I edit to read body=body
as in here for kill_microservice
: https://github.com/chaostoolkit/chaostoolkit-kubernetes/blob/master/chaosk8s/actions.py#L67
The deployment will correctly be deleted. Without that edit, I will get the following:
[2019-06-19 17:32:43 ERROR] => failed: TypeError: delete_namespaced_deployment() takes 3 positional arguments but 4 were given
And the function will not run, no deployment is deleted.
This also appears two other places:
https://github.com/chaostoolkit/chaostoolkit-kubernetes/blob/master/chaosk8s/actions.py#L81
https://github.com/chaostoolkit/chaostoolkit-kubernetes/blob/master/chaosk8s/actions.py#L95
Hello,
I wanted to check logs returned by read_pod_logs with regex but getting the error: "return rx.search(value) is not None".
My probe was similar to:
{
"name": "read-microservices-logs",
"type": "probe",
"tolerance": {
"type": "regex,",
"pattern": "UP"
},
"provider": {
"func": "read_pod_logs",
"arguments": {
"label_selector": "app.kubernetes.io/name=ui",
"ns": "dev",
"container_name": "ui"
},
"module": "chaosk8s.probes",
"type": "python"
}
}
But got an error:
File "/opt/hostedtoolcache/Python/3.8.1/x64/lib/python3.8/site-packages/chaoslib/hypothesis.py", line 304, in _ return rx.search(value) is not None TypeError: expected string or bytes-like object
Can it be caused by function returning dictionary instead of string?
When I run the experiment to dremove service end point I receive the error below:
"TypeError: delete_namespaced_service() missing 1 required positional argument: 'body'\n",
when I check the actions code no "body" argument requested for the delete_namespaced_service
Hi,
I use my ~/.kube config to run my chaos test, this is my probe:
{
name: am-is-healty,
type: probe,
"tolerance": "0",
provider: {
type: python,
module: chaosk8s.probes,
func: microservice_available_and_healthy,
secrets: [kubernetes],
arguments: {
name: am-6c7b66b4c8-4wmph,
ns: default
}
}
}
And this is the error which I'm unable to figure out:
[2019-03-25 13:59:34 DEBUG] [__init__:69] Using Kubernetes context: default
2019-03-25 13:59:50,717 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f66e109ecf8>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /apis/apps/v1beta1/namespaces/default/deployments?labelSelector=name+in+%28am-6c7b66b4c8-4wmph%29
Sorry, OCD. Can't PR that.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.