rockem / busypie Goto Github PK

Expressive busy-waiting for Python

License: Apache License 2.0

Python 100.00%

busypie's Introduction

Easy and expressive busy-waiting for Python

Although you wouldn't want to do much busy waiting in your production code, testing is a different matter. When testing asynchronous systems, it's very helpful to wait for some scenario to finish its course. busypie helps you perform busy waiting easily and expressively.

Quickstart

Most typical usage will be in test, when we have a scenario that require us to wait for something to happen.

from busypie import wait, SECOND

def test_create_user():
    create_user_from(USER_DETAILS)
    wait().at_most(2, SECOND).until(lambda: is_user_exists(USER_DETAILS))

Documentation

Links

This project drew a lot of inspiration from Awaitility.

busypie's People

Contributors

Stargazers

Watchers

Forkers

innadav amoshason dejona perlun jbaldwin-vizio ppopov212 amaziahub

busypie's Issues

Allow to add a custom message for timeout failure

Allow to specify initial delay before polling starts

For example:
with().poll_delay(100, MILLISECOND).until(job_is_done)

Support in Pydoc

Allow to define minimum time for busy waiting

In case the condition is fulfilled before the duration specified by 'at_least', an exception is thrown indicating that the condition shouldn't be completed earlier than the specified duration.
Example:
wait().at_least(5 * SECONDS).during(validation_operation)

Allow to optionally ignore all exceptions

Add option to return a False result instead of throwing an exception.

It would be nice if there was an option to also return a False result instead of throwing an exception.

Something like

wait().at_most(1, SECOND, exception_on_timeout=False).until(condition)

pytest assert failure message to "bubble up" to busypie

Hi,

Thanks for a nice library. I discovered it by googling something like "python awaitility", since I've been using awaitility in the Java world previously.

It all works fairly well, but I wonder if there's any way you can get your assertion failures to "bubble up" and be displayed by busypie when a busy-wait eventually fails? Here's my use case:

def test_tomcat_has_expected_executor_threads(host):
    wait_at_most(TOMCAT_STARTUP_WAIT_TIME).ignore_exceptions().until(lambda:
        tomcat_has_expected_executor_threads_helper(host)
    )


def tomcat_has_expected_executor_threads_helper(host):
    pid = tomcat_pid(host)
    user = tomcat_user(host)

    cmd = host.run(f"sudo -u %s jstack %s | grep http-nio-80-exec-", user, pid)

    stdout_lines = cmd.stdout.rstrip().split("\n")

    assert cmd.stderr == ''
    assert len(stdout_lines) == 13

    # stdout is expected to contain one line for each thread from 1 to 13:
    # http-nio-80-exec-1, http-nio-80-exec-2, ... We assert the first and last
    # entries for simplicity.
    assert len(list(filter(lambda s: ('"http-nio-80-exec-1"' in s), stdout_lines))) == 1
    assert len(list(filter(lambda s: ('"http-nio-80-exec-13"' in s), stdout_lines))) == 1

    assert cmd.rc == 0

    return True

(The test ensures that some custom Tomcat executor settings are being applied properly; this is executed with testinfra towards an LXD container.)

When I run this with busypie, it fails like this which isn't so helpful to the reader:

    def test_tomcat_has_expected_executor_threads(host):
>       wait_at_most(TOMCAT_STARTUP_WAIT_TIME).ignore_exceptions().until(lambda:
            tomcat_has_expected_executor_threads_helper(host)
        )

unit_tests/test_set_up_isp_hbx_node.py:140: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../.local/lib/python3.9/site-packages/busypie/condition.py:48: in until
    return runner.run(self._wait_for(func, lambda f: f()))
../../.local/lib/python3.9/site-packages/busypie/runner.py:31: in run
    raise thread.exception
../../.local/lib/python3.9/site-packages/busypie/runner.py:17: in run
    self.result = runners.run(self.main, debug=self.debug)
../../.local/lib/python3.9/site-packages/backports/asyncio/runners.py:42: in run
    return loop.run_until_complete(main)
/usr/lib/python3.9/asyncio/base_events.py:642: in run_until_complete
    return future.result()
../../.local/lib/python3.9/site-packages/busypie/awaiter.py:29: in wait_for
    self._validate_wait_constraint(func, start_time)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <busypie.awaiter.AsyncConditionAwaiter object at 0x7efc4a116ee0>, condition_func = <function test_tomcat_has_expected_executor_threads.<locals>.<lambda> at 0x7efc4aeec8b0>, start_time = 1631876011.6542747

    def _validate_wait_constraint(self, condition_func, start_time):
        if (time.time() - start_time) > self._condition.wait_time_in_secs:
>           raise busypie.ConditionTimeoutError(
                self._condition.description or describe(condition_func), self._condition.wait_time_in_secs)
E           busypie.awaiter.ConditionTimeoutError: Failed to meet condition of [tomcat_has_expected_executor_threads_helper(host)] within 1 seconds

../../.local/lib/python3.9/site-packages/busypie/awaiter.py:46: ConditionTimeoutError

When running this without any busypie waiting, it's much easier to debug the actual error (AssertionError in common.py):

============================================================================================================================ FAILURES ============================================================================================================================
______________________________________________________________________________________________ test_tomcat_has_expected_executor_threads[paramiko://192.168.97.187] ______________________________________________________________________________________________

host = <testinfra.host.Host paramiko://192.168.97.187>

    def test_tomcat_has_expected_executor_threads(host):
        #wait_at_most(TOMCAT_STARTUP_WAIT_TIME).ignore_exceptions().until(lambda:
>           tomcat_has_expected_executor_threads_helper(host)

unit_tests/test_set_up_isp_hbx_node.py:141: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
unit_tests/test_set_up_isp_hbx_node.py:146: in tomcat_has_expected_executor_threads_helper
    pid = tomcat_pid(host)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

host = <testinfra.host.Host paramiko://192.168.97.187>

    def tomcat_pid(host):
        cmd = host.run('pgrep -f "org.apache.catalina.startup.Bootstrap"')
    
        assert cmd.stderr == ''
>       assert cmd.rc == 0
E       AssertionError

unit_tests/common.py:37: AssertionError

Are the any ways to accomplish something like this with busypie? I.e. get the assertion failure exceptions to "bubble up" and be displayed by busypie in case it times out?

Something like: "ignore exceptions but only until the very last time". 🙂 Thanks for your time!

Allow to change default polling interval

For example:
with().poll_interval(ONE_SECOND).wait().until(job_is_done)

There is no current event loop in thread 'MainThread'.` error

[2020/07/06 16:19:46.494] [ERROR] Exception in main context.
Traceback (most recent call last):
  File "/home/csr_user/csr_e2e_tests/modules/logging/logging.py", line 7, in wrapper
    return func(*args, **kwargs)
  File "features/steps/insert_catalog.py", line 10, in step_impl
    assert catalog_updated(config["General"]["Base"], barcodes)
  File "/home/csr_user/csr_e2e_tests/modules/catalog.py", line 37, in catalog_updated
    wait().until(lambda: are_all_existing_barcodes_equal_to(base_ip, barcodes))
  File "/home/csr_user/csr_e2e_tests/venv/lib/python3.8/site-packages/busypie/condition.py", line 48, in until
    return asyncio.get_event_loop().run_until_complete(self._wait_for(func, lambda f: f()))
  File "/usr/lib/python3.8/asyncio/events.py", line 639, in get_event_loop
    raise RuntimeError('There is no current event loop in thread %r.'
    And I insert catalog                                                    # modules/logging/logging.py:5 0.615s
      Traceback (most recent call last):
        File "/home/csr_user/csr_e2e_tests/venv/lib/python3.8/site-packages/behave/model.py", line 1329, in run
          match.run(runner.context)
        File "/home/csr_user/csr_e2e_tests/venv/lib/python3.8/site-packages/behave/matchers.py", line 98, in run
          self.func(context, *args, **kwargs)
        File "/home/csr_user/csr_e2e_tests/modules/logging/logging.py", line 7, in wrapper
          return func(*args, **kwargs)
        File "features/steps/insert_catalog.py", line 10, in step_impl
          assert catalog_updated(config["General"]["Base"], barcodes)
        File "/home/csr_user/csr_e2e_tests/modules/catalog.py", line 37, in catalog_updated
          wait().until(lambda: are_all_existing_barcodes_equal_to(base_ip, barcodes))
        File "/home/csr_user/csr_e2e_tests/venv/lib/python3.8/site-packages/busypie/condition.py", line 48, in until
          return asyncio.get_event_loop().run_until_complete(self._wait_for(func, lambda f: f()))
        File "/usr/lib/python3.8/asyncio/events.py", line 639, in get_event_loop
          raise RuntimeError('There is no current event loop in thread %r.'
      RuntimeError: There is no current event loop in thread 'MainThread'.

Can't run nested waits

Something to consider
https://stackoverflow.com/questions/55409641/asyncio-run-cannot-be-called-from-a-running-event-loop

Allow to ignore specific exceptions

For example:
wait().ignore_exceptions(ZeroDivisionError, AttributeError).until(job_is_done)

Add support for PyHamcrest

Allow to use Pyhamcrest matcher for condition as they do here: https://github.com/awaitility/awaitility/wiki/Usage#reusing-conditions

Pyhamcrest: https://github.com/hamcrest/PyHamcrest

Add type annotations

It would be nice to have type annotations for at least the public interface, for better static code analysis and IDE support ;)

Create Release

It would be great to have a new release - especially so that the linked documentation matches the PyPI release again :)

Add support for async waiting

Allow to wait for an assertion to pass

Allow to retrieve condition function result

Sometimes I want to do that:

message = wait().until(wait_for_message)
assert message.id = '234'

Allow to change default timeout

Log each retry attempt

Having following code:

import random

from busypie import wait, MINUTE, SECOND


def get_random_number():
    number_random = random.randint(1, 10)
    return number_random


wait().ignore_exceptions().at_most(2 * MINUTE).poll_interval(1, SECOND).until(lambda: get_random_number() == 5)

It would be great if busypie has ability to log each retry.
For example, having something like this:
wait().ignore_exceptions().at_most(2 * MINUTE).poll_interval(1, SECOND).until(lambda: get_random_number() == 5).with_message('Retrying number')

As a result, in the logs would be:
'Retrying number'

In the Awaitility library there is such option as alias:
https://github.com/awaitility/awaitility/wiki/Usage#advanced

Current behaviour

When using ignore_exception() feature with until_asserted, we get this error:

Traceback (most recent call last):
  File "/Users/elisegal/Source/busypie/tests/spike.py", line 14, in <module>
    wait().ignore_exceptions(ZeroDivisionError).at_most(100, MILLISECOND).until_asserted(_failed_assertion)
  File "/Users/elisegal/Source/busypie/busypie/condition.py", line 70, in until_asserted
    self._condition.append_exception(AssertionError)
  File "/Users/elisegal/Source/busypie/busypie/condition.py", line 98, in append_exception
    self.ignored_exceptions.append(exception)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'tuple' object has no attribute 'append'

Expected behaviour

ignore_exceptions should work the same as with until & during