Giter Club home page Giter Club logo

rxpy-backpressure's Introduction

rxpy-backpressure

Back pressure strategies for use with RxPy

Goals

This projects goal is to provide a simple api that allows the addition of backpressure strategies with RxPy.

Getting Started

To get started pull the code from pypi with a pip install rxpy_backpressure the project can be found here

API Examples

Example of usage of the API with rxpy 1.6

from rxpy_backpressure import BackPressure

image_observable
    .observe_on(scheduler)
    .subscribe(BackPressure.LATEST(observer))

Example of usage of the API with rxpy ~3

from rxpy_backpressure import BackPressure

image_observable
    .subscribe(BackPressure.LATEST(observer), scheduler)

If you would like to quickly get started with the library a runnable example can be found here rxpy-backpressure-example

Strategies

Latest

Latest strategy will remember the next most recent message to process and will call the observer with it when the observer has finished processing its current message.

Drop

Drop strategy accepts a cache_size, the strategy will remember the most recent messages and remove older messages from the cache. The strategy guarantees that the oldest messages in the cache are passed to the observer first.

Buffer

Buffer strategy has a unbounded cache and will pass all messages to its consumer in the order it received them beware of Memory leaks due to a build up of messages.

Sized Buffer

Sized buffer has a fix sized cache, the strategy will perform opposite of Drop and will refuse new messages as long as the buffer is full and will accept them only once the buffer has available space.

Considerations

This project supports python 3.6+ as rxpy 3.0.0+ does not support earlier versions. If there is a case to support previous versions this can be discussed however the typing introduced in python 3.6 is very nice.

rxpy-backpressure's People

Contributors

daliclass avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

rxpy-backpressure's Issues

Report Flaky Test on test_sized_buffer_backpressure_strategy.py::TestDropBackPressureStrategy::test_when_on_next_buffer_following_messages

This issue aims to point out the flaky test on test_sized_buffer_backpressure_strategy.py::TestDropBackPressureStrategy::test_when_on_next_buffer_following_messages in case if the developer would like to improve the testing file. I understand this is a relatively old project but I'm doing this for a course project as a practice.

The result

The test pass sometime on local IDE but continually fail on a virtual machine, the error message is the following:
> self.assertEqual(4, buffer.counter.get_stats().get("successful_events")) E AssertionError: 4 != 12
This result is in my VM, my local IDE gives 8 successful_events instead of 4

Steps to reproduce the issue

  1. re-run the test manually in local IDE or in the VM gives changed result

Issue of the code

The reason I suspect is the different machine/environment has different execution time that affects the buffer when locking and on_next.

Proposed solution

Maybe trying to call/on_next one by one could reduce the flaky test since multiple calls would increase the chance of various results in different platforms/environment

back pressure operator

Hi, thanks for creating this library which takes the job of the missing part in RxPY. I wonder whether these back pressure wrapper can be applied to the operator chain? There are some cases that the pipeline is actually limiting the whole process.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.