ets-labs / newsfeed Goto Github PK

Idea of this issue is to create event output microservice (following microservices splitting chart).

Acceptance criteria

• Service is splitted out.
• Integration tests pass.

Idea of this issue is to create subscription microservice (following microservices splitting chart).

Acceptance criteria

• Service is splitted out.
• Integration tests pass.

Refactoring is needed for Subscription entity and related functionality.

Acceptance criteria:

• Attribute Subscription.from_newsfeed_id should be renamed to Subscription.newsfeed_id.

Idea of this issue is to implement subscription storage provider using Redis.

Acceptance criteria

• Provider is implemented.
• Integration tests pass.

AS A Project Leader
I WANT to make service conceptual design
SO THAT I could demonstrate what service is about to wider auditory

Acceptance criteria

• Service is described in one sharp sentence
• Service key features are identified and provided as bullets list

Idea of this issue is to build CI for the project using Travis CI . As a first step, it is needed to run unit tests for each pull request and report results back to GitHub.

Acceptance criteria

• Unit tests are run for each pull requests.
• Unit test results are reported to each pull requests.
• Build status badge is placed to the README.

AS A Project Leader
I WANT to design service API
SO THAT I could demonstrate what service is about to wider auditory

We need to have service API documented, so it could be presented in terms of making service capabilities demonstration. It would also be quite beneficial to have API documented in Swagger-compatible format (like OpenAPI), so that it would be possible to make manual interaction with the service on the fly.

Acceptance criteria

• API is documented in OpenAPI 3 format
• API schema is available on /docs/ endpoint
• Local docker-compose environment runs SwaggerUI that by default shows docs from /docs/ endpoint

This issue is intended to track progress of refactoring of subscription-related domain model.

Acceptance criteria

• Refactor Subscription
• Refactor SubscriptionFactory
• Refactor SubscriptionRepository
• Refactor SubscriptionService

Idea of this task is to implement configurable base path. This base path should be provided via API_BASE_PATH environment variable, and once it is set, API method routes should be prefixed with its value.

For example, with next docker-compose.yml:

  newsfeed:
    environment:
      API_BASE_PATH: '/api-example'

API methods should be accessible like:

GET /api-example/newsfeed/{newsfeed_id}/events/
POST /api-example/newsfeed/{newsfeed_id}/events/
...

Endpoint GET /docs/ should also return API_BASE_PATH value in servers section.

Idea of this issue is to implement event storage provider using Redis.

Acceptance criteria

• Provider is implemented.
• Integration tests pass.

There is a need to refactor subscription_storages module.

Acceptance criteria

• Rename module subscription_storage -> subscription_storages
• Rename AsyncInMemorySubscriptionStorage -> InMemorySubscriptionStorage
• Change order of methods to get*, get, add, delete
• Add exception class for case when subscription could not be found
• Refactor delete to use FQID

AS A Project Leader
I WANT to design service architecture
SO THAT I could demonstrate what service is about to wider auditory

Acceptance criteria

• Service architecture is designed
• Service architecture diagram is created
• Service architecture diagram is placed in project docs

This issue is intended to track progress of refactoring of event-related domain model.

Acceptance criteria

• Refactor EventFQID
• Refactor Event
• Refactor EventFactory
• Refactor EventRepository
• Refactor EventDispatcherService
• Refactor EventPublisherService

Acceptance criteria

• File CONTRIBUTORS.rst is added to root of the repo.

Example: https://github.com/ets-labs/python-dependency-injector/blob/master/CONTRIBUTORS.rst

Idea of this task is to define optimal infrastructure providers for Milestone 2.

Acceptance criteria

• Subscription storage has relational or key-value storage type.
• Event queue has pub/sub queue type.
• Event storage has key-value storage type.
• Priority is given to fewer types of solutions.
• It must be possible to run solution(s) in docker-compose.

Some meditation on our demo application led to next list of ideas (inlined).

We need to create demo application so we could do product demos to people, and people could play with product by their own.

Acceptance criteria

• We see UI components for 3 newsfeeds.
• We see events of each newsfeed in chronological order.
• We can publish events to any of visible newsfeeds.
• We can subscribe newsfeeds to each other.
• We can unsubscribe newsfeeds from each other.
• We can see list of each newsfeed subscriptions.

Idea of this task is to introduce concurrent processing of events by adding multiple event processing tasks.

Acceptance criteria

• Service can process events concurrently
• Event processing concurrency is configurable

Idea of this issue is to add mypy as a lint tool.

Acceptance criteria

• Add mypy checks to make lint
• Add type-hinting to webapi, domain model and infrastructure layer

Idea of this task is to make clean and easy to understand application bootstrap process. It should also include explicit bootstrap of application for unit testing.

Nothing to special, just code skeleton application with all basic components.

Idea of this issue is to add running of static analyzers to CI pipeline.

Acceptance criteria

• Static analyzers are run for each pull requests.
• Static analyzer results are reported to each pull requests.

Idea of this issue is to create a chart that will demonstrate splitting NewsFeed into microservices.

Acceptance criteria

• Chart is created.
• Chart is reviewed.
• Chart is published to README.

Currently there is a possibility to subscribe newsfeed to itself. This is a domain integrity bug and should be fixed.

Idea of this issue is to make a decision about what framework / language / etc we use for API test cases automation.

Acceptance criteria

• Solution meets automation engineer vision & motivation
• Solution is agreed among project team

There is a need to refactor event_storages module.

Acceptance criteria

• Rename module event_storage -> event_storages
• Rename AsyncInMemoryEventStorage -> InMemoryEventStorage
• Change order of methods to get*, get, add, delete
• Add exception class for case when event could not be found

We need this for pre-flight OPTIONS requests in Cross-origin resource sharing (CORS).

Idea of this issue is to introduce limits for in-memory infrastructure providers to protect system from potential resources exhaustion.

Acceptance criteria

• Number of newsfeeds is limited
• Number of events per newsfeed is limited
• Number of events in event queue is limited
• Number of subscriptions per newsfeed is limited
• Limits are set to reasonable values by default
• Limits are configurable via environment variables

This issue is intended to make validation of newsfeed ID length to reasonable value.

Acceptance criteria:

• Length of newsfeed ID is validated agains maximum reasonable value
• Maximum reasonable value is set by default
• Maximum reasonable value is configurable via environment variable

Service doesn't handle a situation when newsfeed A subscribes to newsfeed B multiple times. That leads to creation of multiple subscriptions and further duplication of publishing events. Domain integrity rule here is that there should only one subscription between newsfeed A subscribes to newsfeed B .

There is a need to refactor event_queues module.

Acceptance criteria

• Rename AsyncInMemoryEventQueue -> InMemoryEventQueue
• Change order of methods to get, put
• Add exception class for case when queue is full

Idea of this issue is to implement event queue provider using Redis.

Acceptance criteria

• Provider is implemented.
• Integration tests pass.

Idea of that issue is to make a review of product in the Milestone 1. Review is intended to summarize the list of work that should be done before closing Milestone 1 and may result in list of minor and moderate size issues.

Idea of this issue is to implement nice and fancy automatic approach for API integration testing.

Acceptance criteria

• Test cases are automated.
• Automation code is reviewed.

Idea of this issue is to add an integration test that would cover all functionality introduced in Milestone 1 (e.g. scripts/sample_check.py).

Acceptance criteria:

• Integration test covers all Milestone 1 functionality
• Integration test reports results of its success or failure to terminal
• Integration test is human-readable

Idea of this issue is to create event dispatcher microservice (following microservices splitting chart).

Acceptance criteria

• Service is splitted out.
• Integration tests pass.

Idea of this issue is to add serializer for event entities.

Acceptance criteria

• Event-related endpoints use event serializer
• Swagger docs is updated according event serializer

Idea of this issue is to design integration test cases that would cover existing service API.

Currently we have demo application that could be a nice start point for getting acquainted with service functionality. API documentation could be obtained from swagger by running service locally and visiting http://127.0.0.1:8000/swagger/. We also have scripts/integration_check.py script that plays a role of integration test - it could be used as a source for grabbing some test cases.

Acceptance criteria

• API test cases are designed.
• API test cases cover all existing functionality.
• API test cases cover all edge cases.

Idea of this issue is to create event processor microservice (following microservices splitting chart).

Acceptance criteria

• Service is splitted out.
• Integration tests pass.

Idea of this issue is to refactor application configuration so that application_factory doesn't contain any knowledge on what is the source of configuration.

Idea of this issue is to make a decision on what Redis client library is used.

Acceptance criteria

• All currently know libraries have been analyzed (at least shortly)
• Solution is agreed among project team

Idea of this issue is to find an API results reporting system that could be used the project. As an example of such results reporting system we could use coveralls.io - https://coveralls.io/github/ets-labs/python-dependency-injector?branch=master

Acceptance criteria

• Solution meets automation engineer vision & motivation
• Solution is agreed among project team