Giter Club home page Giter Club logo

ticketing's Introduction

Ticketing (think concert tickets)

This project is a remix of the app built in the course microservices with node.js and react starting in chapter 5. You can find the source code for the course here.

This repository shows:

  • another way to manage shared/common modules (with Nx)
  • tricks to use Fastify with NestJS
  • tricks to consume and produce ES6 modules with NestJS (and Nx)
  • how to integrate Ory network in NestJS and Angular apps for authentication and authorization flows
  • how to set up Ory in local and remote working environments
  • how to use RabbitMQ with NestJS
  • how to define/validate environment variables
  • how to containerize Nx apps with Docker
  • how to integrate Nx into a Kubernetes workflow
  • how to dynamically rebuild Docker images based on the Nx project graph

User story

journey
  title Exchanging tickets online
  section Register
    Create a user: 3: Seller, Buyer
    Signin : 3: Seller
  section Search tickets
    Find by name: 5: Buyer, Seller
  section Create ticket offer
    Write description: 3: Seller
    Define price: 3: Seller
  section Buy ticket
    Reserve tickets: 5: Buyer
    Place order: 3: Buyer
    Pay ticket: 3: Buyer
  section Check orders
    See orders list: 5: Seller, Buyer

Architecture

---
title: Ticketing architecture
---
flowchart LR
%% defining styles
    classDef app fill:#f7e081,stroke:#333,stroke-width:1px

%% defining entities
    FE[Angular app]
    LB[Nginx proxy]
    A[Auth API]
    A-M[(Mongo)]
    T[Tickets API]
    T-M[(Mongo)]
    O[Orders API]
    O-M[(Mongo)]
    P[Payments API]
    P-M[(Mongo)]
    St[Stripe]
    E[Expiration API]
    E-R[(Redis)]
    RMQ[RabbitMQ]
    Kr[Kratos]
    Ke[Keto]
    Hy[Hydra]

%% assigning styles to entities
    %%AS,OS,ES,TS,PS:::service
    %%class A,T,O,E,P,FE app;

%% flow
    FE -->|HTTP| LB
    FE -->|HTTP| St <-->|HTTP| PS
    FE -->|HTTP| ORY <-->|HTTP| AS
    LB --->|HTTP| AS & TS & OS & PS
    RMQ <-.->|AMQP| TS & OS & ES & PS
    TS & OS & PS -->|HTTP| ORY
    subgraph AS [Auth service]
    direction LR
    A --> A-M
    end
    subgraph ORY [Ory Network]
    direction LR
    Kr
    Ke
    Hy
    end
    subgraph TS [Tickets service]
    direction LR
    T --> T-M
    end
    subgraph OS [Orders service]
    direction LR
    O --> O-M
    end
    subgraph ES [Expiration service]
    direction LR
    E <--> E-R
    end
    subgraph PS [Payments service]
    direction LR
    P --> P-M
    end

Entities

---
title: Ticketing entities
---
erDiagram
    User ||--o{ Ticket : owns
    User ||--o{ Order : owns
    User ||--o{ Payment : owns
    Ticket ||--o| Order : "bound to"
    Order ||--o| Payment : "bound to"

    User {
        int id PK
        string email "unique"
    }
    Ticket {
        string id PK
        string title
        float price
        int version
        string userId FK
        string orderId FK "Optional"
    }
    Order {
        string id PK
        string status
        int version
        string ticketId FK
        string userId FK
    }
    Payment {
        string id PK
        string orderId FK
        string stripeId "Charge ID from Stripe"
        int version
    }

Permissions

Permissions are granted or denied using Ory Permissions (Keto) policies.

---
title: Entities namespaces and relationships
---
classDiagram

    NamespaceRelations *-- Namespace
    NamespacePermissions *-- Namespace
    Namespace <|-- User
    Namespace <|-- Group
    Namespace <|-- Ticket
    Namespace <|-- Order
    Namespace <|-- Payment
    Group o-- User : "members"
    Ticket o-- User : "owners"
    Order o-- User : "owners"
    Order *-- Ticket : "parents"
    Payment o-- User : "owners"
    Payment *-- Order : "parents"

    class Context {
      <<Interface>>
      subject: never;
    }

    class NamespaceRelations {
        <<Interface>>
        +[relation: string]: INamespace[]
    }

    class NamespacePermissions {
        <<Interface>>
        +[method: string]: (ctx: Context) => boolean
    }

    class Namespace {
        <<Interface>>
        -related?: NamespaceRelations
        -permits?: NamespacePermissions
    }

    class User {
    }

    note for Group "<i>Users</i> can be <b>members</b> of a <i>Group</i>"
    class Group {
        +related.members: User[]
    }

    note for Ticket "<i>Users</i> (in owners) are allowed to <b>edit</b>. \nHowever <i>Ticket</i> <b>owners</b> cannot <b>order</b> a Ticket. \nImplicitly, anyone can <b>view</b> Tickets"
    class Ticket {
        +related.owners: User[]
        +permits.edit(ctx: Context): boolean
        +permits.order(ctx: Context): boolean
    }

    note for Order "<i>Order</i> is bound to a <i>Ticket</i>. \n<i>Users</i> (in <b>owners</b>) are allowed to <b>view and edit</b>. \n <i>Order's Ticket</i> <b>owners</b> are allowed to <b>view</b>."
    class Order {
        +related.owners: User[]
        +related.parents: Tickets[]
        +permits.edit(ctx: Context): boolean
        +permits.view(ctx: Context): boolean
    }

    note for Payment "<i>Payment</i> is bound to a Ticket's <i>Order</i>. \n<i>Users</i> (in <b>owners</b>) are allowed to <b>view and edit</b>. \n<i>Payment</i> can be <b>viewed</b> by <i>Order's Ticket</i> <b>owners</b>."
    class Payment {
        +related.owners: User[]
        +related.parents: Order[]
        +permits.edit(ctx: Context): boolean
        +permits.view(ctx: Context): boolean
    }

Events

sequenceDiagram
  participant Tickets service
  participant Orders service
  participant Payments service
  participant Expiration service
  participant RMQ

  loop ticket:created
    %% event emitted by tickets service
    Tickets service->>+RMQ: Publish new ticket
    RMQ-->>-Orders service: Dispatch new ticket
    Note left of Orders service: Orders service needs to know <br> about tickets that can be reserved.
  end

  loop ticket:updated
    %% event emitted by tickets service
    Tickets service->>+RMQ: Publish updated ticket
    RMQ-->>-Orders service: Dispatch updated ticket
    Note left of Orders service: Orders service needs to know <br> if tickets price have changed and <br>if they are successfully reserved
  end

  loop order:created
    %% event emitted by orders service
    Orders service->>+RMQ: Publish new order
    par RMQ to Tickets service
      RMQ->>Tickets service: Dispatch new order
      Note left of Tickets service: Tickets service needs to know<br>if a ticket has been reserved<br>to prevent its edition.
      and RMQ to Payments service
      RMQ->>Payments service: Dispatch new order
      Note left of Payments service: Payments service needs to know<br>there is a new order that a user<br>might submit a payment for.
      and RMQ to Expiration service
      RMQ->>Expiration service: Dispatch new order
      Note left of Expiration service: Expiration service needs to start<br>a timer to eventually time out<br>this order.
    end
  end


  loop order:cancelled
    %% event emitted by orders service
    Orders service->>+RMQ: Publish cancelled order
    par RMQ to Tickets service
      RMQ->>Tickets service: Dispatch cancelled order
      Note left of Tickets service: Tickets service should unreserve ticket<br>if the corresponding order has been<br>cancelled so this ticket can be <br>edited again
      and RMQ to Payments service
      RMQ->>Payments service: Dispatch cancelled order
      Note left of Payments service: Payments service should know that<br>any incoming payments for this order<br>should be rejected
    end
  end

  loop expiration:complete
    %% event emitted by expiration service
    Expiration service->>+RMQ: Publish complete expiration
    par RMQ to Orders service
      RMQ->>Orders service: Dispatch expired order
      Note left of Orders service: Orders service needs to know that an order<br>has gone over the 15 minutes time limit.<br>It is up to the order service to decide<br> wether or not to cancel the order.
    end
  end

  loop payment:created
    %% event emitted by payments service
    Payments service->>+RMQ: Publish payment created
    par RMQ to Orders service
      RMQ->>Orders service: Dispatch payment created
      Note left of Orders service: Orders service needs to know that an order<br>has been paid for.
    end
  end

Environment variables

I am using dotenv-vault to manage environment variables. You can fork the project and use the following links to create your Dotenv project by forking the corresponding Dotenv project.

project fork
docker fork with dotenv-vault
auth fork with dotenv-vault
expiration fork with dotenv-vault
moderation fork with dotenv-vault
orders fork with dotenv-vault
payments fork with dotenv-vault
tickets fork with dotenv-vault

Useful commands

... to run after configuring the required environment variables

# build custom Nginx Proxy
yarn docker:proxy:build

# build custom RabbitMQ node
yarn docker:rmq:build

# start the Storage and Broker dependencies (mongo, redis, rabbitmq)
yarn docker:deps:up

# start Nginx Proxy (for backend services and frontend app)
yarn docker:proxy:up

# Generate Ory network configuration from .env
yarn ory:generate:kratos
yarn ory:generate:keto

# start Ory network (Kratos and Keto with database migrations)
yarn docker:ory:up

# start backend services
yarn start:backend

# start (Angular) frontend app
yarn start:frontend:local

ticketing's People

Contributors

getlarge avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

ticketing's Issues

refactor: consider `typia` and `nestia`

They both seem great candidates to :

  • replace class-validator and class-transformer to declare DTOs used for validation
  • replace nests-swagger the same interface created for Typia can also be inferred to generate API properties and response can be generated by Nestia TypedRoute

This results in classes containing fewer decorators and more performant validation steps.

I already successfully tested Nestia integration in this app.

References

feat: create public MQTT API for users

Events are only propagated across microservices at the moment.
It could be great to provide an example implementation of an MQTT ( and Websocket ) API for users and use it in the front end to refresh state in real time without polling.

  • Enabling MQTT in RabbitMQ
  • Enabling external authentication plugins (HTTP + cache) in RabbitMQ config
  • Create NestJS service to interact with RabbitMQ queues over (REST) API (to limit number of queues / user)
  • Create authentication endpoints and service in auth service
  • Define MQTT topic pattern ( users/${user_id}/${resource}/${action} or reusing Zanzibar notation ? ) and payload schema
  • Trigger MQTT events via a new service subscribing to all existing events ? could this facilitate event sourcing ?

ops: enable Ory self hosting for local development

This would make demonstration and forks setup much simpler.

  • setup Docker-compose
  • adapt applications configuration (?)
  • restore sign-in and sign-up UI to not depend on Account Experience (https://www.ory.sh/docs/kratos/bring-your-own-ui/custom-ui-overview) which is only available via Ory network the guys are so nice they have a docker image that provides it
  • script services configuration using Ory CLI custom CLI
  • automate services configuration using the above and Github Actions (for remote Ory tenant)

—-
See:

fix: improve communication between services

In the happy path, the current setup of message production/consumption does not trigger issue.
Still at the moment when an error occurs on the consumer side :

  • the producer is not aware and cannot handle the error (because of @EventPattern usage)
  • the consumer might stay stuck in an infinite error loop because in case of error the message is Nacked and re-queued.

  • Replace @EventPattern by @MessagePattern where errors needs to be handled by producer
  • Replace ClientProxy.emit usage by ClientProxy.send where errors needs to be handled by producer
  • replace usage of channel.nack(message) by channel.nack(message, false, true) as the default behaviour

feat: improve Error classification and creation

Consider approach suggested by defekt to construct and handle errors.
This could be particularly helpful for error handling between micro services when a clear strategy needs to be in place to know wether an error can be retried (producer side), re-queued (consumer side), dropped (consumer side) or added to DLX (consumer side).


See this article for good explanation about correct error handling in Event driven systems.

feat: create fastify upload lib

Dealing with File Upload in NestJS is straightforward, as the official documentation shows. However, it differs when using Fastify adapter.

To remove the pain, one potentially good candidate developed by one of the NestJS core team members is @nest-lab/fastify-multer, but it depends on fastify-multer, which is abandoned and not maintained anymore.

Another one is nest-file-fastify, which depends on fastify-multipart, and the Fastify team maintains it. But nest-file-fastify is outdated and not maintained anymore.

Let's make a local fork of nest-file-fastify and update the fastify-multipart dependency to the latest version, so that we can add file upload/download endpoints to the tickets controller.

feat: add example implementation of OpenTelemetry SDK

On the backend apps, create a tracing module imported before everything else in main.
Init OTel SDK with instrumentations library for the following modules:

  • Net
  • HTTP
  • Fastify (if exists?)
  • MongoDB
  • BullMQ
  • AMQP lib

Create a service in docker.compose with Jaeger to export traces to.

feat: add RequestIdMiddleware

Create Nest middleware that read or add X-Request-Id HTTP request header. Apply to apps :

  • auth
  • orders
  • payments
  • tickets

refactor: replace @ory/client built-in axios

To improve error responses handling for calls to Ory APIs, it would be useful to provide a custom axios instance.
This instance would have custom interceptors to wrap errors in standard class, and would allow to add retry strategies in case of rate-limiting.

The BaseAPI class from @ory/client, which all API extends, allow to pass a custom axios instance in the constructor.
The OryPermissionsService and OryAuthenticationService could make use of this improvement.

In OryPermissionsModule and OryAuthenticationModule, import HttpModule :

// ...
declare module 'axios' {
  interface AxiosRequestConfig {
    retries?: number;
    retryCondition?: (error: AxiosError) => boolean;
    retryDelay?: (error: AxiosError, retryCount: number) => number;
  }
}

const HttpModuleWithRetry = HttpModule.register({
  timeout: 5000,
  responseType: 'json',
  validateStatus(status: number) {
    return status >= 200 && status < 300;
  },
  retries: 3,
  retryCondition(error) {
    const statusToRetry = [429];
    return error.response?.status
      ? statusToRetry.includes(error.response?.status)
      : false;
  },
  retryDelay(error, retryCount) {
    if (error.response?.status === 429) {
      const headers = error.response.headers;
      const remaining = headers['x-ratelimit-remaining'];
      const resetTimestamp = headers['x-ratelimit-reset'];
      if (Number(remaining) === 0) {
        return Number(resetTimestamp) * 1000 - Date.now();
      }
    }
    return retryCount * 250;
  },
});

// ...

In OryPermissionsService, inject HttpService, create axios interceptors and pass custom axios instance:

  constructor(
    @Inject(OryPermissionsModuleOptions) options: OryPermissionsModuleOptions,
    @Inject(HttpService) private readonly httpService: HttpService,
  ) {
    this.httpService.axiosRef.interceptors.response.use(
      (response) => response,
      async (error) => {
        if (isAxiosError(error)) {
          if ('response' in error) {
            const { config, response } = error;
            const shouldRetry = config?.retryCondition(error) ?? true;
            if (config?.retries && shouldRetry) {
              const retryDelay =
                config?.retryDelay(error, config.retries) ?? 250;
              config.retries -= 1;
              await delay(retryDelay);
              return this.httpService.axiosRef(config);
            }
            const oryError = new OryError(error);
            return Promise.reject(oryError);
          }
          const oryError = new OryError(error);
          return Promise.reject(oryError);
        }
        const oryError = new OryError(error);
        return Promise.reject(oryError);
      },
    );
    const { ketoAccessToken, ketoAdminApiPath, ketoPublicApiPath } = options;
    this.relationShipApi = new RelationshipApi(
      new Configuration({
        basePath: ketoAdminApiPath,
        accessToken: ketoAccessToken,
      }),
      '',
      this.httpService.axiosRef,
    );
    this.permissionApi = new PermissionApi(
      new Configuration({
        basePath: ketoPublicApiPath,
        accessToken: ketoAccessToken,
      }),
      '',
      this.httpService.axiosRef,
    );
  }

feat: update Ory Keto setup to enable and test RBAC permissions

  • Update namespaces.ts with Group.members (namespace.relation)
  • Create namespace depending on Group subject set
  • add shortcut to easily add relations and check | expand permissions (create CLI based NestJS app consuming OryPermissionService ? )
  • create one time cronjob to ensure base relationships are created at init
# add User <user_id> to as admin Group members
Group:admin#members@User:<user_id>

# allow admin Group members to view Moderation <moderation_id>
Moderation:<moderation_id>#view@(Group:admin#members)

class User implements Namespace {}

class Group implements Namespace {
  related: {
    members: User[];
  };
}

class Moderation implements Namespace {
  related: {
    editors: (SubjectSet<Group, "members">)[]
  };

  permits = {
    view: (ctx: Context) => this.related.editors.includes(ctx.subject),
    edit: (ctx: Context) => this.related.editors.includes(ctx.subject),
  };
}

—-
References

feat: implement OAuth2 client credentials flow to access API

To demonstrate usage of Ory Hydra, let's implement the following use case:

  • A user can create clients (used in applications) to act on their behalf
  • The client will be bound to the user with the owner property during the client creation
  • The client will be replicated into the application's database
  • When requesting an OAuth2 token extra information will be appended to the access_token (using the Ory Hydra webhook)
  • When sending requests authenticated with an access token (ory_at_xxx), a guard should validate the access token with the introspection method from OryOauth2Service (Consider using or-guard with OryAuthenticationGuard and OryOAuth2AuthenticationGuard)
  • Check authorization using the permissions of the client's owner (with Keto) the scopes contained in the access token which is a bit of an OAuth2 spec violation but we will survive it!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.