Giter Club home page Giter Club logo

fully-featured-scalable-chat-app's Introduction

FULLY FEATURED SCALABLE CHAT APP

Project Status: WIP – Initial development is in progress, but there has not yet been a stable, usable release suitable for the public.

The goal is to build a chat app with a complete feature set using Ably Realtime in combination with other services to store, manipulate and share data.

If you have any questions, ideas or want to contribute, please raise an issue or reach out to us.

Things you will need to make this run locally!

  1. Node 16 installed

  2. The Azure Functions Runtime from NPM. To install this run:

    npm install -g azure-functions-core-tools@4

  3. an .env file in ./api:

JWT_SIGNING_KEY=key used to sign tokens for users. Can be anything you decide it to be

# Ably
ABLY_API_KEY=YOURKEY:HERE
APP_ID=[YOUR ABLY APP ID](https://faqs.ably.com/how-do-i-find-my-app-id)
CONTROL_KEY=[YOUR ABLY CONTROL KEY](https://ably.com/documentation/control-api#authentication)

# Azure
COSMOS_ENDPOINT=https://yourcosomsdb.documents.azure.com
COSMOS_KEY=ASK FOR THIS OR MAKE YOUR OWN
COSMOS_DATABASE_ID=metadata
AZURE_STORAGE_CONNECTION_STRING=your string here
AZURE_STORAGE_CONTAINER_NAME=container name here

# Auth0
AUTH0_DOMAIN=yourdomain.auth0.com
AUTH0_CLIENTID=yourclientid
AUTH0_REDIRECT_URI=http://localhost:8080/auth0-landing
  1. Setup the node modules and run the app
# from root folder
npm run init # installs node modules for api & integrations
npm run start # runs the dev server

Services setup

As seen in the .env file, there are a few services you'll need to sign up with to use the project in its current form.

  • Ably
  • CosmosDB
  • Azure Blob Storage
  • Auth0

Ably credentials

In order to obtain the Ably credentials, you will first need to sign up for a free Ably Account. Once you have an Ably account, you can go to the default app generated and get the Root API key for it. This will be used for the ABLY_API_KEY environment. The APP_ID env variable should be set to the first part of your API key, before the fullstop. If your API key is 12345.jh40fj23jkd0-,32c3-j-, you should set it to 12345.

For the CONTROL_KEY, which is used for controlling the creation of API keys, apps and more programatically, you will need to go to the Access Token as a logged in user, and click 'Create new access token'. Give it a name, from the 'Account' dropdown select the Account which has the app you have an API key for, and make sure to select the read:key and write:key permissions. Create the token, and use its value as the CONTROL_KEY.

CosmosDB

CosmosDB is used for storing data for this app. To get an Azure account and create a CosmosDB resource, follow the steps in Azure's setup tutorial. You should set the COSMOS_ENDPOINT to be the URI provided in your new subaccount. The COSMOS_KEY is the primary key for the CosmosDB account, which you can access as described by Azure.

COSMOS_DATABASE_ID is the name of the container you will be using within your CosmosDB account.

Azure Blob Storage

Using the same Azure account created for CosmosDB, create a new Data Storage Container. Once you have that setup, go to the 'Access Keys' section in the sidebar to obtain a connection string for AZURE_STORAGE_CONNECTION_STRING, and then set AZURE_STORAGE_CONTAINER_NAME to whatever you named the container.

Auth0

If you want to include Auth0 as an authentication method, you will need to create an Auth0 account. Once you have an Auth0 account, create an Application with 'Regular Web Applications' selected as the application type. In that app you can then copy the domain for AUTH0_DOMAIN, and the clientID for the AUTH0_CLIENTID. The AUTH0_REDIRECT_URI should point to the appropriate URI that Auth0, after authenticating a user, should redirect to. The default value should work for running locally.

Further Auth0 setup

On the settings tab of your Auth0 app, scroll down to the section called ‘Application URIs.’ In it, you should see a field for ‘Allowed Callback URLs’ and ‘Allowed Logout URLs.’ For context, the flow of a webpage using Auth0 is:

Your site links a user to your Auth0 app’s login page, where they sign in The Auth0 page redirects the user back to your website’s ‘callback’ page When the user wants to log out, they are directed to the Auth0 app’s logout page and then redirected back to the page specified in the ‘returnTo’ query passed to the logout page

To avoid potential misuse and abuse, you need to specify what URLs Auth0 can redirect to. When hosting this chat app locally, it is hosted on localhost:8080, so set the Allowed Callback URLs to ‘http://localhost:8080/auth0-landing’. When a user logs out, we’ll have the user redirected back to our main page, so set the Allowed Logout URLs to ‘http://localhost:8080/’.

Design

The chat app is made up of the following:

  • A Web Application that is hosted in Azure Static Web Apps (React)
  • A "BFF" API built to run in Azure Functions (Node.js)
  • A CosmosDB database to store metadata (user accounts, chat channel metadata)
  • An Ably Realtime account to send and receive chat messages
  • An Archive API to receive events from Ably Reactor and maintain a chat history
  • A Storage bucket to store Chat Archive.

The React Application

The React application is a default, single page application. It uses a mixture of react-router-dom and a custom AppProvider to provide the security context for the application.

The app uses @ably-labs/react-hooks to interact with Ably Channels, and the application is composed of modern React Functional Components.

snowpack is the development server, which will transparently build ES6 code for production.

The BFF (Backend-for-Frontend) API

The BFF is an application specific API that contains all of the serverside logic for the chat app. Because it is hosted on Azure Static Web Apps, we can use the azure-functions-core-tools run the API server.

In addition to this, the Azure Static Web Apps runtime will auto-host the APIs for us - so we don't need to worry about configuring hosting. The BFF is executed on serverless infrastrucutre, and Azure SWA will auto-scale it to meet demand.

To add new API endpoints, you will need to add a new directory to the api folder.

First, create a directory for the new API - for example, api/messages. Then, create a function.json file in the new directory.

{
  "bindings": [
    {
      "route": "messages",
      "authLevel": "function",
      "type": "httpTrigger",
      "direction": "in",
      "name": "req",
      "methods": ["get", "post"]
    },
    {
      "type": "http",
      "direction": "out",
      "name": "res"
    }
  ],
  "scriptFile": "../dist/messages/index.js"
}

Next, you'll need to create your TypeScript API:

import "../startup";
import { Context, HttpRequest } from "@azure/functions";

export default async function (context: Context, req: HttpRequest): Promise<void> {
  context.res = { status: 200, body: "I'm an API" };
}

This API will now be mounted at http://localhost:8080/api/messages.

And that's it! The tooling and SDK will auto-detect your code as you change it and rebuild your functions for you.

Authentication and Security

The app uses JWT token authentication between the Web application and the BFF. We store user credentials and salted, one way hashed passwords (done with bcrypt) in the CosmosDB database.

When a user authenticates, the app signs a JWT token with the user's id and username that is then sent to the BFF in subsequent requests for authenticated data. This means, with a small amount of code in the APIs, we can ensure that the user is who they claim to be, and that they are entitled to access API data.

We can expand this model to include a collection of roles for claims-based authentication to resources in the application.

Creating an Authenticated User Only API call

We can create a JWT token authenticated API call by using the following convenience methods in the BFF API.

import "../startup";
import { Context, HttpRequest } from "@azure/functions";
import { authorized, ApiRequestContext } from "../common/ApiRequestContext";

export default async function (context: Context, req: HttpRequest): Promise<void> {
  await authorized(context, req, () => {
    // This code will only run if the user is authenticated

    context.res = {
      status: 200,
      body: JSON.stringify("I am validated and authenticated")
    };
  });
}

If you want to access the authenticated users information as part of one of these API calls, you can do the following:

import "../startup";
import { Context, HttpRequest } from "@azure/functions";
import { authorized, ApiRequestContext } from "../common/ApiRequestContext";

export default async function (context: Context, req: HttpRequest): Promise<void> {
  await authorized(
    context,
    req,
    ({ user }: ApiRequestContext) => {
      // user is the userDetails object retrieved from CosmosDb

      context.res = {
        status: 200,
        body: JSON.stringify("I am validated and authenticated")
      };
    },
    true
  ); // <- true to include the userDetails object in the ApiRequestContext
}

Accessing user data in the React application

The in-app authentication is implemented in AppProviders.jsx.

It provides a React Hook that will return the userDetails object if the user is authenticated (along with ensuring that the user is authenticated at all). If a given user is not authenticated, they will be redirected to the login page in all cases.

Because the AppProvider takes care of authentication, you'll need to use hooks to access user data, and to make authenticated API calls in any components.

Here is an example of accessing the userDetails object of the currently authenticated user.

import { useAuth } from "../../AppProviders";

const MyComponent = () => {
  const { user } = useAuth();

  return <div>{user.username}</div>;
};

export default MyComponent;

You can also access an instance of the BffApiClient class, which will allow you to make authenticated API calls and already contains the currently logged in users JWT token.

import { useAuth } from "../../AppProviders";

const MyComponent = () => {
  const { api } = useAuth();
  const [channels, setChannels] = useState([]);

  useEffect(() => {
    const fetchChannels = async () => {
      const response = await api.listChannels();
      setChannels(response.channels);
    };
    fetchChannels();
  }, []);

  return <div>... bind channel data here</div>;
};

export default MyComponent;

The above example uses the useEffect hook to fetch the channels when the component mounts - the API request is made using the api instance, provided by the useAuth hook.

This is the only way you should make API calls to the BFF from a component, as it'll ensure the JWT token is valid and present.

If you're adding new BFF APIs to the application, you'll need to implement a new function in /app/src/sdk/BffApiClient.js to make it available to your components.

These BffApiClient calls are simple, and look like this:

async listChannels() {
    const result = await this.get("/api/channels");
    return await result.json();
}

Some utility code in the client will make sure the correct JWT token is present when the request is made.

The CosmosDb datastore

We're using CosmosDb to store our application metadata because it is a scalable, highly available, managed database that we don't have to administer ourselves. It can run in a pre-provisioned or serverless mode, helping keep costs low when the application isn't in use (at the cost of some performance).

We use a single CosmosDb database to store all of the metadata, and inside, we've created a collection for each type of Entity we're storing.

For example: The User collection, stores our User records - and can be queried using SQL-like syntax. CosmosDb makes this easy by automatically indexing json documents.

Each of the stored metadata entities have an id and a type field, and we're using a generic repository class (/api/common/dataaccess/CosmosDbMetadataRepository) to load and save these entities.

For local development, you can either use the cloud hosted version of Cosmos, or use one of the available docker container images to run a local copy of the database.

Ably for Chat

We're using Ably channels to store our chat messages and to push events to our React application. Each connected user will receive messages for channels that they are actively viewing in real-time, and we're using Channel rewind to populate the most recently sent messages.

Future work

Messages may be corrected asyncronously after they have been received - for instance, to apply profanity filtering, or to correct spelling errors. These correction messages will be part of the stream, and applied retroactively in the react application. (Further development on this in later epics)

This design allows us to stand up extra APIs that consume these events, and publish their own elaborations on the channels for clients to respond to.

Suggested Chat Archiving

The Chat Archive API

Because Ably events will vanish over time, we're going to store copies of inbound events on each channel into our Chat Archive via the Archive API.

The Archive API will receieve reactor messages for all of our channels, and append them to channel-specific Azure Storage Blobs. The API will append to a single file until it reaches a size threshold (~500kb) and then create a new file for subsequent messages.

The Archive API will maintain a record of the currently active archive file in the Metadata database for each channel.

The Archive API will be able to update a search index as messages are received and archived to later expose them in search.

Testing

Tests are written in jest with ts-jest used to execute the APIs TypeScript tests.

fully-featured-scalable-chat-app's People

Contributors

davidwhitney avatar dependabot[bot] avatar fliptopbox avatar marcduiker avatar srushtika avatar thisisjofrank avatar tomczoink avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

fully-featured-scalable-chat-app's Issues

Any plans to add a lockfile for pnpm?

You know, to save hundreds of mb in precious storage and a pound or 2 of co2 emissions per download. Could be done with "pnpm import" but I haven't tried.

Message editing and moderation architectures

This is a proposal for two different message processing architectures that I think it would be interesting/useful to see in FFS.

It's common for chat apps to need to edit, augment or moderate messages. A chat is usually formed around a single shared channel to which all users subscribe for messages. Depending on the use case there are a few different patterns that may be appropriate for extending functionality to allow for better experiences:

Delayed augmentation

For use cases where some further processing needs to happen on a message, for example a bot looking up address details from a message and providing a maps link.

A user publishes to the shared chat channel so that the message is immediately shared with all users. The channel also has a rule configured that invokes a cloud function and runs whatever extended functionality is desired and publishes back on the channel. User's apps then augment the original message with this additional message.

The processing for the cloud function is relatively straightforward to be able to process multiple different chat channels as it just publishes messages back onto the channel that they came from.

Benefits:

  • the original message never leaves ably so you get all the delivery guarantees
  • if the cloud function only sometimes needs to augment messages then it doesn't need to send any additional messages and customers can save on message quota
  • architecturally simple, users need pub/sub auth for a single channel to join a chat

Cons:

  • if you need to filter messages for inappropriate content then the original objectionable message is still shared with all participants which is not desirable
  • the cloud function publishing back on the channel will invoke itself again and the function must ignore messages from itself
  • if every message needs to be reformatted and only the augmented message matters then message quota is wasted delivering the original message

Message filtering

For use cases where you want messages checked, edited or rejected according to a content policy and messages cannot be broadcast to the chat group without being checked.

Users now only have permissions to subscribe to the chat channel and cannot publish to it directly. Instead they publish to separate channel for which they do not have subscribe permissions. This publish channel has a rule configured that invokes the cloud function that performs content checking editing. The function then publishes the checked message to the subscribe only chat channel (it alone has the permissions to do so).

In the case where the function may reject or discard messages then this may need to be fed back to the publisher. The publisher will need a dedicated inbox channel to which the function may publish.

The tricky part is for the function to know which channel name to forward the the filtered message on to or what the inbox channel for the client is. The naive approach would be to include that in the published message but the contents of the inbound message are not trusted so a client could pretend another channel was actually their inbox and we don't have a means with tokens yet to authenticate those claims or pass them on to the function. So I think you have to rely on the inbound channel name and the client ID (as those can be authorised by the token) and have a consistent name mapping for inbound channel -> outbound channel and client ID -> inbox.

Benefits:

  • messages that violate the content policy are never broadcast
  • only a single broadcast message per publish

Cons:

  • more complex, you have to manage multiple different permissions in a token
  • going through an external function impacts guaranteed message delivery
  • may increase latency on message delivery

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.