Giter Club home page Giter Club logo

algodex-service's Introduction

Algodex Service

This is the 2.0 backend (essentially a custom blockchain indexer) for Algodex and also includes the rewards calculations.

@algodex/service algodex/service:image

You can find the contribution guides for getting started in CONTRIBUTING.md

Algodex Documentation: https://docs.algodex.com

Getting Started

Todo: Below instructions are very incomplete!

Building rust rewards calculation on ubuntu

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
sudo apt install build-essential
sudo apt-get install pkg-config libssl-dev
cd rewards-calc
cargo build

Testing (Localhost with Docker Data)

First make sure CouchDB and Redis services are running according to .integration.test.env. The node.js should not be running as these will be started by the end-to-end test.

Quick Start for CouchDB and Redis Services

cp .testnet.docker.env .env
docker-compose up -f docker-compose.yml -f docker-compose.docker.yml

Set up reverse-proxy

(todo) The reverse proxy sits in front of algodex-service. The basic request flow is nginx -> reverse-proxy-rust -> algodex-service API

Some instructions are at https://github.com/algodex/reverse-proxy-rust

Integration Test Run

npm run end-to-end-light

Core Packages

Core Services

  • Broker (Event Publisher->Queue|Sockets): ./services/broker.js

    • All "Events" are published and pushed to Queue for basic example

  • Worker (BullMQ Worker): ./services/worker.js

    • Removes items from the Queue and stores them in the Database

  • Socket (Event Subscriber and Socket Publisher - only partially implemented): ./server.js

    • Push Broker Events to Subscribed Sockets.
    • Future example: ws://localhost/asset/{id} would subscribe to the appropriate redis channel coming from the Broker

  • API endpoint: ./api/api_server.ts

  • Rewards Calculator ./rewards-calc/src/main.rs

algodex-service's People

Contributors

atrefonas avatar phearzero avatar semantic-release-bot avatar imranzahoor avatar

Stargazers

avatar

Watchers

avatar

Forkers

atrefonas

algodex-service's Issues

The automated release is failing 🚨

🚨 The automated release from the main branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you can benefit from your bug fixes and new features again.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can fix this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the main branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here are some links that can help you:

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

Cannot push to the Git repository.

semantic-release cannot push the version tag to the branch main on the remote Git repository with URL https://github.com/algodex/algodex-node-gradation.git.

This can be caused by:

Good luck with your project ✨

Your semantic-release bot 📦🚀

The automated release is failing 🚨

🚨 The automated release from the main branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you can benefit from your bug fixes and new features again.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can fix this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the main branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here are some links that can help you:

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

Cannot push to the Git repository.

semantic-release cannot push the version tag to the branch main on the remote Git repository with URL https://x-access-token:[secure]@github.com/algodex/algodex-node-gradation.git.

This can be caused by:

Good luck with your project ✨

Your semantic-release bot 📦🚀

✨ Feature: Asset Database

ℹ Overview

Worker: For reach block

Update asset analytics

const block = 12345; // From Event/Queue
const blockDirtyAssets = []; // From Event/Queue, should include information about new orders or dirty accounts

// Fetch from Account Analytics
const previousState = getAssetsAnalytics(blockDirtyAssets, block - 1 ); // Last saved state


const state = [];
// Merge dirty records with previous state and fetch missing information from Algorand Indexer and Explorer
blockDirtyAssets.forEach(()=>{
 // update state and calculate rewards
});

// Return the assets state
// This can be sent to db.bulkDocs()
return [{
 _id: `0-${block}`, // <Asset Id/Index>-<Block Round>
 type: 'asset_analytics',
 created: 12, // Block the asset was added
 modified: block, // Last changed block
 asset: {
   index: 0,
   decimals: 6,
   params: {},
 },
 analytics: {
   price: {
     algo: 20,
     usd: 10,
   },
   spread: {
     buy: {algo: 10},
     sell: {algo: 20},
     price: {algo: 15},
   },
   tvl: {
     algo: 100,
     usd: 50,
   },
 },
}];

📝 Related Issues

🐛 Fix: Add app ids to .env files

ℹ Overview

Testnet
ALGODEX_ALGO_ESCROW_APP=22045503
ALGODEX_ASA_ESCROW_APP=22045522

Mainnet
ALGODEX_ALGO_ESCROW_APP=354073718
ALGODEX_ASA_ESCROW_APP=354073834

📝 Related Issues

✨ Feature: Accounts Database

ℹ Overview

For Each Block

Keep track of all escrow accounts state

// Run After Asset Worker Completes
const block = 12345; // From Event/Queue
const blockDirtyAccounts = []; // From Event/Queue, should include information about new orders or dirty accounts

// Fetch from Account Analytics for the previous block
const epochs = getAccountsAnalyticsEpochs(block - 20, block -1 ); // All Epochs in the range
const previousState = epochs[epochs.length - 1] // Last saved state

// Asset Analytics includes latest spreads since it is run before. Dirty accounts come from the Asset Worker
const assetsInfo= getAssetAnalytics(blockDirtyAccounts.map((acct)=>acct.assetId));

const state = [];
// Merge dirty records with previous state and fetch missing information
// Use threading or Low Level Language to calculate state:
assetsInfo.forEach(()=>{
  // update state and calculate rewards
});

// Return the assets state
// This can be sent to db.bulkDocs()
return [{
  _id: `WYWRYK42XADLY3O62N52BOLT27DMPRA3WNBT2OBRT65N6OEZQWD4OSH6PI-${block}`, // <OwnerAddress>-<Block Round>
  type: 'account_analytics',
  created: 12, // Block the account was added
  modified: block, // Last changed block
  owner: 'WYWRYK42XADLY3O62N52BOLT27DMPRA3WNBT2OBRT65N6OEZQWD4OSH6PI',
  "orders":{
      "6Z4EP6IGHFVPAVPD7MAKNCSB53X2FV4N23RSQATC5IGZGCAK5YX2OXOX2A": { } // Escrow Order State 
    }
  analytics: {
    'qFinal': 6, // Overall Account Score for this block, Qfinal
    'qEpoch': 5, // Sum of all qMin's for a range. Fetch from index of account_analytics
    'qBid': 1, // Show Bid Score for this block
    'qAsk': 2, // Show Ask Score for this block
    'qMin': 5, // Show Result of Bid/Ask Score for this block
    'volume_algo': 100, // On Balance Volume for Block?
    'volume_assets': {
      1234: 20,  // On Balance Volume?
    },
  },
}];

📝 Related Issues

🐛 Fix: Issue when no blocks are in DB and trying to sync

ℹ Overview


5|broker   | (Use `node --trace-warnings ...` to show where the warning was created)
5|broker   | TypeError: Cannot read properties of undefined (reading 'value')
5|broker   |     at getMaxBlock (/home/ubuntu/algodex-service/src/sync-parallel/sync-parallel.js:58:27)
5|broker   |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

📝 Related Issues

✨ Feature: Add new synced_blocks DB

ℹ Overview

This new DB will allow us to separate the state machine DBs from the append-only DBs. Basically, we can clear out this new block_sync_complete DB to allow it to know to resync all the "state machine" state. While preserving the append-only DBs instead of deleting them as well.

📝 Related Issues

🐛 Fix: Update views to use environment variables for app IDs

ℹ Overview

For Tinyman:


  const txns = getTinymanSwapTxns(
    process.env.ALGORAND_NETWORK === 'testnet' ?
      62368684 :
      552635992
    , block)

For Algodex:

Mainnet:

ALGODEX_ALGO_ESCROW_APP=354073718
ALGODEX_ASA_ESCROW_APP=354073834

Testnet:

ALGODEX_ALGO_ESCROW_APP=22045503
ALGODEX_ASA_ESCROW_APP=22045522

📝 Related Issues

✨ Feature: Create Orderbook API endpoints

Orderbook Display for Orderbook Component

Data from algodex-backend/orders.php?assetid=24891477 is currently aggregated in the frontend using this map function: https://github.com/algodex/algodex-react/blob/development/hooks/useAlgodex.js#L232 and displayed here in the Orderbook:

Capture

Reduce data sent to client in the <Orderbook /> Component. The component will fetch both sides of the Orderbook summary.

Orderbook Buy/Sell will be our first to include a page= and limit= query parameter. They also should optionally contain a valid Response Object for Infinite Scroll

Notes:

This should also add size filter for orderbook display. Size is the minimum amount a order can have to be displayed in the orderbook.

  • SIze filter comes before aggregation to limit small orders from being aggregated
  • Pagination comes after aggregation

API Route

./public-api/orderbook.php?id={assetId}&type="buy"&page=0&limit=100&size=100

Example Response:

{
  "ok": true,
  "rows": 1,
  "offset": 0,
  "data": [{
     "price": 1234,
     "amount": 23,
     "total": 100000
  }]
}

Stretch Goals:

The reduction in API responses and separation of Business/Presentation layers gets us ready for Infinite Queries (aka Infinite Scroll). This would be subsetting the responses with pagination and adding Infinite Scroll to the components.

React-Query Docs:
https://react-query.tanstack.com/guides/infinite-queries
Working Example:
https://react-query.tanstack.com/examples/load-more-infinite-scroll

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.