reecepbcups / cosmos-cache Goto Github PK

My redis instance took up all my memory since it kept saving all keys to a db. We need to disable this

code /etc/redis/redis.conf

# maxmemory <bytes>
maxmemory 16gb

# maxmemory-samples 5
maxmemory-samples 5

# stop-writes-on-bgsave-error yes
stop-writes-on-bgsave-error no

# then comment out the following lines
save 900 1
save 300 10
save 60 10000

systemctl restart redis-server

Iterate each in order of request array, pair to return values (which comes back in the same array order)

in the .env, comma seperated.

I guess Ill state sync another node?

in the Req_handler.py, randomly select one.
If bad, time it out in memory for X period of time.

Allow backup to be round robin? Maybe in the future

clear non long term data (blocks, Txs, etc) on the mint of a new block
ensures data is up to date within time params of block production

allow configuration for networks like EVMOS with 1-2s blocks

In config:
0 = no cache
-1 = disabled
-2 = clear cache when new block is made

docker run, run Txs and queries, ensure data matches. Good

webhooks, given a list of endpoints (normal server + cache) query, check, on timeout / error post

Just edit .env and cache_times.json (not pretty yet)

for best performance

# == QUERY INCREMENT LOGGING ===
ENABLE_COUNTER=false
INCREASE_COUNTER_EVERY=250
STATS_PASSWORD="" # blank = no password   -- https://rest_endpoint/stats?password=mypass

required redis to be installed. Default config is private no password / outside connection

point the Reverse proxy to it instead of your RPC / REST API endpoints

This will break Keplr support, but handles normal requests for webapps fine

./run_rpc.sh has all the info you need to setup the the service, and start
RPC is required to restart 1 time per day via crontab for memory with redis for now

WORKERS * THREADS = the number of CPU threads you have. Threads is more important than workers. You can increase the other 2 as much as you want depending on request amount

Example of basic Sei RPC page (customized by Sei):
https://sei-rpc.lavenderfive.com/

When trying to access the Cosmos Cache version, there is an "Internal Server Error":
https://sei-rpc-cached.lavenderfive.com/

The sub pages work just fine:
https://sei-rpc-cached.lavenderfive.com/status

I think it's possible regex is breaking / or something else related to Sei's customizations

This may help: https://medium.com/@jgleeee/sharing-data-across-workers-in-a-gunicorn-flask-application-2ad698591875

In the RPC POST, if the method is method != "abci_query" we can NOT cache it for some reason

error: response id (0) does not match request id (1)

temp fix is method = abci_query, we do not cache it.

In the future, we should only allow specific paths (via .env) so we can bypass? ex: /cosmwasm.wasm.v1.Query/SmartContractState, Account. Balances, etc

Hit endpoints with configurable data (status, abci_query, etc) on the RPC

curl, python, TS, go - what ever is easiest
dockerize it

can then launch on akash with like 20 instances to spam requests at a node

Easy solution:

• move RPC from 26657 to another port
• launch run_rpc.sh on 26657
• this way you don't have to alter nginx / change efirewall

can we still use gunicorn? or do need to use univicorn instead. Still multithreaded?

(removes need of jsonify'ing data)

Ensure to use httpx properly with async requests rather than blocking

Store code: https://github.com/Reecepbcups/py_kvstore

Ex: https://github.com/DA0-DA0/rpc-cache-cf-worker/tree/main

dont leak intel to node queries

Could be cool based off user requests

• On load, get all possible REST endpoints via the openapi. Use * regex for any { } areas
• if user requests a regex match, forward to 1317
• else 26657 rpc / websocket req

Compile RPC links -> open api, and inject into REST openapi. Place on top of the REST api UI

IP address or special query data required?
JSON file.

Could have groups which can be applied. not sure this makes sense though since a node op can just have a seperate A record pointing to the machine's non cached endpoint

Save all blocks to a local DB (sqlite, mongodb, JSON?)

When does this happen? What endpoint? The try catch needs to output more info

Rn it just errors

ermove / from the url and backup rul at the end

https://computingforgeeks.com/how-to-install-python-on-ubuntu-linux-system/

normal url for standard (machine)
fail over to akash, fails asdnokasnodjkasjniod.dns.com

stats page %'s are off unlike previous versions since I changed it to enums