Database explosion - longRunner possible bug. about nodejs-pool HOT 10 CLOSED

For pools is tiny, ~ 10 miners, number of shares are small. So Gb means abnormal.

from nodejs-pool.

This pool db abnormal increase happens in vanilla Snipa pool code as well. It happens when you restart worker module in the middle of DB transaction (read or write, I'm not sure). DB increase is not very harmful since basically it only increase DB size up to 24GB filling it with zeros and this can be later reverted by using mdb_copy with shrink option. Also this increase do not really consume space in share table so it really only increase number of free pages in DB.

from nodejs-pool.

Thank for your info (I’ll keep an eye on your finding). Now I understand better for Graft case. Graft block time is 2 minutes. But it is under attacks and block time is 5 hours or more. So, 1 block shares database is equal 150 normal blocks. So the pool will record huge amount of shares. Database explosion to limit.

from nodejs-pool.

Do not worry about DB explosions. If you use share accumulator code then you can fit around 2 weeks of shares for MO size pool in 24 GB without much problems.

from nodejs-pool.

@bobbieltd You can decrease max size.
change this in local_comms.js
this.initEnv = function(){
global.database.env = new this.lmdb.Env();
global.database.env.open({
path: global.config.db_storage_path,
maxDbs: 10,
mapSize: 24 * 1024 * 1024 * 1024,

if you change to 4
then
but you after this you need to dump base msb_dump -f dump -a pool_db
and edit file dump
change all mapsize to 4294967296
then msb_load -f dump pool_db_new

i have changed it on all my pools.

from nodejs-pool.

@1rV1N-git Thanks Cryptopool for tweak. It could have helped me not to delete pool_db in emergency to give disk space to daemon.
I’m using MoneroOcean merged shares codes. So it’s abnormal to have that size. I already restarted all moddules and restarted server and reducing shareMulti / shareLog but it kept growing. longRunner was stuck by some how (no error log). If it rehappens again, I’ll debug it.

from nodejs-pool.

I do not know enough about LMDB internals to provide fix for this DB implosion that happens sometimes due to worker module restart. Hence the mark wontfix. But let this issue be open.

from nodejs-pool.

@1rV1N-git When you dumped your DB was it the original size or was it smaller?

from nodejs-pool.

@1rV1N-git Thanks for your help, I already try to tweak local commit for DB open size limit.
I run into this big database growing again when ETN got attacked with block time > 2 hours. But when block time comes back to normal, the database get back to normal. So I guess the problem happens when block time is abnormal.

from nodejs-pool.

#43 Had a similar issue and the commit on that thread fixed it.

from nodejs-pool.