luajit-mm's Introduction

luajit-mm

Luajit take full advantage of lower 2G memory on AMD64 platform.

Rumdimentary implementation. Not yet fully tested. Not yet cleanup the code. Quite a few optimizatio is not yet implemented.

Immediate todo

o.Refine and finish this README.
o.test, add enhancements.

problem statement:

On Linux/x86-64 platform, Luajit can use no more than 1G memory due to the combination of bunch of nasty issues. 1G is way too small for server-side application.

This package is trying to replace mmap/munmap/mremap with hence provide up to about 2G space.

Basic ideas

o. When a application, which contain luajit, is launched, reserve the the space
   from where `sbrk(0)` indidate all the way to 2G.

o. Perform page allocation on the reserved space. the mmap/munmap/mremap is built
   on this page allocation. Currently, we use buddy allocation for page allocation
   with some optimizations in an attemp to reduce working set.

luajit-mm's People

Contributors

Stargazers

Watchers

luajit-mm's Issues

Is it possible to support valgrind?

By running luajit-mm + luajit with valgrind 3.9.0, I'm getting the following:

nginx: [error] failed to initialize Lua VM in conf/nginx.conf:129
==28642== Invalid read of size 4
==28642==    at 0x4E48216: lm_fini2 (page_alloc.h:179)
==28642==    by 0x382D00F8D9: _dl_fini (in /usr/lib64/ld-2.18.so)
==28642==    by 0x382D4390E8: __run_exit_handlers (in /usr/lib64/libc-2.18.so)
==28642==    by 0x382D439134: exit (in /usr/lib64/libc-2.18.so)
==28642==    by 0x382D421D6B: (below main) (in /usr/lib64/libc-2.18.so)
==28642==  Address 0x1fc is not stack'd, malloc'd or (recently) free'd
==28642== 
==28642== 
==28642== Process terminating with default action of signal 11 (SIGSEGV)
==28642==  Access not within mapped region at address 0x1FC
==28642==    at 0x4E48216: lm_fini2 (page_alloc.h:179)
==28642==    by 0x382D00F8D9: _dl_fini (in /usr/lib64/ld-2.18.so)
==28642==    by 0x382D4390E8: __run_exit_handlers (in /usr/lib64/libc-2.18.so)
==28642==    by 0x382D439134: exit (in /usr/lib64/libc-2.18.so)
==28642==    by 0x382D421D6B: (below main) (in /usr/lib64/libc-2.18.so)
==28642==  If you believe this happened as a result of a stack
==28642==  overflow in your program's main thread (unlikely but
==28642==  possible), you can try to increase the size of the
==28642==  main thread stack using the --main-stacksize= flag.
==28642==  The main thread stack size used in this run was 8388608.
==28642== 1,024 bytes in 1 blocks are definitely lost in loss record 634 of 658
==28642==    at 0x4A0858C: memalign (vg_replace_malloc.c:755)
==28642==    by 0x4A08651: posix_memalign (vg_replace_malloc.c:908)
==28642==    by 0x4394B7: ngx_memalign (ngx_alloc.c:57)
==28642==    by 0x41E028: ngx_create_pool (ngx_palloc.c:21)
==28642==    by 0x41C741: main (nginx.c:299)
==28642==

I'd like to contact some one for discus eventual winndows port ...

I'm developing similar solution, but I choose much more simpler
aproach, I just replace mmap/VirtualAlloc calls in lj_alloc.c
with mine cooked mmap/VirtualAlloc calls . My code at start reserve 1-2GB of low address space
and then i just give it, in use to luajit ( I let luajit use it's memory allocator, I just provide blocks of address pages on demand from previously reserved space using RESERVE/COMMIT ram trick in win and similar aproach in nix.). I'd like to discuss some aspects of mine solution in respect of
yours, maby we can share experiences.

regards Ladisalv.

Recommend Projects

cloudflare / luajit-mm Goto Github PK

luajit-mm's Introduction

luajit-mm

Immediate todo

problem statement:

Basic ideas

luajit-mm's People

Contributors

Stargazers

Watchers

Forkers

luajit-mm's Issues

Is it possible to support valgrind?

I'd like to contact some one for discus eventual winndows port ...

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent