Comments (9)
as when I graduate from university. :)
Congratulations! 🎉
I would like to take a deeper look into Moka's internal functionalities,
I wish I had more time to write design docs, so all you can read now are:
- The one that I already mentioned: https://docs.rs/moka/latest/moka/#implementation-details
- A wiki page about admission and eviction policies: https://github.com/moka-rs/moka/wiki
- And the source code.
Moka's design is heavily inspired by the Caffeine library for Java. So you can learn the base design from Caffeine's publications: https://github.com/ben-manes/caffeine#in-the-news
I started to develop Moka because I needed a concurrent cache for the storage layer of a NoSQL database (as my hobby project) and found there was no such cache library in Rust that would meet my needs. I thought I will finish implementing Moka in a couple of months so I can go back to my original project quickly. I never thought it would take some years (but I am enjoying).
from moka.
I merged #169 into the master branch. I will release it as a part of v0.9.3.
Closing this issue.
from moka.
Sorry, it took longer than I thought. But I have published Moka v0.9.3 to crates.io.
from moka.
Hi. Thank you for using Moka.
-
The expiration is
lazy
by design, so expired entries will be preserved in the cache for a short period of time.- As of v0.8.3, they will be usually removed from the cache within 300 milliseconds after expiration, but this delay can get longer if the cache is receiving very heavy reads and/or writes from clients.
- The 300 ms delay (actually a clean-up interval) is currently hard-coded, but we can make it configurable in the future.
- If you are curious about internals, there is a (little, work-in-progress) doc about it: https://docs.rs/moka/latest/moka/#implementation-details
-
There is also a known issue in an underlying library (crossbeam-epoch) that will prevent entries from dropping for much longer period after the removal from the cache.
- Our cache internally uses a lock-free hash table to store the entries and it uses crossbeam-epoch for epoch-based garbage collection (GC): https://docs.rs/crossbeam-epoch/latest/crossbeam_epoch/
- crossbeam-epoch's garbage collector is not optimal for some use cases and may keep ~100 garbages (objects waiting to be dropped) in the heap. We are hitting this issue and some entries will not be dropped immediately.
As for 1, please let me know whether you are okay with the current delay or not. We could make the delay configurable per cache.
As for 2, let me explore some ideas to resolve/mitigate the GC issue.
from moka.
Hi. Thank you for using Moka.
1. The expiration is `lazy` by design, so expired entries will be preserved in the cache for a short period of time. * As of v0.8.3, they will be usually removed from the cache within 300 milliseconds after expiration, but this delay can get longer if the cache is receiving very heavy reads and/or writes from clients. * The 300 ms delay (actually a clean-up interval) is currently hard-coded, but we can make it configurable in the future. * If you are curious about internals, there is a (little, work-in-progress) doc about it: https://docs.rs/moka/latest/moka/#implementation-details 2. There is also a known issue in an underlying library (crossbeam-epoch) that will prevent entries from dropping for much longer period after the removal from the cache. * Our cache internally uses a lock-free hash table to store the entries and it uses crossbeam-epoch for epoch-based garbage collection (GC): https://docs.rs/crossbeam-epoch/latest/crossbeam_epoch/ * crossbeam-epoch's garbage collector is not optimal for some use cases and may keep ~100 garbages (objects waiting to be dropped) in the heap. We are hitting this issue and some entries will not be dropped immediately.
As for 1, please let me know whether you are okay with the current delay or not. We could make the delay configurable per cache.
As for 2, let me explore some ideas to resolve/mitigate the GC issue.
Thanks for your reply, I think preserving an evicted connection for less than 3s is ok (after all the connection pool is just a toy project), and it seems hardly occur.
And I would like to take a deeper look into Moka
's internal functionalities, as when I graduate from university. :)
from moka.
crossbeam-epoch's garbage collector is not optimal for some use cases and may keep ~100 garbages (objects waiting to be dropped) in the heap. We are hitting this issue and some entries will not be dropped immediately.
This one could be something we are actually noticing. Our cache entries are quite large and not that many so having about ~100 of them hanging around might look like a memory leak to us. Is there any way to mitigate this?
from moka.
Sorry for not getting back to you sooner.
I wonder if we could mitigate it by using the flush
method of crossbeam_epoch::Guard
.
https://docs.rs/crossbeam-epoch/0.9.9/crossbeam_epoch/struct.Guard.html#method.flush
Clears up the thread-local cache of deferred functions by executing them or moving into the global cache.
Call this method after deferring execution of a function if you want to get it executed as soon as possible. Flushing will make sure it is residing in in the global cache, so that any thread has a chance of taking the function and executing it.
We use some "deferred functions" to drop entries removed from cache.
Let me try it in next few days. I will measure performance impacts for both memory usage and speed.
from moka.
No worries, thanks for getting back to me.
Thank you for looking into it! Sounds like an approach that could work.
from moka.
As for the flush
stuff, I am working on it here: #169
from moka.
Related Issues (20)
- Question about usage with channels to create a Stream like interface HOT 6
- can you please add an example of concurrent modification in Moka? HOT 2
- Add an example to show how to run pending tasks in an interval HOT 3
- Add a way to pass the exact eviction time to the eviction listener? HOT 1
- Add support for something like an event handler, which can be used to refresh expiring entry?
- entries are leaked when sync::Cache is dropped HOT 5
- Try `seize` as a replacement of `crossbeam-epoch`? HOT 3
- Refactoring: Cache policy implementations HOT 2
- Do not swallow the panic when the eviction listener is panicked
- Crash in iOS HOT 3
- Unable to use moka when miri testing HOT 3
- Inefficient size aware eviction when inserting new entries with larger size HOT 3
- Consider switching to TinyUFO HOT 2
- Pending `run_pending_tasks` may cause busy loop in `schedule_write_op` HOT 14
- Miri errors on the `timer_wheel` module. (Miri v0.1.0 8b2459c1 2024-04-09) HOT 1
- CI: Kani verifier v0.49.0 uses Rust `nightly-2024-03-29`, which cannot compile `[email protected]` HOT 2
- Should Moka work in WASM? HOT 2
- Size Aware Eviction: First-In-First-Out, or evicting largest items first? HOT 2
- Latest triomphe update breaks MSRV
- The `once_cell` dependency should be optional
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from moka.