Comments (10)
Hi @tatsuya6502 ,
Thanks for looking into it.
The "keys() -> Vec" would work if it is easier...
JR
from moka.
@tatsuya6502 ,
Excellent..
I'll be testing it here in the next few day's...
I'll let you know how it goes.
Thanks for fixing it...
JR
from moka.
Hi. Currently (v0.8.1), moka::future::Cache
does not provide a way to iterate through all entries or retrieve all keys in a cache. There is an experimental cache implementation called moka::dash::Cache
that has iter()
method, but it is not optimized for asynchronous use.
Let me spend a few days to explore some options to provide either iter()
or keys()
method.
Providing such a method in a concurrent collection will be much more complicated than in a not-thread-safe collection (e.g. std::collections::HashTable
). This is because concurrent collections allow updates from other threads while some threads are iterating. So some experimentation will be needed.
I will get back to you when I decide what to do.
from moka.
I believe I found a good solution for providing iterators. Although it has some disadvantages (below), I hope it will be good enough for many use cases:
Disadvantages:
- It's
next
method may not (or may) return some entries (key-value pairs) if they have inserted after the iterator was created. - It has some memory overheads as it creates and holds a snapshot of keys (
Vec<std::sync::Weak<K>>
) of an internal hash table segment:- Each cache has 64 segments (hard-coded since Moka v0.1.0), except
sync::SegmentedCache
. - An iterator creates and hold one snapshot of only one segment at a time.
- So this will consume less memory than creating/holding a snapshot of the entire hash table.
- Each cache has 64 segments (hard-coded since Moka v0.1.0), except
Advantages:
- It does not reduce concurrency.
- Allows concurrent reads and writes on the cache while iterators are alive.
- Its implementation is simple and reliable.
I found implementing such an iterator requires about the same amount of work to implement keys()
-> Vec
method. Since the iterator is more commonly used in Rust libraries and it will consume 64 times smaller amount of memory than keys()
, I took the iterator path (#114).
from moka.
@tatsuya6502 ,
Iterator works for me..
if it implements the 'core::iter::Iterator' trait, that would be awesome..
Thanks for looking into it...
JR
from moka.
Yes. The iter
method returns an iterator, which implements the core::iter::Iteranor
trait. Also &Cache
implements the core::iter::IntoIterator
trait.
So you can either do this (iter
method):
Line 1455 in 52c9f55
or this (IntoIterator
):
Lines 1491 to 1495 in 52c9f55
from moka.
Sweet...
from moka.
@JRAndreassen — I am going to merge #114 with the iterator supports. Then, I will do some pre-release testing and release v0.8.2 in a day or so. I will let you know when v0.8.2 is released.
from moka.
@tatsuya6502 ..
Super awesome..
JR
from moka.
Hi @JRAndreassen — OK. I have published Moka v0.8.2 to crates.io. Please let me know if you have any questions.
from moka.
Related Issues (20)
- entries are leaked when sync::Cache is dropped HOT 5
- Try `seize` as a replacement of `crossbeam-epoch`? HOT 4
- Refactoring: Cache policy implementations HOT 2
- Do not swallow the panic when the eviction listener is panicked
- Crash in iOS HOT 3
- Unable to use moka when miri testing HOT 3
- Inefficient size aware eviction when inserting new entries with larger size HOT 3
- Consider switching to TinyUFO HOT 2
- Pending `run_pending_tasks` may cause busy loop in `schedule_write_op` HOT 14
- Miri errors on the `timer_wheel` module. (Miri v0.1.0 8b2459c1 2024-04-09) HOT 1
- CI: Kani verifier v0.49.0 uses Rust `nightly-2024-03-29`, which cannot compile `[email protected]` HOT 2
- Should Moka work in WASM? HOT 2
- Size Aware Eviction: First-In-First-Out, or evicting largest items first? HOT 2
- Latest triomphe update breaks MSRV
- The `once_cell` dependency should be optional
- Update Key With One Evaluation of the `init` Future HOT 12
- Feature Request: Alternative compute API that skips if other thread/async task is already computing for the key
- Locking `triomphe` is the wrong approach to preserving MSRV HOT 1
- CI: Run Linux arm64 tests only once a week on `arm1.small` instance at AWS CodeBuild
- Use SIEVE as underlying LRU?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from moka.