Comments (4)
Sure. Here is my constructor:
func NewBigCache(config *config.Config, datadog dd.ClientInterface) IBigCache {
bcconfig := bigcache.Config{
Shards: 1024,
LifeWindow: 300 * time.Second,
CleanWindow: 5 * time.Minute,
MaxEntriesInWindow: 1000 * 10 * 60,
MaxEntrySize: 500,
Verbose: true,
HardMaxCacheSize: config.BigCache.MaxMB, // 600MB
OnRemove: nil,
OnRemoveWithReason: nil,
}
cache, initErr := bigcache.NewBigCache(bcconfig)
if initErr != nil {
log.Fatal().Msgf("error bigcache init: %s", initErr.Error())
}
go func() {
for {
time.Sleep(time.Duration(5) * time.Second)
datadog.Histogram("bigcache_keys", float64(cache.Len()), []string{}, 1)
datadog.Histogram("bigcache_hits", float64(cache.Stats().Hits), []string{}, 1)
datadog.Histogram("bigcache_misses", float64(cache.Stats().Misses), []string{}, 1)
}
}()
return BigCache{
cache: cache,
enabled: config.BigCache.Enabled, // true
datadog: datadog,
}
}
And here is getting a key:
func (b BigCache) GetVendors(ctx context.Context, vendors []string) ([]models.Vendor, []string) {
span, newCtx := tracer.StartSpanFromContext(ctx, "bigcache.GetVendors", tracer.ResourceName("bigcache"))
ctx = newCtx
defer span.Finish()
if !b.enabled {
return []models.Vendor{}, vendors
}
leftOver := make([]string, 0, len(vendors))
out := make([]models.Vendor, 0, len(vendors))
for _, v := range vendors {
key := b.GetKeyName(v)
value, err := b.cache.Get(key)
if err != nil {
leftOver = append(leftOver, v)
continue
}
tmp := models.Vendor{}
err = json.Unmarshal(value, &tmp)
if err != nil {
log.Ctx(ctx).Error().Msgf("error while unmarhsaling for key from BigCache %s: %s", key, err.Error())
leftOver = append(leftOver, v)
continue
}
out = append(out, tmp)
}
return out, leftOver
}
from bigcache.
Could you share your config and example payload to help reproduce it?
from bigcache.
OK, looks like a bug. If I understand correctly, bigcache allocates array every 1 ms but instead of growing array (which will be expected) it allocates the same size. What is not clear for me is the HardMaxCacheSize
what's exact value passed here? Maybe it's a typo and instead of 600MB we have 614400
and that will explain why it does so. If this is the case then we should add a check if we are really growing the array and if not skip it and do not display confusing message.
if q.capacity == q.maxCapacity {
return
}
Lines 108 to 116 in cffbf48
from bigcache.
In that value I am passing this number: 600. It was meant to use 600MB of RAM to cache stuff. Is not a plain int of megabytes I have to provide there?
from bigcache.
Related Issues (20)
- LifeWindow Not valid HOT 1
- Error logs about bytes allocation? HOT 1
- why did my key expired HOT 7
- resetKeyFromEntry HOT 1
- Excessive memory allocation HOT 1
- The index(int) of entries(queue.BytesQueue) overflows the hashmap's uint32 value HOT 4
- Memory so high, and when clean not reduce size HOT 5
- Benchmark is not fair, So the speed can't be trusted HOT 4
- Whether it is a bug ? HOT 1
- Why doesn't bigcache consider designing expiration time for each cache key? HOT 4
- How does bigcache handle hash collisions HOT 1
- Support for the map? HOT 1
- compile a 32-bit LSB executable on arm and Execution error-> panic: unaligned 64-bit atomic operation HOT 4
- Is there a memory leak in bigcache? HOT 4
- Add multi-level cache design HOT 2
- Why do we need to configure both lifeWindow and cleanWindow? HOT 2
- GRPC server support HOT 4
- feature: Support uint64 keys without converting to string then back again HOT 6
- Proposal to Integrate SIEVE Eviction Algorithm
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bigcache.