nursik / go-expire-map Goto Github PK

As it is mentioned in docs, expiration rate is limited to 2200 keys per second. So if there is a high unique keys insertion rate with small TTL (less than 1 hour), it is possible that map will consume a lot of memory, because it stores a lot of expired keys. So there are several improvements, which can solve this issue.

• Statistics - we can count number of insertions, deletions and etc. and using this statistics make some decisions.
• Methods - add new methods like ForceCheck (forces to check all keys for expiration)
• Expiration rate tuning - add new methods, which can change default constants used for expiration strategies (like expired ratio trigger threshold, number of keys per rotation and etc)
• Better strategies - changing current strategies or adding new one. For example, we can use statistics of insertion and expiration, distribution of unique TTLs and etc, and based on this information make a better decision.

Some tests are failing on my setup:

=== RUN   TestExpireMap_StoppedAndClose                     
--- PASS: TestExpireMap_StoppedAndClose (0.01s)
=== RUN   TestExpireMap_Curtime
--- PASS: TestExpireMap_Curtime (5.00s)
    expire_map_test.go:429: Curtime() - average diff 481423.750, max diff 1945113 (in nanosecs)
=== RUN   TestExpireMap_Curtime_WithHeavyLoad
--- FAIL: TestExpireMap_Curtime_WithHeavyLoad (10.01s)
    expire_map_test.go:489: Curtime() - average diff exceeded: max average diff 1000000.000, average diff 4419762.797 (in nanosecs)
=== RUN   TestExpireMap_GetAll
--- PASS: TestExpireMap_GetAll (1.01s)
FAIL
exit status 1

I'm new to Golang but run into problem getting short-lived codes using go-expire-map.

We create the map:

    exMap := expiremap.New()
    defer exMap.Close()

and use it in two endpoints. In the first, we generate random code, put it into the map and pass the code to the user:

    code := c.codes.Generate()
    c.exMap.Set(code, true, time.Minute)

In the second, we check if the code is in the map. If it is, we delete it from the map and do our logic:

    _, found := j.ExMap.Get(code)
    if (!found) {
        log.Printf("Not found code: %s", query)
        w.WriteHeader(http.StatusBadRequest)
        return
    }
    j.ExMap.Delete(code)
   // our logic

Occasionally, I observe that when I check existence of the code returned to me earlier, the code is not found in the map. But the second check for the same code finds it.

Is this a bug or I use the library wrongly?

Since GO111MODULE=on is gonna be default option starting from 1.13, consider adding modules support.

We may want to collect some metrics from a map, for example, rate of insertion, rate of expiration (active vs passive) etc.