See etcd-io/bbolt#201 for more details.

HEAD branch is master, not main. This makes automating things harder.

Any reason to not move to main?

I've been using bitmapist for storing user activity for quite some time. But apparently some data despite still being in the database is not accessible anymore since a couple days ago. I'm backing up the data periodicallyevery day and also checking out the data representation over the web app, but now data regarding last 3 months, since may 3rd to be precise, is not accessible anymore.

I've also checked the daily backups back to july and i can't access the data there either even though i could read the data over the app and did not have any problems those days.

This is how i check the backups:

➜  ~ ~/.golang/bin/bitmapist-server -addr localhost:6380 -debug -db 2018_07_29_bitmapist.db 
2018/08/26 20:50:40 loading data from 2018_07_29_bitmapist.db
2018/08/26 20:50:40 loaded in 4.656269ms
2018/08/26 20:50:45 conn:1	[COMMAND]
2018/08/26 20:50:47 conn:1	[keys trackist_user:join_2018-5*]
2018/08/26 20:53:40 periodic saving...
2018/08/26 20:53:40 saved in 36ms
2018/08/26 20:56:40 periodic saving...
2018/08/26 20:56:40 saved in 30ms

....

➜  ~ redis-cli -p 6380                                    
127.0.0.1:6380> keys trackist_user:join_2018-5*
1) "trackist_user:join_2018-5-2"
2) "trackist_user:join_2018-5-1"
3) "trackist_user:join_2018-5"
4) "trackist_user:join_2018-5-3"
127.0.0.1:6380> 

Go: v1.10
Bitmapist server: 1.0.1 and latest

I know the data exists in the db and backups, since i was able to read it online but is there any other way to access the data to confirm ?

Hi.

Interesting project we are looking at to implement in a new project. Still in its infancy it seems, but we are starting to look at things we had with redis like clustering. Considered having a 24hr file and then main file for persistence? This allows a fail-over scenario that you can recover by copying main file to new cluster node and systematically send the chunks in the 24hour file to the new node.

I am very keen to write some code and test clustering scenarios. Currently our work around is using a streaming service like kafka or Kinesis (AWS) to stream to 2 nodes. One node for active usage and another for hot-standby. So the streaming would be consumed by a client on the same node and written to bitmapist-server via python client lib.

Thanks
Christo Goosen

Took me ~30 minutes to figure out how to make a Dockerfile for Golang applications.

I don't even know if this is the best way.

FROM golang:1-stretch

ENV GOPATH=/go
ENV BACKUP=/app/backup
ENV DATABASE=/app/bitmapist.db
EXPOSE 6379

RUN mkdir /app

RUN go get -v -u github.com/Doist/bitmapist-server

CMD /go/bin/bitmapist-server -addr 0.0.0.0:6379 -bak $BACKUP -db $DATABASE

The workflow docker.yml is referencing action actions/checkout using references v1. However this reference is missing the commit a6747255bd19d7a757dbdda8c654a9f84db19839 which may contain fix to the some vulnerability.
The vulnerability fix that is missing by actions version could be related to:
(1) CVE fix
(2) upgrade of vulnerable dependency
(3) fix to secret leak and others.
Please consider to update the reference to the action.

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

github.com/Doist/bitmapist-server/internal/bitmapist: cannot find module providing package github.com/Doist/bitmapist-server/internal/bitmapist

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

github.com/Doist/bitmapist-server/internal/bitmapist: cannot find module providing package github.com/Doist/bitmapist-server/internal/bitmapist

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

github.com/Doist/bitmapist-server/internal/bitmapist: cannot find module providing package github.com/Doist/bitmapist-server/internal/bitmapist

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

github.com/Doist/bitmapist-server/internal/bitmapist: cannot find module providing package github.com/Doist/bitmapist-server/internal/bitmapist

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

It would be nice to have binary builds for new releases. This way people would be able to start using the server without the need to install Go.

Newer versions of the Python Redis client implement a health check ref that send a PING command to the Redis server.

It's enabled by the setting to non-zero an argument health_check_interval for the Redis client. It would be nice to have support for the PING command in the bitmapist-server so that automatic health checks could be enabled on the client-side in bitmapist.

I wanted to document the usage of bitmapist-server from Docker and found out that it's challenging.

As it turns out, github package registry is quite useless for public builds ("docker pull" doesn't work).

Likely, GitHub fixes this issue soon (they promised in the thread), and we can simply put it on hold until then, or we can migrate to Docker Hub.

It would be nice to have a -v, -version or --version flag to return back the current version of bitmapist-server. Coupled with a proper changelog (#11), it would greatly simplify the life of third-party project users.

The output of Redis KEYS is always a list, even if there are no matches. In the same case, bitmapist-server returns Null (converted to None in Python).

How to reproduce (Bitmapist-server behavior)

Start bitmapist-server. Then run the Redis client against the empty database

$ redis-cli
127.0.0.1:6379> keys foo*
(nil)

What I expect (Redis behavior)

$ redis-cli
127.0.0.1:6379> keys foo*
(empty list or set)

Relevant

• Doist/bitmapist4#2
• Doist/bitmapist#31
• https://redis.io/commands/keys

Hi,

I'm thinking of using bitmapist-server but am unsure how stable it is. Are you using it in production? With how much data? Can you recommend using it for live data, or would you say it's still too early?

Thanks,
Michael

Hi

Is the vendor directory for version pinning? Can we not use something like Go Package Manager and a file with all the dependencies?

Thanks
Christo

Thank you for the fantastic implementation. The RAM savings it achieves are amazing.

I have one problem: It seems that the regular "periodic saving" calls block the server. In my case, this leads to regular spikes in response times from <0.03s to 5s. Is there a way to prevent this blocking behavior while still regularly persisting the database?

I think the project deserves a proper changelog. It would be easier to communicate changes, especially when it comes to issues like #10