andreasbriese / bbloom Goto Github PK

Casting to a uintptr, saving the uintptr, and then casting it back is unsound.

Hi Andreas,

congrats for your work on bbloom, I am integrating it at the core of my new project.
I was wondering if you were planning to add methods to calculate union and intersection of 2 bloom filters or if you could put me in the right direction to learn how to do it.

Let me know, best,
Romain.

This may be a stupid question, but I'd very much appreciate your help. I want to implement a page view counter that will be pushed to a database every X hours. Before it gets pushed, then I intend to store it in memory and make sure that the same device (checked using IP + UA) doesn't count for multiple views (unless 1000 other people have visited the page, or it has been X hours since last visit)

I have the following code:

package main

import (
	"fmt"
	"sync"
)

type page struct {
	mu     *sync.Mutex
	values map[string]int64
}

var pageCounters = page{
	values: make(map[string]int64),
}

func (p *page) Delete(key string) {
	p.mu.Lock()
	defer p.mu.Unlock()
	delete(pageCounters.values, key)
}

func (p *page) Get(key string) int64 {
	p.mu.Lock()
	defer p.mu.Unlock()
	return p.values[key]
}

func (p *page) Incr(key string) int64 {
	p.mu.Lock()
	defer p.mu.Unlock()
	p.values[key]++
	return p.values[key]
}

func main() {
	_ = pageCounters.Incr("/some/url/here")
	_ = pageCounters.Incr("/another/url")
	_ = pageCounters.Incr("/some/url/here")
	
	fmt.Println(pageCounters.Get("/some/url/here"))
	fmt.Println(pageCounters.Get("/another/url"))
	fmt.Println(pageCounters)
	pageCounters.Delete("/another/url")
	fmt.Println(pageCounters)

}

The plan now is to update the page struct to include a bloomfilter field, I'm thinking it would be done like this:

type page struct {
	mu     *sync.Mutex
	values map[string]int64
	bloom  bbloom.Bloom
}

I'm now wondering how I should update the Incr func so that:

• if a bloom filter doesn't exist, then it will create one containing 1000 items
• each item will be a combination of user IP address + User Agent (curious how you would recommend storing it?)
• when the bloom filters items have been filled up, then the bloom filter should either be deleted and replaced by a new bloom filter, or have the previous items be overridden (I assume this is impossible)

I'm thinking it would look something like this:

func (p *page) Incr(key, ipUA string) int64 {
	p.mu.Lock()
	defer p.mu.Unlock()
	if (p.values[key] == 0) {
		p.bloom = bbloom.New(1000.0, 7.0) // probably use wrong values here
	}
	added := p.bloom.AddIfNotHasTS([]byte(ipUA))
	if added == true {
		p.values[key]++
	}
	return p.values[key]
}

Right now then I don't really know what will happen if the bloomfilter is full, and I'm not sure how to check for this and how to replace it. I guess if (p.values[key] == 0) could become if (p.values[key] == 0 || p.bloom.Size == 999)?

I also wonder, would there be any benefit to store the bloom as JSON? Keep in mind that each website page will have its own filter and they'll only be deleted after X hours (meaning there could potentially be many thousands of page structs initialized at any given moment)

I hope you don't mind me abusing your repo & time with this question

I want to use this in one of my projects but have to make sure the license is compatible

Is there a reason for it?