Possible errors in metric computations about stringmetric HOT 5 CLOSED

I'll dig into this a bit later today. Thanks for the submission!

from stringmetric.

There are a couple different ways to intersect/union sets. One, which you describe, is by exact position (like bitwise ops). The other, which Scala uses, is more loose. I've seen the algorithms both ways, but more often I see the loose version used:

http://www.planetcalc.com/1664/

Besides all that, there is a bug in the Jaccard algorithm that has been fixed: 6068b4b. It should have been:

intersection set length / (set1 length + set2 length - intersection set length)

I was missing the latter operator.

from stringmetric.

I could make a bitwise type jaccard, dice, etc. Would that be of interest?

from stringmetric.

It would be nice to have the more traditional similarity metrics for bit sequences. I believe that many similarity metrics were initially described as so. Although I must say that if you are focusing on strings (as sequences of alphanumeric letters), as the name "stringmetric" suggests, you're approach is probably more in line with what someone using your library might expect. I say "probably" because I don't actually know how these metrics are supposed to work in that case, namely what union, intersection, length and cardinality might mean.

In a previous response, which I'm now not seeing in this thread, you hinted at representing a binary sequence as a long and then make bitwise operations. I would just point out that if you do so, you are limiting the sequence length to the number of bits available in the internal representation of a long, which depends on, well, internals (e.g. the system's architecture). Java's BitSet overcomes this using an elastic array of long words, not without it's caveats. Or you could simply use a string of 0s and 1s - which one is more efficient is hard to tell.

from stringmetric.

Agreed on the alternate intersect/union implementation on these couple metrics.

• The definition of set intersection in the scala collections library (which is more of a filter): https://github.com/scala/scala/blob/master/src/library/scala/collection/GenSetLike.scala#L57
• The definition of set union in the scala collections library:
  https://github.com/scala/scala/blob/master/src/library/scala/collection/GenSetLike.scala#L74

You are spot on about the representation of the data structures. The original idea (which I did not detail) was to back the sets with longs and then perform bitwise operations, only should they be able to be represented properly. Else, bitset. However, I would still like to more than just bits to be used. There are some uses with strings and also with specific domain enumerations, like DNA.

from stringmetric.