beacon-biosignals / keywordsearch.jl Goto Github PK

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Document's could be TextAnalysis.StringDocument and Corpus could be TextAnalysis.Corpus?

• This would mean KeywordSearch complements TextAnalysis by providing fuzzy-matching capabilities its StringDocument types and Corpuses of StringDocuments.
• It would also mean that it's easy to do other types of analyses (provided by TextAnalysis) beyond keyword searching on documents loaded into this library, since they would already be TextAnalysis.Corpuses.
• Currently, we allow attaching arbitrary NamedTuple metadata to our documents or corpuses, which I use for storing document UUIDs and corpus UUIDs. This maybe could be upstreamed to maintain the functionality.
• This would also enlarge the dependency tree of KeywordSearch, since we would need to add a dependency on TextAnalysis (https://juliahub.com/ui/Packages/TextAnalysis/5Mwet/0.7.1?t=1).

This would be for the future (not the 0.3 release I want to get out soon).

MWE:

julia> original = "abcdef"
"abcdef"

julia> max_dist = 5
5

julia> for i in 2:length(original)
           modified = "$(original[1:i-1])X$(original[i:end])"
           @info modified => (; _findmin=KeywordSearch._findmin(original, modified, Partial(DamerauLevenshtein()); max_dist),
                                Partial=Partial(DamerauLevenshtein(max_dist))(original, modified))
       end
[ Info: "aXbcdef" => (_findmin = (1, 2:7), Partial = 1)
[ Info: "abXcdef" => (_findmin = (2, 1:6), Partial = 1)
[ Info: "abcXdef" => (_findmin = (2, 1:6), Partial = 1)
[ Info: "abcdXef" => (_findmin = (2, 1:6), Partial = 1)
[ Info: "abcdeXf" => (_findmin = (1, 1:6), Partial = 1)

Instead of just strings. This might make some common queries easier to write (and possibly more performant!). Of course, if you only need regexes, there might be other tools available, but with this a user could mix regexes and things like FuzzyQuerys.

Just wanted to signal that the distance DamerauLevenshtein defined in the StringDistances actually corresponds to the Optimal String Alignement distance.

In StringDistances 0.11, DamerauLevenshtein will return the true Damerau-Levenshtein distance between two strings. You can use OptimalStringAlignement to get the old version of DamerauLavenshtein.

I want to make a breaking 0.3 release and register it in General. First...

• Improve docs, add quick tutorial (#8)
• Add docs badges (#8)
• Rename to Document and Corpus (#4)
• Remove And (#5), since it's breaking so better to do it now
• Improve table interface (#7) since it's breaking

With FuzzyQuery's:

it looks like FuzzyQuery doesn't respect word_boundary? It allows the edits to occur on the leading space
@ararslan

With augment:

julia> word_boundary(augment(Query("hello goodbye")))
ERROR: MethodError: no method matching reconstruct(::KeywordSearch.Or{Tuple{Query, Query}})
Closest candidates are:
  reconstruct(::FuzzyQuery) at /Users/eph/KeywordSearch.jl/src/helpers.jl:34
  reconstruct(::Query) at /Users/eph/KeywordSearch.jl/src/helpers.jl:35
Stacktrace:
 [1] word_boundary(Q::KeywordSearch.Or{Tuple{Query, Query}})
   @ KeywordSearch ~/KeywordSearch.jl/src/helpers.jl:75
 [2] top-level scope
   @ REPL[39]:1

julia> augment(word_boundary(Query("hello goodbye")))
Or
├─ Query(" hello goodbye ")
├─ Query("hello goodbye ")
├─ Query(" hellogoodbye ")
├─ Query("hellogoodbye ")
├─ Query(" hello goodbye")
├─ Query("hello goodbye")
├─ Query(" hellogoodbye")
└─ Query("hellogoodbye")

The first case errors, which is annoying, and the 2nd case makes way to many things; we only want to augment the inner space, not the outer ones.

PR is matthieugomez/StringDistances.jl#44, this issue is just to track it from this side