Unique Mode is Very, Very Slow about purr HOT 3 OPEN

Hey! This is actually intentional for now. Until I get both the bugs filed for this fixed, I'm hesitant to advertise this in the help, as it's really more of an in-development feature at this point.

Thanks for pointing it out, though! I had forgotten that I excluded it and I need to remember to add it when I get it fully done.

from purr.

this mode (ctrl-u) is also not mentioned in the readme nor in the preview-help....

from purr.

Massive parallelization is hard here because each streams needs the messages to come out in order. Unfortunately, just running the messages through a while loop in zsh with tail is pretty slow at about 1000QPS in Linux. That effectively caps how quickly we can process each message, and the overhead on each pipe IO operation pushing between threads is also very slow; 1ms per query is prohibitive.

There's a bunch of fancy solutions I can do here if I want to keep this pure zsh. Thread coordination is hard, but it's not impossible. But I think that's not required; I'm hammering screws at this point.

We already declare a dependency on perl. I think the best thing to do is to bundle a trie implementation in perl and do all the processing in a sub-process running perl, which should be much much faster to begin with, and perl is pretty good at text manipulation and string comparison to begin with.

from purr.