purebred-mua / hs-notmuch Goto Github PK

Waiting on language-c to relax its bounds (visq/language-c#95), so that c2hs can build. After that, we should be good to go.

see notmuch-search-terms(7).

To fix purebred-mua/purebred#99 hs-notmuch needs to expose notmuch_thread_get_total_messages.

We have to convert the SearchTerm to a C string to pass to libnotmuch.
Converting it to String then marshalling to CString is undesirable.

Ideally should use Builder to build the bytestring, convert to lazy bytestring, then convert
to strict bytestring. If we null terminate the string we can use Data.ByteString.Unsafe.unsafeUseAsCString to avoid the final copy that would be
incurred by B.useAsCString.

The query_create function should be updated to use B.ByteString at the same time.

Currently there is no way to have a stable query result (I'm hypothesizing). The binding already provides query_set_sort, but it isn't exposed in hs-notmuch.

expose functions for add/remove tags. This depends on #10 (r/w database mode).

I've tried composing messages with more than just a subject and it ended in a core dump. Basically this:

Mail <$> messageHeader "Subject" m <> messageHeader "To" m<> messageHeader "From" m

ends up in a segfault. Maybe I'm referencing the wrong headers? I did not manage to get a back trace, but I'm kind of hoping you might know where the problem lies?

It would be good to have a nice way to express the version of libnotmuch in use.
This can help to distinguish what code paths are in use when debugging issues.

In order to display something meaningful, hs-notmuch should parse date headers as date types.

While working on purebred-mua/purebred#119 I can now write mails into a Drafts directory. But they're not indexed just by writing them there. Unless I do a subprocess call to notmuch new which will have other side effects, I think it would be good to leverage the use of hs-notmuch and expose the API to add messages with the given tags to the index. That way I can pick draft mails up again with a simple search (e.g. tag:draft)

Observe that message_get_header will be called with a small number of strings
(headers), but will be called many times.

Currently, every time message_get_header gets invoked it uses B.useAsCString which
allocates (length s + 1) bytes, copies the string (which will be null terminated), and calls the action.

It would be better to define a new type for header names that holds a pointer to a null-terminated
string that can be used without further ado. Conversion from B.ByteString would be O(n) but
if a distinguished value is used everywhere, the cost need only be incurred once.

A similar approach should be used for tag names, too.

When we get a list of tags, we do a string copy of every tag. But there will be a relatively small
number of tags in a database. Investigate whether interning them is a win.

See also the http://hackage.haskell.org/package/intern package.

The first thing here is to determine what versions of notmuch we need to support.

A quick survey of what's available on different OSes:

• FreeBSD (v0.25)
• Fedora 26 (v0.24.2)
• Ubuntu trusty 14.04 (v0.17)
• Ubuntu xenial 16.04 (v0.21)

For the full range of versions we want to support, we should build that version of notmuch,
then build and test hs-notmuch against it.

see notmuch-search-terms(7).

Add a (allowed to failed) build to build matrix that checks out tip of notmuch source
and tries to build against that. This will provide early warning of breaking changes.

When I retrieve the values of a message header, the type is String, but I guess what we want is either Text or ByteString?

String (or Text) is the correct type for tags (e.g. spaces are allowed) but they must be
properly encoded in the query string or notmuch will interpret it as multiple subterms.

The correct encoding is to utf-8 encode, then hex-encode all non-printable characters,
as well as SPACE and DOUBLEQUOTE.

Not quite sure why this situation arises, but my tests are failling like so:

  user acceptance tests                                                                                  
    user can view mail:                                       OK (2.04s)                                 
      shows tag                                     
      view mail                                     
    manipulating notmuch search query results in empty index: FAIL (6.05s)                               
      focus command                                 
        Wait time exceeded. Expected: '37;40mtag' last screen shot:                                      
                                                    
          <[email protected] 17/Aug  Testmail                                          inbox                
         <[email protected] 16/Aug  This is Purebred                          inbox replied                 
                                                    
                                                    
                                                    
                                                    
                                                    
                                                    
                                                    
                                                    
                                                    
                                                    
                                                    
                                                    
                                                    
                                                                                                         
                                                    
                                                    
                                                    
                                                    
                                                    
        Purebred: Item 0 of 2                       
        tag:inbox                                   
                                                    
                                                    
         raw: "\ESC[1m\ESC[37m\ESC[43m <[email protected] 17/Aug  Testmail                                          \ESC[0m\ESC[36m\ESC[43minbox\n\ESC[94m\ESC[40m <[email protected] 16/Aug  This is Purebred           
               \ESC[36minbox replied\n\ESC[94m\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\ESC[30m\ESC[mPurebred: Item 0 of 2\n\ESC[37m\ESC[40mtag:inbox\n"

Above you will see that I added a raw print of the string captured from tmux, to
inspect the raw escape codes.

As you can see, before the NM search widget there is

\ESC[37m\ESC[40m

instead of what the search is looking for:

\ESC[37;40m

Semantically these are the same, but as I said, for whatever reason (maybe some weird
terminfo or environment settings) the codes being sent are different.

As a first step to resolving this, I'd propose exfiltrating $TERM or other terminal information
from the tmux session and reporting that somehow, to see what might differ. In my environment
TERM=screen-256color

Some term values require quoting, including free text and subject.

I think I've introduced a regression with adding support for threads. When the query yields a lot of search results, I'm running into a SIGABRT with purebred. I don't have a reproducer yet, only data to offer.

Compiled against: b6dd230

(gdb) backtrace
#0  0x00007ffff65bf69b in raise () at /lib64/libc.so.6
#1  0x00007ffff65c13b1 in abort () at /lib64/libc.so.6
#2  0x00007ffff799858c in  () at /lib64/libtalloc.so.2
#3  0x00007ffff7999ec9 in talloc_named_const () at /lib64/libtalloc.so.2
#4  0x00007ffff7bc7dfd in _notmuch_thread_create () at /lib64/libnotmuch.so.5
#5  0x00007ffff7bc4d89 in notmuch_threads_get () at /lib64/libnotmuch.so.5
#6  0x00000000004bcaa2 in notmuchzm0zi1zi0zi0zmCHTTkRw0ag8J6oXLxvNsa1_NotmuchziBinding_queryzusearchzuthreads2_info ()
#7  0x0000000000000000 in  ()

Implement a way to open a database in read/write mode e.g. for writing tags.

The API should make it easy to do this transiently, automatically closing when the operation
is complete.