(M2 Pro) 2023-11-03 is so slow, not usable at all about neomutt HOT 22 OPEN

set mail_check = 0

That may be a problem.
It's telling NeoMutt to check for new mail every loop.
Every keypress is an event that is processed and the loop starts again.

Try setting that to set mail_check = 5 (to start with)

from neomutt.

If I understood correctly, it should look like

mailboxes "a" -nopoll "b" … 

It's a prefix option, so above the -nopoll affects "b" only.

from neomutt.

i'm not sure, i'm a linux user with ~0 mac experience.
anyway, you should probably start without '-s', but with '-e'.
attach the first log (don't paste it; it likely will be huge) and see what rich thinks of it.

from neomutt.

Hmm...

I routinely test with a 50K emails directory without any problems.

It typically takes (my cheap laptop):

• 6s cold
  • scan the directory
  • read all the header
  • cache the headers

Then, after:

• 3s warm
  • scan the directory
  • read the cached headers

Once started, navigating the Index and viewing emails is instantaneous.

from neomutt.

Please can you try NeoMutt without any config, on a large mailbox, e.g.

neomutt -n -F /dev/null -f ~/mailbox/big

How fast is:

• Navigation in the Index?
• Reading emails?
• Switching between emails?
• Sorting the Index between:
  • Date od
  • Thread ot

from neomutt.

Often, a new release of NeoMutt means that the header cache becomes invalid.
(The cache is tightly-bound to the structs in memory)

If you temporarily move the cache dir and let NeoMutt rebuild it, do you see the same slowdown?

from neomutt.

Please can you try NeoMutt without any config, on a large mailbox, e.g.

neomutt -n -F /dev/null -f ~/mailbox/big

When I run this command

neomutt -n -F /dev/null -f ~/.mail

I get this

/Users/ahmedelgabri/Mail does not exist. Create it? ([yes]/no):

Then quits with

Unable to open mailbox /Users/ahmedelgabri/.mail

If you temporarily move the cache dir and let NeoMutt rebuild it, do you see the same slowdown?

Didn't help

from neomutt.

Unable to open mailbox /Users/ahmedelgabri/.mail

You need to point it at a maildir mailbox, i.e. a directory that contains cur, new and tmp subdirs
(and hopefully LOTS of emails)

from neomutt.

I did both the root folder and my account folder. The result is the same.

For example if I run this where personal contains all my emails. It yields the same exact result.

neomutt -n -F /dev/null -f ~/.mail/personal 

from neomutt.

er... I don't understand.
Can you run that again with debugging, to see if you can find out why it's failing.

neomutt -n -F /dev/null -f ~/.mail/personal -d5

from neomutt.

/Users/ahmedelgabri/Mail does not exist. Create it? ([yes]/no):

That can be ignored

from neomutt.

er... I don't understand. Can you run that again with debugging, to see if you can find out why it's failing.

neomutt -n -F /dev/null -f ~/.mail/personal -d5

I made an embarrassing mistake 🤦🏼‍♂️, I passed personal instead of work because I'm debugging my work email, sorry.

This command works as expected and it's fast… without any configs. I wonder what could be the culprit here that leads to such massive performance degradation and only on M2?

from neomutt.

ah, no worries :-)

This command works as expected and it's fast… without any configs.

Great. That's easily debug-able

I wonder what could be the culprit here that leads to such massive performance degradation and only on M2?

No idea, so it's "bisect time"

Please can you comment out half of your config and see if that helps.
If not, then comment out the other half.
Repeat using smaller and smaller chunks.

Hopefully, that'll lead you to a line of config, or a particular NeoMutt command that's causing the trouble.
Then I can work out what's changed and fix it.

Thanks!

from neomutt.

Wild guess, because this is sometimes the reason for NeoMutt to be perceived as "slow": what is the value of sleep_time in your config?

Does setting sleep_time = 0 help?

from neomutt.

I found the issue, I didn't need to debug too much. I had a hunch because this gave me issues before too. It was mailboxes

So I have this in my config

mailboxes "+Work/INBOX" "+Work/Starred" \
  "+Work/Sent" \
  "+Work/Drafts" \
  "+Work/Trash" \
  "+Work/Spam" \
  `tree ~/.mail/Work -l -d -I "Archive|cur|new|tmp|certs|.notmuch|INBOX|[Gmail]" -afinQ --noreport | awk '{if(NR>1)print}' | tr '\n' ' '`

If I just comment the last line

`tree ~/.mail/Work -l -d -I "Archive|cur|new|tmp|certs|.notmuch|INBOX|[Gmail]" -afinQ --noreport | awk '{if(NR>1)print}' | tr '\n' ' '`

It's very fast again. But the question now is what changed between versions to make this slow and why it's only slow on M1 while the Intel machine which is older is working fine.

from neomutt.

tree ...

How many mailboxes are we talking? 10s, 100s, 1000s?

What are you settings for the following config?

neomutt -Q check_new -Q mail_check -Q mail_check_recent -Q mail_check_stats -Q mail_check_stats_interval -Q maildir_check_cur

from neomutt.

How many mailboxes are we talking? 10s, 100s, 1000s?

21

$ tree ~/.mail/Work -l -d -I "Archive|cur|new|tmp|certs|.notmuch|INBOX|[Gmail]" -afinQ --noreport | awk '{if(NR>1)print}' | wc -l
21

What are you settings for the following config?

neomutt -Q check_new -Q mail_check -Q mail_check_recent -Q mail_check_stats -Q mail_check_stats_interval -Q maildir_check_cur

set check_new = yes
set mail_check = 0
set mail_check_recent = yes
set mail_check_stats = yes
set mail_check_stats_interval = 1
set maildir_check_cur = no

I think I have set those years back and didn't revise them.

from neomutt.

set mail_check = 0

That may be a problem. It's telling NeoMutt to check for new mail every loop. Every keypress is an event that is processed and the loop starts again.

Try setting that to set mail_check = 5 (to start with)

I'm going to set it to 5 anyway because 0 doesn't make sense but it doesn't help with the main issue, the only thing that helps is removing the line where it sets mailboxes dynamically.

from neomutt.

... removing the line where it sets mailboxes dynamically

We can rule this out, if you evaluate this and replace it with the static mailboxes "a" "b" ...
(temporarily)

That leaves:

~9GB/~213k emails

combined with:

set mail_check_stats = yes
set mail_check_stats_interval = 1

That's causing NeoMutt to have to scan some large directories frequently.
Even with everything in the filesystem cache, that takes time.
If it takes >1s, then I can see how things could get clogged up.

Increasing the interval should help.
Also, if you have mailboxes that don't receive new mail, you could add -nopoll to the mailboxes command.

from neomutt.

... removing the line where it sets mailboxes dynamically

We can rule this out if you evaluate this and replace it with the static mailboxes "a" "b" ... (temporarily)

So I found one mailbox that when enabled (with the rest) the slow down happens, but when it's added alone it works fine. It's also interesting because that mailbox is nearly dead, I haven't received any emails there for maybe over a year. And it's not even the biggest one (both email count & size on disk)

set mail_check_stats = yes
set mail_check_stats_interval = 1

That's causing NeoMutt to have to scan some large directories frequently. Even with everything in the filesystem cache, that takes time. If it takes >1s, then I can see how things could get clogged up.

Updated this to 10 but still didn't help much.

Increasing the interval should help.
Also, if you have mailboxes that don't receive new mail, you could add -nopoll to the mailboxes command.

If I understood correctly, it should look like

mailboxes "a" -nopoll "b" … 

If yes, I also did this but didn't work, if not I can try again when I know the correct format.

from neomutt.

i'd dtruss the thing, and/or run it in gdb and interrupt it during the delay - multiple times, and compare the backtraces. even just looking at top in parallel should give a first clue as to where time is spent.

from neomutt.

i'd dtruss the thing, and/or run it in gdb and interrupt it during the delay - multiple times, and compare the backtraces. even just looking at top in parallel should give a first clue as to where time is spent.

@ossilator didn't use dtruss before, read the man page and it seems that this is what I need, correct?

dtruss -fasp <neomutt PID> 

from neomutt.