Slow on very big file about gitsigns.nvim HOT 14 CLOSED

I was also going to create same issue, but I'll second this and slightly elaborate. It is slow not only when opening a file, but also when editing it.

My initial use case was 583000 line .csv file, but also tested on 99000 line. When making any change to a file (delete line, insert single symbol in insert mode, etc.) Neovim freezes for noticeable amount of time (around half a second with 99000 lines) and then slows down (for around extra second) until git signs show up.

Also tested the same operations on the same file with 'airblade/vim-gitgutter'. There is no noticeable lag when opening the file and no lag when editing. This seems to be due to combination of its g:gitgutter_async (when turned off, slowdown is comparable to 'gitsigns') and its usage of 'updatetime' option to wait until starting sign update.

from gitsigns.nvim.

I've just opened #67 which generally (and significantly) improves performance on large files. It uses an unstable API so is disabled by default. Can you guys give it a go?

from gitsigns.nvim.

Untracked

One thing gitsigns does differently to gitgutter (I believe) is that it will attach to untracked files if there are not ignored. This will be the reason gitgutter runs ok; because it isn't running at all. We may want to add some config around this so untracked files are never attached to. We could also have a different line threshold for tracked/untracked files because if a file is tracked you are more likely want to have signs applied.

Drive-by comment: I was surprised by this change as well. I can see the rationale for this behavior, although I'm not sure how useful it really is (until you add the file, won't all lines always be treated as added, no matter what you change)? So an option to ignore untracked files would be great, even if it defaults to false!

from gitsigns.nvim.

Forgot to say that if I was fast at going to the end of the file with when opening my file with 99000 lines, I got an error saying: ...

Thanks. That branch is still pretty rough. It's mostly just a proof-of-concept at the moment. Still mention if you see any weird behaviour though!

Drive-by comment: I was surprised by this change as well. I can see the rationale for this behavior, although I'm not sure how useful it really is (until you add the file, won't all lines always be treated as added, no matter what you change)? So an option to ignore untracked files would be great, even if it defaults to false!

I personally find it useful as it allows me to open new files (vim foo), add some text, then add the entire file to the index using my stage_hunk mapping. Since the entire file is just one hunk it effectively just does a git add. And this works without even needing to save the buffer.

So an option to ignore untracked files would be great, even if it defaults to false!

Noted 👍

from gitsigns.nvim.

I added 093da28 to workaround the issue (default max file length is 40000 lines), however I still think it's worth investigating some of the performance concerns mentioned here.

Points to note:

• Gitsigns applies for the entire buffer on every update. This means for untracked files it will loop though every line to add an added sign. If this is really a problem we can think about only adding signs to the visible lines in the viewport though this will be complicated to implement with a higher risk of bugs.
• Gitsigns is asynchronous with work split between the main thread and uv.spawn callback threads. It could be that there is some heavy work being done in the main thread slowing things down. Worth investigating...
• Gitsigns updates are debounced, similar to how updatetime is utilised. This value is currently hardcoded to 100ms.

@seblj, @echasnovski: Are any of the files you are having issues on public repositories? If so could you point me?

from gitsigns.nvim.

Thank you! My file is not on any public repository, and it was only a file with some numbers I used for plotting something

from gitsigns.nvim.

Another question, are these large files managed in source control? If not are they in .gitignore?

from gitsigns.nvim.

In my case, it was not in .gitignore. I created the file some time ago and never added and committed it either. I was just cleaning up my folder structure when I noticed it.

from gitsigns.nvim.

No, sorry, that file is not public (work related). It was tracked (commited at some point in the past and even pushed to remote repository), so no added sign on every line. For example, after I had added blank line, Neovim froze for some time, added added sign and immediately unfroze. And it certainly didn't feel like gitsigns treats updatetime the same as vim-gitgutter (although, I have updatetime = 300).

@lewis6991, you don't have the same slowdown on some manually created big file?

from gitsigns.nvim.

Both the tracked and untracked cases are quite different from one-another and are handled are handled by gitsigns very differently.

Untracked

config.max_file_length will mostly solve this case however there is probably more we can still do.

In my case, it was not in .gitignore. I created the file some time ago and never added and committed it either. I was just cleaning up my folder structure when I noticed it.

One thing gitsigns does differently to gitgutter (I believe) is that it will attach to untracked files if there are not ignored. This will be the reason gitgutter runs ok; because it isn't running at all. We may want to add some config around this so untracked files are never attached to. We could also have a different line threshold for tracked/untracked files because if a file is tracked you are more likely want to have signs applied.

Tracked

And it certainly didn't feel like gitsigns treats updatetime the same as vim-gitgutter (although, I have updatetime = 300).

Well it won't treat it exactly the same; they are both different plugins implemented in different languages and API's. However the intention is mostly the same. I think gitsigns is a more optimistic about when to apply signs and gitgutter might have a bit more logic to handle things more incrementally. I'll take a look at what gitgutter does in more detail and see if there is any low hanging fruit.

@lewis6991, you don't have the same slowdown on some manually created big file?

I don't generally store very large files in source control and pretty much always add them to my gitignore. There are much better systems for handling large files than git. But I don't want that to deter from the fact that gitsigns should be handling large files gracefully as best it can.

The fact you are seeing slowdowns on tracked files gives me a better idea on where it is happening. It's likely not going to be the application of signs, because since it's tracked it wouldn't have many to place. So the issue is probably due to the hunk processing, or maybe even the jobs themselves as I imagine running git diff on a large file to be slow.

from gitsigns.nvim.

For the untracked case, I can confirm it is the sign application which is causing the slowdown.

For the tracked case I don't see the slowdown. I created a file with:

yes "Some text" | head -n 100000 > large-file

And added it to the index:

git add large-file

And for me gitsigns works ok when opening and editing. My guess is the performance will scale with the amount of signs that are being place/unplaced. For this case very few/no signs are being placed.

EDIT: Increasing lines to 600000 introduces a slowdown for me.

from gitsigns.nvim.

It seems to be significantly faster to add the signs from my testing just now. I updated the maximum number of lines for the signs to appear, and tried to enable config.use_decoration_provider. It also seems to lets me navigate in normal mode without any interruption both before and after loading the signs. I did however notice some lag when adding new lines and typing, but this is also a massive improvement compared to without config.use_decoration_provider enabled.

The final thing I noticed, was that is seems to be a bit slower to exit neovim, and hangs before the shell prompt is loaded. This did however only happen a few times, so this could be a coincidence. But I couldn't reproduce the lag when quitting without loading the signs.

EDIT:
Forgot to say that if I was fast at going to the end of the file with when opening my file with 99000 lines, I got an error saying:
gitsigns: Error executing lua: ...nvim/site/pack/packer/opt/gitsigns.nvim/lua/gitsigns.lua:570: attempt to index field 'signs' (a nil value)

from gitsigns.nvim.

I personally find it useful as it allows me to open new files (vim foo), add some text, then add the entire file to the index using my stage_hunk mapping. Since the entire file is just one hunk it effectively just does a git add. And this works without even needing to save the buffer.

Ah, good point; I have to shamefully admit I've never used the "advanced" mappings and just treated gitsigns as a gitgutter replacement... Then this makes sense, of course.

from gitsigns.nvim.

Closing with a98df15

from gitsigns.nvim.