Wrong tabstop position with non-latin characters about ultisnips HOT 17 OPEN

I cannot reproduce this. I copy and pasted your snippet, unintended it, so that ,{ is starting at the begging of the line. In an empty file I type:

sehello worldblub

output is as expected:

------------------- SNIP -------------------
,{
'AUTHOR': 'Williams',
'TEXT': 'hello world',
}blub
------------------- SNAP -------------------

Launchpad Details: #LPC SirVer - 2013-12-01 14:42:58 +0100

from ultisnips.

what version of ultisnips are you using (and from where)? Which version of Vim and which OS are you using? Also, gvim or vim on console. Please try to reproduce the bug in a simple fashion and describe your steps similar to what I just did.

Launchpad Details: #LPC SirVer - 2013-12-01 14:44:01 +0100

from ultisnips.

I use vim v7.4 (not gvim) on console. My OS is Ubuntu 13.10 64-bit. Ultisnips v2.2 (according to ChangeLog).

------------------- SNIP -------------------
$ vim --version
VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Aug 12 2013 00:23:19)
Modified by
Compiled by buildd@
Huge version without GUI. Features included (+) or not (-):
+arabic +file_in_path +mouse_sgr +tag_binary
+autocmd +find_in_path -mouse_sysmouse +tag_old_static
-balloon_eval +float +mouse_urxvt -tag_any_white
-browse +folding +mouse_xterm -tcl
++builtin_terms -footer +multi_byte +terminfo
+byte_offset +fork() +multi_lang +termresponse
+cindent +gettext -mzscheme +textobjects
-clientserver -hangul_input +netbeans_intg +title
-clipboard +iconv +path_extra -toolbar
+cmdline_compl +insert_expand -perl +user_commands
+cmdline_hist +jumplist +persistent_undo +vertsplit
+cmdline_info +keymap +postscript +virtualedit
+comments +langmap +printer +visual
+conceal +libcall +profile +visualextra
+cryptv +linebreak +python +viminfo
+cscope +lispindent -python3 +vreplace
+cursorbind +listcmds +quickfix +wildignore
+cursorshape +localmap +reltime +wildmenu
+dialog_con -lua +rightleft +windows
+diff +menu -ruby +writebackup
+digraphs +mksession +scrollbind -X11
-dnd +modify_fname +signs -xfontset
-ebcdic +mouse +smartindent -xim
+emacs_tags -mouseshape -sniff -xsmp
+eval +mouse_dec +startuptime -xterm_clipboard
+ex_extra +mouse_gpm +statusline -xterm_save
+extra_search -mouse_jsbterm -sun_workshop
+farsi +mouse_netterm +syntax
system vimrc file: "$VIM/vimrc"
user vimrc file: "$HOME/.vimrc"
2nd user vimrc file: "~/.vim/vimrc"
user exrc file: "$HOME/.exrc"
fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: gcc -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,--as-needed -o vim -lm -ltinfo -lnsl -lselinux -lacl -lattr -lgpm -ldl -L/usr/lib/python2.7/config-x86_64-linux-gnu -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions
------------------- SNAP -------------------

------------------- SNIP -------------------
$ uname -a
Linux insula 3.11.0-13-generic #20-Ubuntu SMP Wed Oct 23 07:38:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
------------------- SNAP -------------------

Launchpad Details: #LPC Andrei - 2013-12-01 17:43:55 +0100

from ultisnips.

From where did you install this version of UltiSnips?

Launchpad Details: #LPC SirVer - 2013-12-01 18:20:30 +0100

from ultisnips.

I used Vundle to install and update it from github.

Launchpad Details: #LPC Andrei - 2013-12-01 19:05:26 +0100

from ultisnips.

Where on github - there are 192 forks :)

Launchpad Details: #LPC SirVer - 2013-12-01 19:17:47 +0100

from ultisnips.

https://github.com/SirVer/ultisnips

I've tried to minimize the number of plugins and reduced my .vimrc to the following (see below), but the problem still persists. The exact look of the snippet is the following:

------------------- SNIP -------------------
snippet sse "" b
,{
'AUTHOR': '${1:Williams}',
'TEXT': '${VISUAL}$2',
}
endsnippet
------------------- SNAP -------------------

I select a sentence with <vis>, than <tab>sse<tab><c-j>hello world<c-j>
and the 3rd line of my snippet looks like that:

'TEXT': ''hello world,

My reduced .vimrc:
------------------- SNIP -------------------
set nocompatible
set encoding=UTF-8

" VUNDLE
filetype off
set rtp+=~/.vim/bundle/vundle/
call vundle#rc()

Bundle 'gmarik/vundle'
Bundle 'SirVer/ultisnips'

filetype plugin indent on
" VUNDLE END
------------------- SNAP -------------------

Launchpad Details: #LPC Andrei - 2013-12-01 19:46:25 +0100

from ultisnips.

I still cannot reproduce the problem. With the same vimrc everything works nicely for me. Can you try doing a screencapture of your vimsession or something? Maybe it has to do with the sentence you copy in your visual selection.

Launchpad Details: #LPC SirVer - 2013-12-02 06:43:18 +0100

from ultisnips.

I suppose you are right. My sentences are in cyrillic script. If they are in latin, everything works pretty fine. You can try this one to reproduce the bug:

------------------- SNIP -------------------
Так в доме кроме кота и ежа стала жить третья живая душа — Желтухин.
------------------- SNAP -------------------

And here is a video record with reproducing the bug: http://www.youtube.com/watch?v=YU5FdfxUTgA
Thank you very much for Ultisnips and for being responsive and cooperative.

Launchpad Details: #LPC Andrei - 2013-12-02 09:47:38 +0100

from ultisnips.

May be it is a bug related to the variable width of a unicode character in bytes in UTF-8?

Launchpad Details: #LPC Andrei - 2013-12-02 11:12:06 +0100

from ultisnips.

Yep, that seems to be the issue. I can reproduce the problem using the cyrillic script and I guess that you are right about the width of unicode characters. Theoretically, UltiSnips should deal fine with that (there is a bunch of code that deals with the details and ugliness of Vims internal encoding stuff), but I only speak latin based languages, so there was not enough testing apparently.

If you have the option, you can try a Vim that supports python3 - there the situation is less difficult and it could maybe work. Otherwise this bug needs careful inspection to get fixed.

Launchpad Details: #LPC SirVer - 2013-12-03 07:21:46 +0100

from ultisnips.

This problem also occur if there is non-latin character in your snippet.
Using the example above, if I define a snippet like:

snippet sse "" b
{
'AUTHOR': '${1:Williams}',
'TEXT': 'Так в доме$2',
}

when you jump forward to $2, the position will also be wrong position. So it's nothing to do with VISUAL mode but the non-latin words causes ultisnips getting wrong on tabstop.
I found this error is very very very strange one:

• The error only happen if $1 has default text (williams in this case), if $1 has no default text, the position of $2 tabstop will be correct.
• The error only happen with $2, if replace $2 with $0, the error not happen.
• The most strange one: the following snippet doesn't have this problem.

snippet sse "" b
,{
'AUTHOR': '${1:Williams}',
'TEXT': 'Так в доме$2',abcdefg
}
endsnippet

with 'a', 'ab' ... 'abcdef', the error occurs, but abcdefg or longer string eliminate this error.

from ultisnips.

currently track, both the correct and wrong snippet set _vim.select at same position (snippet_manager.py:392), ntab.start and ntab.end are identical in both cases.
So I think the problem may lies in _vim.select?

from ultisnips.

Can still reproduce in 324a4f8.

from ultisnips.

I though this might have been fixed with 9c73ce9, but I can still repro the error there.

from ultisnips.

Still exists in 943156d, both in python2 and python3.

from ultisnips.

The behavior is still buggy in 3a8b054, but much less so. I can only reproduce a off-by-one error with the ${VISUAL} example and no error using the example with non-ascii text in the snippet definition.

I was completely unable to trigger any bug by typing Так в доме кроме кота и ежа стала жить третья живая душа — Желтухин. in any placeholder position.

from ultisnips.