Backslash, then Ctrl-B produces unexpected results in Emacs mode about ksh HOT 30 CLOSED

I'm just a ksh user, but here's the page.

Learning the Korn Shell, 2nd ed dances around the same issue beginning on page 27.

from ksh.

I noticed that the behavior works with other characters other then Ctrl-B. For instance backslash then Ctrl-D.

yes. as already stated in my first reply: "\ Escapes next character". any character that is. those with no special meaning. those special to the shell. those special to the editor. other control characters.

Some questions:
Should Ctrl-B be removed from the possible control characters that can be entered after a backslash.? Potentially only in interactive mode. Thereby giving precedence to the Emacs editing of the buffer? Unsure if the book describes precedence of input characters in their interactive editing mode.

I would say "no" since one cannot predict whether overall users prefer the emacs "meaning" of ^B (users of vi mode obviously don't...) or want consistent means to input all control characters.

and, more important, \ escapes all characters. it would seem totally wrong to exclude one or two of those (^B,^F) from this treatment "just" because they happen to have a meaning to the chosen editor mode.

I understand that the behaviour (can't backspace with ^B over \ immediately after typing it) irritated you but it seems tolerable to now remember it and use the left arrow key for a change if the situation happens again (or switch to vi mode ;)).

Can we conclude that typing backslash then a control character is a feature based on p106 of the book David Korn authored?

I believe the manpage together with the book is the de facto "specification" of the KornLanguage. in this special case, as @marcastel already pointed out in this thread, it is explained not only in the book but also in the manpage in identical words. so it is not a random thing. it is fully intentional and defined behaviour.

in view of all this, my proposal would be to close this issue, right?

Thanks

from ksh.

maybe this difference is also documented and thus a 'feature' (did not check), but probably not. in which case this is simply inconsistent behaviour and it would be desirable to change it in either vi or emacs mode.

From what I can see this is not documented and might be a bug indeed.

Looks like a no-editor mode (neither vi not emacs) does behave the same as vi. I prefer emacs behavior. But we should not break the functionality on totally dumb terminals.

from ksh.

Also found backspacing mode in vi which makes me stuck forever behind the backslash:

• Start editing in vi mode
• After backslash type something then ESC to switch to command mode
• Use shift-T, backslash command to get behind the backslash
• Use shift-C to replace the text till the end of line
• Type something in the input mode
• Backspace all out - you’ll be stuck “forever” behind the backslash

Note: in the vi’s change command it is not allowed to backspace to the left where one has started

Weird, but consistent with how the change command behaves...

from ksh.

It's worth pointing out here that ksh's "raw" POSIX (neither vi nor emacs) editing mode is implemented in two different ways, depending on whether a UTF-8/multibyte locale is active or not. By default, raw POSIX editing mode is handled in edit/edit.c. But in UTF-8 locales, control is passed to edit/vi.c.

And vi.c contains a number of conditionals that are supposed to disable vi features if the -o vi shell option is not set. Unfortunately that is very buggy, so you get e.g. extremely broken tab completion (which is already fixed in 93u+m).

This also explains the bugs @saper made videos of (see above); evidently, a UTF-8 locale was active. If you set LANG=C or LC_ALL=C you'll see those bugs disappear.

from ksh.

@nickpapadonis There are some bits missing in the way you describe how to reproduce the issue, probably something got cut out by the markdown editor

from ksh.

Fixed the comment thanks

from ksh.

I am using vi mode so never noticed this behaviour. I now have looked it up in the bolsky/korn book (@nickpapadonis: if you like and care for ksh I can really recommend that you get a hardcopy (long out of print, used samples you can get for 5$ or so from abebooks/amazon)) and it proofs that the behaviour you see is explicitly intended. from page 106:

\ Escapes next character. Editiing characters and the user's Erase, Kill, and interrupt characters may be entered in a command line or search string if preceded by \. The \ removes the next character's editing features, if any.

this seems clear enough to me. I disagree with krader's "Having said all that I agree with you the behavior is confusing and should be changed regardless of how many years it has been in effect."

while it is correct, that control chars like ^B might also be inserted via ^V^B this convenience feature for emacs mode users has to be kept in place and the issue then could be closed as "not-a-bug" I believe. @dannyweldon @marcastel (and everybodye;)): what do you think?

from ksh.

This is also documented with the same wording in the man page (src/cmd/ksh93/sh.1):

\
Escape next character. Editing characters, the user's erase, kill and interrupt (normally ^?) characters may be entered in a command line or in a search string if preceded by a . The \ removes the next character's editing features (if any).

I concur with @jghub in that this appears compliant. I am not an emacs user either.

from ksh.

It also works this way in the vi mode as well without any editing mode at all.

There is one problem though - the documentation says this is a "command", so I'd expect that typing `` (backslash) and then some character quotes it. So far so fine.

However, it seems to work like a permanent marker though: just type backslash, a, b to get \ab. Then press backspace once (\a), again (\) and again (^? in my case - my erase character).

Is the "command" something that is there permanently in the line buffer or is this just a keystroke pressed?

from ksh.

oh, you are right regarding also happens in vi mode! it never happened to me since I have added emacs-like cursor movement keybindings to my vi mode so that I can use ^B and ^F from within vi insert mode. and in this combo \^B actually just backspaces :).

regarding your question: it seems to come about because you never leave input mode (in vi, I mean, in emacs no difference...) so you delete the b than the a. at this point your whole input is \ and the next keystroke looses its special meaning -- including del (ascii octal 177). and when you hit del the \del combination inserts the \0177 char. seems consistent, no?

from ksh.

so I have learned something new. I never was aware of \ doing more than protecting symbols special to the shell. I completely missed the line editor support to input control chars in this way...

from ksh.

I update the issue as some characters were lost. There are many comments above. I'm confused, is this behavior a feature? What does the feature enable the user to do?

from ksh.

yes, it is a "feature" since explained in the book and the manpage. it can also be confusing, obviously, since the two builtin editors (vi and emacs) in ksh thus treat \ specially, not as an ordinary character.

what it allows, is to easily enter verbatim those control-chars into string definitions. if you type \^B it is interpreted as "insert ascii character with octal code 002 (stx)" (i.e. the control-B character) instead of "move cursor one character to the left". it just is not what you expected to happen ...

from ksh.

What page is this described in the book? I have it. Thanks

from ksh.

p. 106

from ksh.

at this point your whole input is \ and the next keystroke looses its special meaning

I have mistakenly used the word "command" which means something different in the Korn shell. This is an "edit command". "All edit commands operate from any place on the line (not just at the beginning)". So in that way '' is just a keystroke (like ^A moving the cursor) but it leaves some visual indication the next character is about to quoted.

An '' already entered in the buffer is a different thing in my opinion. It just sits there.

Regarding ^V - ksh uses this as "Display version of the shell", it works in the emacs/gmacs mode as well as in the command mode of vi mode. However, it is also a default value of the lnext capability of the terminal discipline - this means the next character (keystroke!) is about to be entered literally.

This can be changed or turned off using stty lnext ..., at least on FreeBSD, this leaves `' edit command as the only way to enter them.

from ksh.

An '' already entered in the buffer is a different thing in my opinion. It just sits there.

I see your point but I am not averse to the present behaviour (which also might be justified). but I only now have noted that the two editors behave different in this situation. vi does what you describe (and what is ok behaviour with me) while emacs does what you seem to prefer in this situation (just treat \ as ordinary character in the buffer when deleting)

maybe this difference is also documented and thus a 'feature' (did not check), but probably not. in which case this is simply inconsistent behaviour and it would be desirable to change it in either vi or emacs mode.

from ksh.

stty lnext ...

Did not resolve the issue in Debian 10

Did not resolve the issue in FreeBSD 12.1

Did I enter the correct command?

I am still unclear on how this feature is useful. What does it provide? I.e. typing \ (back slash) then Ctrl-B shows ^B. Is the intension to type \ (back slash), then some control characters?

from ksh.

stty lnext is not a solution. I was referring to this:

You can always type [ctrl-V][ctrl-B] to insert a literal [ctrl-B].

[ctrl-v] is not a feature of ksh, it is a feature of a so-called "line discipline" that controls the terminal behaviour in UNIX. It can be turned off or assigned to a different key with stty lnext.

To answer your question:

The way I understand this is the intention - to add control characters that would otherwise do something else, similar to Ctrl-V with default setting of most terminals.

from ksh.

I noticed that the behavior works with other characters other then Ctrl-B. For instance backslash then Ctrl-D.

Some questions:
Should Ctrl-B be removed from the possible control characters that can be entered after a backslash.? Potentially only in interactive mode. Thereby giving precedence to the Emacs editing of the buffer? Unsure if the book describes precedence of input characters in their interactive editing mode.

Can we conclude that typing backslash then a control character is a feature based on p106 of the book David Korn authored?

Thanks

from ksh.

If one types backslash and Ctrl-C, Ctrl-C is not output to the screen with a carrot in front of it. In this case Ctrl-C is captured for it's special meaning. If one types backslash and Ctrl-D, Ctrl-D is output to the screen and it is not captured for it's special meaning. In this sense, I would argue that the behavior is non-uniform for at least Ctrl-C. Does this make sense?

from ksh.

Can someone add the lines from Korn's book p106 that justify this behavior? I have my book in storage. Thanks

from ksh.

Wow there it is. I agree we can close this issue out as expected behavior. Thanks for all the input!

from ksh.

oh, you are right regarding also happens in vi mode! it never happened to me since I have added emacs-like cursor movement keybindings to my vi mode so that I can use ^B and ^F from within vi insert mode. and in this combo \^B actually just backspaces :).

Sorry to comment on a closed issue, but is there any chance you could share those keybindings? Maybe start a wiki page in this project? There's a general lack of information around the web for the esoteric way to get KSH to interpret key strokes.

from ksh.

@dwierenga Excellent idea. Could I kindly ask you to open an issue here and we will follow-up.

We are working on setting up centralised information and documentation on the KornShell and this definitively belongs there :-)

from ksh.

from ksh.

from ksh.

from ksh.

What is shown in this comment is a bug: #8 (comment)

What is shown in this comment is part intended, part bug: #8 (comment)
The ^? is a bug, but it's not a bug that Shift+C puts you in a separate buffer that you can't backslash past until you escape out of it again; that is standard vi behaviour (though vim acts differently).

from ksh.