scan_folded_as_literal not behaving plausible with trailing whitespace about yamlfmt HOT 4 OPEN

Thanks for opening an issue. This looks like it's probably caused by the same thing as I've been tracking in #86, which I haven't been able to resolve yet due to its complexity.

from yamlfmt.

This is not limited to trailing whitespaces in the last line of a folded string:

foo: >
  lorem 
  ipsum

is being formated to

foo: "lorem \nipsum\n"

from yamlfmt.

Can also confirmed this is happening with normal block scalars.

Before

block: |
  key: value (single whitespace appended)

After

block: "key: value \n"

Expected

block: |
  key: value (whitespace removed)

If there is no trailing whitespace, everything words as expected. As per @bewee-i any line with a trailing whitespace will cause this occur, it is not limited to the last line.

from yamlfmt.

Thank you for the additional example.

On the root issue #86 I added an explanation for why this bug occurs, but I'll copy that explanation here for visibility:

This is caused by the hack around the fact that yaml.v3 doesn't retain plain line-break information. yamlfmt will insert a magic string before being serialized into yaml.v3's node structure. Then after the new output is produced, we use the magic strings to put them back in. However, one place that newline information will actually be retained is in block scalars. So the magic line string being thrown on there messes with the serialization. I can't think of a way around this without somehow getting rid of the hack. I've tried in the past to fix this in yaml.v3 but I came up short. I've been wanting to build my own yaml parser instead to rid myself of yaml.v3 in general, and in that case this and many other fixes/features I've wanted to implement would be possible. Haven't had the time to invest to make that happen though.

from yamlfmt.